Patent Grant
9354809
The present invention mainly relates to a display device that displays information obtained from sensors.
Conventionally, display devices which are connected with two or more kinds of sensors and displays information obtained from these sensors as a sensor image have been known. This kind of display device may adopt a configuration in which a display screen is divided so as to simultaneously display a plurality of sensor images. Patent Documents 1 and 2 disclose this kind of display device.
The display devices of Patent Documents 1 and 2 are carried in a ship, acquires positional information from a GPS antenna and creates an image showing, for example, a nautical chart around the ship, while they acquires a measurement result from a depth sounder and creates an image showing an underwater state. The display devices can simultaneously display the created two images so that the display screen is divided.
In the meantime, when dividing the display screen to display the image as described above, the feeling of operation on the image may not be unified between the images displayed. In this case, a user needs to study operation methods for every image, and will get confused.
Especially, in the display device which adopts a touch panel, instinctive operations are often required, and if the feeling of operation is not unified between the images, much more confusion will be caused.
The present invention is made in view of the above situation, and one object of the invention is to provide, in a display device that can divide a display screen so as to display information obtained from two or more kinds of sensors, and can detect a touch operation, a configuration for performing processing, such as scrolling, taking characteristics of each image into consideration.
Problems to be solved by the invention is described above, and means for solving the problems and effects thereof will be described below.
According to a first aspect of the invention, a display device having the following configuration is provided. Thus, the display device includes a detector, an image acquirer, a display unit, a gesture detector, an operated location detector, a divided area identifier, and an image processor. The detector detects a touch operation. The image acquirer acquires at least two of a navigation image containing map or nautical chart information created based on information obtained from a GNSS sensor for detecting a location of the display device, an underwater detection image created based on information obtained from an ultrasonic transducer for detecting underwater directly under the display device, and a radar image obtained from a radar antenna for detecting a location of an object around the display device. The display unit displays acquired images in respective divided areas of a display screen. The gesture detector detects a predetermined touch gesture based on the touch operation detected by the detector. The operated location detector detects an operated location of the touch gesture. The divided area identifier identifies which image among the images is displayed in the divided area corresponding to the operated location detected by the operated location detector. The image processor is provided with a function to perform scrolling or zooming of the display screen corresponding to the touch gesture, and changes a direction or a motion range according to the image being displayed in the identified divided area, upon performing the scrolling or zooming.
Thus, it can apply a certain limitation to one of the plurality of images (inhibiting the scrolling in a predetermined direction, limiting the motion range, etc.) against the scrolling or zooming processing, while preventing similar limitations from being applied to other images. Therefore, the scrolling or zooming processing can be performed in a mode according to characteristics of the image.
In the above display device, it is preferred to have the following configurations. Thus, the image processor does not fix the direction of scrolling or zooming when the image in the identified divided area is the navigation image. When the image in the identified divided area is the underwater detection image, the direction of scrolling or zooming is fixed.
Thus, it can apply a certain limitation (for example, inhibiting the scrolling in a depth direction) to the underwater detection image, while preventing similar limitations from being applied to the navigation image.
In the above display device, it is preferred to have the following configurations. Thus, the image processor does not limit the motion range of scrolling or zooming when the image in the identified divided area is the navigation image. When the image in the identified divided area is the radar image, the motion range of scrolling or zooming is limited.
Thus, it can apply a certain limitation (for example, limiting the motion range to a predetermined distance from a ship concerned) to the radar image, while preventing similar limitations from being applied to the navigation image.
In the above display device, it is preferred to have the following configurations. Thus, the image processor fixes the direction of scrolling or zooming when the image in the identified divided area is the underwater detection image. When the image in the identified divided area is the radar image, the direction of scrolling or zooming is not fixed.
Thus, it can apply a certain limitation (for example, inhibiting the scrolling in a depth direction) to the underwater detection image, while preventing similar limitations from being applied to the radar image.
In the above display device, it is preferred that the image processor limits the motion range of scrolling or zooming centering on the location of the display device when the image in the identified divided area is the radar image.
Thus, in a radar image, it can prevent that the location of the display device deviates from the radar image.
In the above display device, it is preferred to have the following configurations. Thus, the underwater detection image is an image where an underwater state directly under the display device is associated with time. The image processor fixes the direction of scrolling in a time-axis direction and fixes the direction of zooming in a direction perpendicular to the time-axis, when the image in the identified divided area is the underwater detection image.
Thus, the scrolling and zooming can be performed only in the desirable direction when verifying the underwater detection image.
In the above display device, it is preferred to be carried in a ship.
Thus, the effects of the present invention can be demonstrated in the sensor images obtained from sensors as ship instrument.
According to a second aspect of the invention, a display device of the following configurations is provided. Thus, the display device includes a detector, an image acquirer, a display unit, a gesture detector, an operated location detector, a divided area identifier, and an image processor. The detector detects a touch operation. The image acquirer acquires at least two or more images generated based on signals acquired by different sensors. The display unit displays acquired images in respective divided areas of a display screen. The gesture detector detects a predetermined touch gesture based on the touch operation detected by the detector. The operated location detector detects an operated location of the touch gesture. The divided area identifier identifies which image among the images is displayed in the divided area corresponding to the operated location detected by the operated location detector. The image processor is provided with a function to perform scrolling or zooming of the display screen corresponding to the touch gesture, and changes a direction or a motion range according to the image being displayed in the identified divided area, upon performing the scrolling or zooming. Thus, it can apply a certain limitation (inhibiting the scrolling in a predetermined direction, limiting the motion range, etc.) to one of the plurality of images (not limited to, for example, the above navigation image), while preventing similar limitations from being applied to other images.
According to a third aspect of the invention, the following display method is provided. Thus, this display method includes a detection step, an image acquisition step, a display step, a gesture detection step, an operated location detection step, a divided area identification step, and an image processing step. A touch operation is detected at the detection step. At the image acquisition step, at least two of a navigation image containing map or nautical chart information created based on information obtained from a GNSS sensor for detecting a location of the display device, an underwater detection image created based on information obtained from an ultrasonic transducer for detecting underwater directly under the display device, and a radar image obtained from a radar antenna for detecting a location of an object around the display device are acquired. Acquired images are displayed in respective divided areas of a display screen at the display step. At the gesture detection step, a predetermined touch gesture is detected based on the touch operation detected at the detection step. An operated location of the touch gesture is detected at the operated location detection step. At the divided area identification step, which image among the images is displayed in the divided area corresponding to the operated location detected at the operated location detection step is identified. At the image processing step, it has a function to perform scrolling or zooming of the display screen corresponding to the touch gesture, and changes a direction or a motion range according to the image being displayed in the identified divided area, upon performing the scrolling or zooming.
Thus, it can apply a certain limitation (inhibiting the scrolling in a predetermined direction, limiting the motion range, etc.) to one of the plurality of images, while preventing similar limitations from being applied to other images.
According to a fourth aspect of the invention, a display program of the following configurations is provided. Thus, this display program causes a computer to execute a detection procedure, an image acquisition procedure, a display procedure, a gesture detection procedure, an operated location detection procedure, a divided area indentifying procedure, and an image processing procedure. A touch operation is detected in the detection procedure. In the image acquisition procedure, at least two of a navigation image containing map or nautical chart information created based on information obtained from a GNSS sensor for detecting a location of the display device, an underwater detection image created based on information obtained from an ultrasonic transducer for detecting underwater directly under the display device, and a radar image obtained from a radar antenna for detecting a location of an object around the display device are acquired. In the display procedure, acquired images are displayed in respective divided areas of a display screen. In the gesture detection procedure, a predetermined touch gesture is detected based on the touch operation detected in the detection procedure. An operated location of the touch gesture is detected in the operated location detection procedure. In the divided area indentifying procedure, which image among the images is displayed in the divided area corresponding to the operated location detected in the operated location detection procedure is identified. In the image processing procedure, it has a function to perform scrolling or zooming of the display screen corresponding to the touch gesture, and changes a direction or a motion range according to the image being displayed in the identified divided area, upon performing the scrolling or zooming.
Thus, it can apply a certain limitation (inhibiting the scrolling in a predetermined direction, limiting the motion range, etc.) to one of the plurality of images, while preventing similar limitations from being applied to other images.
Next, one embodiment of the invention is described with reference to the drawings.
The ship instrument network system 1 of this embodiment includes a plurality of ship instrument connected to a ship network 10. Ship instrument can exchange detected information via the ship network 10. As a standard of the ship network 10, LAN (Local Area Network) or CAN (Controller Area Network) can be employed, for example.
As shown in
The touch panel device 11 has a configuration to create and display an image (sensor image) based on information detected by other ship instrument (sensors), detect a touch operation to a display screen, and perform processing according to a detection result. Specifically, the touch panel device 11 includes a display unit 21, a user interface 22, a memory 23, a detector 24, and a controller 25.
The display unit 21 is comprised of, for example, a liquid crystal display, and can display, for example, a sensor image, and various setting screens on the display screen as described above.
The user interface 22 is comprised of hardware keys, such as a rotary key to which a clockwise or counterclockwise rotational operation is possible, and a menu key to call a menu screen. Note that, since the touch panel device 11 can be inputted and instructed by a touch operation on the display screen, it can reduce the number of keys in the user interface 22.
The memory 23 stores contents of program(s) to be executed by the controller 25, nautical chart information, traveling route(s) set by the user, etc.
The detector 24 detects the touch operation on the screen by a user. In this embodiment, a projected capacitance system is used as a system for detecting the touch operation. This system has a configuration in which a plurality of high transparency electrodes are arranged on a display panel and a touched location is detected based on a change in electric capacity of each electrode which occurs when a fingertip approaches the panel. With this configuration, not only the touched location but a finger motion in a touching state can be detected. Moreover, the detector 24 can also detect touched locations and changes of each touched location when two or more locations are touched simultaneously. The touched location and the change in the touched location detected by the detector 24 are outputted to the controller 25. Note that the system for detecting the touch operation is not limited to the projected capacitance system, but any other suitable system may be employed. Moreover, other than the touch operation with a finger, a configuration in which a touch operation using a tapered stick member may also be detected, for example.
The controller 25 creates the sensor images (such as a radar image and a nautical chart around the ship) based on the stored content of the memory 23 and information received from other ship instrument, and displays them on the display unit 21. The controller 25 receives information from a plurality of ship instrument, and creates a plurality of sensor images. The controller 25 can switch between a mode in which only one of the plurality of sensor images is displayed on the display screen (full screen mode) and a mode in which the display screen is divided and, the plurality of sensor images are displayed (split screen mode: see
Moreover, the controller 25 identifies which touch gesture is performed by the user by executing a matching of the content of the touch operation set in advance (touch gesture) and a change of the touched location detected by the detector 24. Then, the controller 25 performs processing corresponding to the touch gesture identified.
An example of the touch gesture includes a “drag operation.” The drag operation is a touch gesture in which a touching finger (usually one finger) is moved in a predetermined direction without separating from the display screen. Scrolling of the image is usually associated with the drag operation. Note that the drag operation includes an operation in which a finger is quickly moved while the finger touches the display screen (flick operation). Moreover, another example of the touch gesture includes a “pinch operation.” The pinch operation is an operation in which, without separating two touching fingers from the display screen, the fingers bringing close to each other (pinch in) or separating from each other (pinch out). Processing for changing a scale of the image (processing for performing a zooming) is usually associated with the pinch operation. Note that the controller 25 can also identify various touch gestures other than the example illustrated above.
The GPS antenna 12 receives positioning signals from GPS Satellites (GNSS satellites), and outputs them to the touch panel device 11 and the like via the ship network 10. The controller 25 of the touch panel device 11 obtains a location of the ship based on the positioning signals (in detail, a location of a GPS antenna or an absolute location in a terrestrial reference frame). Note that a configuration in which a calculation for finding a location based on the positioning signals is performed on the GPS antenna 12 side, and the location of the ship is outputted to the touch panel device 11 may also be possible.
The touch panel device 11 can exhibit a function as a navigation equipment based on the obtained location of the ship and the nautical chart information stored in the memory 23. Specifically, the controller 25 can superimpose the location of the ship on the nautical chart based on the acquired location of the ship and the nautical chart information stored in the memory 23, and display it on the display unit 21. Moreover, the controller 25 can find a ground speed of the ship using the location of the ship which changes with time and find a trace of the ship to display them on the display unit 21. Further, the controller 25 can create a traveling route and display it on the display unit 21 by a user selecting a destination and a waypoint (a stopover) with a touch operation (see a first sensor image 31 shown in
The radar antenna 13 transmits a microwave and receives a reflection wave from an object. This reflection wave is outputted to the touch panel device 11 after suitable signal processing is performed. The touch panel device 11 creates a radar image based on the reflection wave. Specifically, the controller 25 of the touch panel device 11 finds a distance of the object based on a time length after transmitting the microwave until it receives the reflection wave. The controller 25 finds a direction in which the object exists based on a transmitted direction of the microwave. Thus, the controller 25 creates the radar image and displays it on the display unit 21 (see a second sensor image 32 shown in
The fish finder 14 is comprised of a transducer and an analyzer. The transducer is installed on, for example, the bottom of the ship, and it discharges an ultrasonic wave directly below into the sea and receives the reflection wave from a seabed or a school of fish. The analyzer creates fish finding data (data acquired by the fish finder or data of the school of fish or the seabed) based on the reflection wave. Moreover, the fish finder 14 of this embodiment has a function to determine an underwater state (a bottom sediment type) based on the acquired fish finding data. Specifically, the analyzer can determine whether the seabed falls under any of rocks, gravel (stones), sand, and mud with high possibility, by analyzing the reflection wave received. The fish finding data and the determined bottom sediment type are outputted to the touch panel device 11. Then, the controller 25 of the touch panel device 11 creates a third sensor image 33 (see
The heading sensor 15 is configured to detect a heading of the ship (a direction of a bow of the ship) by an absolute azimuth in the terrestrial reference frame. Generally, a ship travels forward, toward the bow direction. Therefore, it can also be said that the heading sensor 15 detects an azimuth of the forward direction of a hull of the ship. A magnetic azimuth sensor, a GPS compass or the like may also be used for the heading sensor 15, for example.
The autopilot device 16 is a device for automatically operating a rudder so that the ship travels along the set traveling route. Specifically, the autopilot device 16 finds how much the bow of the ship is to be changed based on the heading acquired from the heading sensor 15 and the traveling route acquired from the touch panel device 11. Then, the autopilot device 16 coincides the course of the ship with the traveling route by changing an angle of the rudder according to the calculated value.
The ship instrument network system 1 of this embodiment is configured as described above. Note that the ship instrument which constitutes the ship instrument network system 1 may be arbitrary configured, and the system may have a configuration in which ship instrument other than illustrated above may also be connected, or may have a configuration in which two or more ship instrument of similar type may also be connected. Moreover, the processing of the data acquired by the ship instrument may be configured to be performed by the ship instrument concerned, or may be configured to be performed in the controller 25 of the touch panel device 11.
Next, processing performed by the controller 25 in the divided display mode when the touch operation is performed on the image respectively displayed on the divided screens will be described. Below, an image based on the information obtained from the sensor(s) is referred to as a “sensor image.” More specifically, an image containing the map or the nautical chart created based on the information obtained from the GPS antenna 12 is referred to as a “first sensor image 31” (navigation image). An image obtained from the radar antenna 13 is referred to as a “second sensor image 32” (radar image), and an image created based on the information obtained from the above transducer is referred to as a “third sensor image 33” (underwater detection image).
Further, in this embodiment, it is considered that in each sensor image, a drag operation is associated with scrolling of the display image, and a pinch-out operation is associated with zooming processing. Thus, the touch panel device 11 has a common correspondence relation of the sensor image between the touch gesture and processing performed by the controller 25 when it detects the touch gesture. However, the controller 25 is configured such that the touch panel device 11 changes a direction or a motion range of the scrolling or the zooming according to the sensor image on which the touch gesture is performed.
First, a detailed configuration of the controller 25 will be described with reference to
The image acquirer 51 acquires the first sensor image 31, the second sensor image 32, and the third sensor image 33. Note that, in this embodiment, the sensor image is configured to be created by an image creator (not illustrated) of the touch panel device 11, and the image acquirer 51 acquires the sensor image from the image creator. Note that, alternatively to this configuration, it may be a configuration in which instrument external to the touch panel device 11 (such as the fish finder 14) creates the sensor image. In this case, the image acquirer 51 acquires the sensor image from the instrument external to the touch panel device 11.
The gesture detector 52 detects a predetermined touch gesture (such as the drag operation or the pinch operation) based on the touch operation detected by the detector 24, as described above.
The operated location detector 53 detects an operated location of the touch gesture. As the operated location, a first touched location where the touch gesture is performed may be used, or a central location of the touch gesture may be used, for example. The central location of the touch gesture refers, for example, if a drag operation is performed, to a location corresponding to a central part of a line segment indicating a locus of the drag. The central location if the pinch operation is performed is a location corresponding to a central part of a line segment drawn by connecting two touched locations.
The divided area identifier 54 identifies which image is displayed in a divided area corresponding to the operated location detected by the operated location detector 53. Thus, when the operated location is on the first sensor image 31, the divided area identifier 54 identifies that the touch gesture is performed on the first sensor image 31.
The image processor 55 has a function to perform scrolling or zooming processing of the display screen corresponding to the touch gesture, and changes the direction or the motion range according to the image being displayed in the identified divided area, upon performing the scrolling or zooming.
Next, processing performed by the controller 25 against the detection result of the detector 24 will be described with reference to
Next, with reference to
Note that, the controller 25 can recognize that the drag operation is performed by the processing at S102 of
First, a case where the drag operation is performed on the first sensor image 31 will be described. Here, it is assumed that the user moved his/her finger from the location shown in the first sensor image 31 of
Next, a case where the drag operation is performed on the second sensor image 32 will be described. Here, it is assumed that the user moved his/her finger from the location shown in the second sensor image 32 of
Next, a case where the drag operation is performed on the third sensor image 33 will be described. Here, it is assumed that the user moved his/her finger from the location shown in the third sensor image 33 of
Thus, each sensor image can be scrolled by performing the drag operation. Although there is no limitation for the scrolling of the first sensor image 31, limitations are provided for the scrolling of the second sensor image 32 and the third sensor image 33 according to characteristics of the sensor images. By providing the limitations in this way, for the second sensor image 32, since the ship mark will not disappear from the second sensor image 32 even if the user drags greatly, accuracy is not required for a drag amount (scroll amount) of the drag operation performed on the second sensor image 32. For the third sensor image 33, since the third sensor image 33 will be scrolled in the time-axis direction even if the user drags obliquely, accuracy is not required in the drag direction (scroll direction) of the drag operation performed on the third sensor image 33. Therefore, a user's operability can be improved.
Next, with reference to
Note that, the controller 25 can recognize that the pinch-out operation is performed by the processing at S102 of
First, a case where the pinch-out operation is performed on the first sensor image 31 will be described. Here, it is assumed that the user performs the pinch-out operation at a location shown in the first sensor image 31 of
Next, a case where the pinch-out operation is performed on the second sensor image 32 will be described. Here, it is assumed that the user performed the pinch-out operation at a location shown in the second sensor image 32 of
Next, a case where the pinch-out operation is performed on the third sensor image 33 will be described. Here, it is assumed that the user moved his/her finger from the location shown in the third sensor image 33 of
Thus, each sensor image can be zoomed in by performing the pinch-out operation. Although there is no limitation for the zoom-in of the first sensor image 31, limitations are provided for scrolling of the second sensor image 32 and the third sensor image 33, according to characteristics of the sensor images. By providing limitation in this way, since the ship does not disappear from the second sensor image 32 even if the user performs the pinch-out operation to arbitrary locations for the second sensor image 32, accuracy is not required for the locations at which the pinch-out operation is performed. Moreover, for the third sensor image 33, since the third sensor image 33 is zoomed in in the vertical direction even if the user performs the pinch-out operation in arbitrary directions, accuracy is not required for the directions the pinch-out operation is performed. Therefore, the user's operability can be improved.
As illustrated above, the touch panel device 11 (display device) acquires the first sensor image 31 (navigation image), the second sensor image 32 (radar image), and the third sensor image 33 (underwater detection image). The touch panel device 11 displays the acquired images in the respective divided areas on the display screen. The touch panel device 11 detects the predetermined touch gesture, and detects the operated location of the touch gesture. The touch panel device 11 identifies which image is displayed corresponding to the operated location. The touch panel device 11 executes processing corresponding to the identified touch gesture (scrolling or zooming processing of the display image), and changes the direction or the motion range of the processing according to the identified sensor image.
Therefore, the processing performed according to the contents of the touch gestures are made common between the sensor images to unify a feeling of operation and, thus, the processing can be performed in a mode according to the characteristics of the sensor images.
Although a suitable embodiment of the present invention is illustrated above, the above configuration may be modified as follows, for example.
Although the above embodiment illustrates the configuration in which the display screen is divided into three, it may be divided into two (the first sensor image and the second sensor image are displayed), as shown in
The touch gesture is not limited to the example illustrated above, but various touch gestures, such as a drag operation with two fingers, may also be employed.
Although the present invention has a configuration of providing limitations according to the sensor image, this limitation is not limited to the example illustrated above.
The present invention may be configured to be carried not only in the ship but also in other movable bodies (for example, an airplane).
The sensors illustrated above is merely an example and may be configured to acquire the information from sensors other than illustrated above. For example, as the acoustic sensor, a scanning sonar may be used instead of the fish finder 14.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2011/005577 | 10/3/2011 | WO | 00 | 4/2/2014 |
| Publishing Document | Publishing Date | Country | Kind |
|---|---|---|---|
| WO2013/051047 | 4/11/2013 | WO | A |
| Number | Name | Date | Kind |
|---|---|---|---|
| 5327398 | Wansley et al. | Jul 1994 | A |
| 5414809 | Hogan | May 1995 | A |
| 5465622 | Freking | Nov 1995 | A |
| 5867150 | Bricklin et al. | Feb 1999 | A |
| 6009373 | Shimojima et al. | Dec 1999 | A |
| 6195609 | Pilley | Feb 2001 | B1 |
| 6360168 | Shimabara | Mar 2002 | B1 |
| 6628299 | Kitayama | Sep 2003 | B2 |
| 6642936 | Engholm et al. | Nov 2003 | B1 |
| 6697015 | Kiya | Feb 2004 | B2 |
| 6833851 | Brunk | Dec 2004 | B1 |
| 7543241 | Brunk | Jun 2009 | B1 |
| 8296001 | Kabel et al. | Oct 2012 | B1 |
| 8712605 | Schulte et al. | Apr 2014 | B1 |
| 20020145556 | Kiya | Oct 2002 | A1 |
| 20020152266 | Burfeind et al. | Oct 2002 | A1 |
| 20040003958 | Fujimoto et al. | Jan 2004 | A1 |
| 20040064250 | Hirano | Apr 2004 | A1 |
| 20060038796 | Hinckley et al. | Feb 2006 | A1 |
| 20060089794 | DePasqua | Apr 2006 | A1 |
| 20090093275 | Oh et al. | Apr 2009 | A1 |
| 20090271054 | Dokken | Oct 2009 | A1 |
| 20100005419 | Miichi et al. | Jan 2010 | A1 |
| 20100211920 | Westerman et al. | Aug 2010 | A1 |
| 20100235794 | Ording | Sep 2010 | A1 |
| 20100309753 | Misonoo et al. | Dec 2010 | A1 |
| 20110004822 | Nezu et al. | Jan 2011 | A1 |
| 20110013484 | Coleman et al. | Jan 2011 | A1 |
| 20110022238 | Pollock et al. | Jan 2011 | A1 |
| 20110074716 | Ono | Mar 2011 | A1 |
| 20110275317 | Ryu | Nov 2011 | A1 |
| 20120078684 | Maciocci | Mar 2012 | A1 |
| 20130018254 | Drucker | Jan 2013 | A1 |
| 20130191066 | Spillane et al. | Jul 2013 | A1 |
| Number | Date | Country |
|---|---|---|
| 101285880 | Oct 2008 | CN |
| 1185398 | Mar 1999 | JP |
| 2008282390 | Nov 2008 | JP |
| 2006017511 | Feb 2006 | WO |
| 2008031880 | Mar 2008 | WO |
| Entry |
|---|
| ISA Japanese Patent Office, International Search Report of PCT/JP2011/005577, Mar. 19, 2012, WIPO, 1 page. |
| European Patent Office, Extended European Search Report Issued in Application No. 11869512.1, Aug. 17, 2015, Germany, 9 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20150035772 A1 | Feb 2015 | US |
