Patent Application
20100070162
The present invention relates to a communication-type pedestrian navigation system for searching a guide route from a departure point to a destination and providing guidance using image information in which the guide route is displayed in a display means, and particularly relates to a pedestrian navigation system configured so as to capture an image of a peripheral scene at an intersection or the like, display the image as a peripheral image in the display means, and display a route guidance image on the displayed peripheral image, whereupon the system is guided toward an imaging direction in which a route image can be displayed in a case in which a route image cannot be displayed on the peripheral image.
In the past, when traveling to a destination in an unfamiliar place, a traveler followed an atlas or the like and arrived at the destination by identifying a transport facility, a road, a landmark, or an address shown in the map. In an automobile equipped with a car navigation system (hereinafter referred to as a car navigation system), a traveler reaches a destination while receiving guidance displayed on a monitor screen or voice guidance (navigation information) from the navigation system by activating the car navigation system and inputting the destination.
The car navigation system described above uses a GPS (Global Positioning System) in which GPS signals transmitted from a plurality of GPS satellites orbiting the earth are received by a GPS antenna, and a position is determined by analyzing satellite positions, time information, and the like that are included in the GPS signals. At least four or more of the plurality of GPS satellites are needed to determine a position. The accuracy of independent positioning in a GPS in general is a little over 10 m, but the accuracy is enhanced to 5 m or less through the use of a DGPS (Differential GPS). A trend in all models of so-called third-generation mobile telephones is the installation of GPS receivers that are currently installed only in some mobile telephones.
Techniques in various fields have been proposed for utilizing a mobile terminal that has this type of positioning capability. For example, a proposal has been made for the evolution of an automobile navigation device (car navigation system) into a communication-type navigation system used by a walker, in which map/route information is delivered from an information delivery server (route search server) using a mobile telephone as a terminal.
The performance of mobile telephones, PHS, and other mobile communication terminal devices has improved dramatically in recent years, and these devices are being provided with multifunction capability. Besides voice communication capability, data communication capability in particular is being emphasized, and various types of data communication services are provided to users over the Internet. A navigation service is one of these services, and a communication-type navigation system has been implemented for providing route guidance from a current position to a destination to not only the driver of an automobile, but also the user of a mobile telephone.
An example of the route search device and route search method used in a typical navigation device and communication navigation system is disclosed in Patent Document 1 (Japanese Laid-open Patent Application No. 2001-165681) below. In this navigation system, information relating to the departure point and the destination is transmitted from a mobile navigation terminal to an information delivery server, the information delivery server searches a route that matches search conditions from road network or transportation network data, and guidance is provided. The search conditions include the means of travel from the departure point to the destination, examples of which include walking, an automobile, a railroad, a combination of railroad and walking, and the like. The route search is performed using this factor as one search condition.
The information delivery server is provided with a database of cost information (distance or required time) for all links, wherein the links are routes that connect nodes, and nodes are the positions of junctions and turns in roads (routes) in the map data. The information delivery server can guide the mobile navigation terminal along the shortest route by referring to the database, sequentially searching the links that lead from the departure point node to the destination node, and creating a guide route by tracing nodes and links for which the cost information of the link indicates the smallest cost. A technique known as a label setting algorithm or a Dijkstra algorithm is used as the technique for performing this type of route search. In Patent Document 1 mentioned above, a route search method that uses this Dijkstra algorithm is also disclosed.
In a typical communication-type navigation system, the map information or guide route information displayed by the terminal device is downloaded from a route search server or other server device connected to the terminal device via a network. The terminal device to which the map information or guide route is delivered from the route search server or other information delivery server is provided with a liquid crystal display device or other display means, the map and guide route are displayed in the display means as well as the current position (current position of the terminal device) of the walker or automobile on the guide route, and the walker or automobile is guided to the destination.
The map displayed in the display means is typically a plan view, i.e., a two-dimensional map display, but methods have been proposed for displaying a planar aerial map that brings a sense of perspective to the display image, or an aerial map in which buildings are simulated in three dimensions in order to make landmarks or buildings easier to recognize, or for displaying roads or buildings as three-dimensional polygon images by a Z buffer method.
For example, Patent Document 2 (Japanese Laid-open Patent Application No. 2001-27535) discloses a map display device in which buildings or roads are displayed in three dimensions. In the map display device disclosed in Patent Document 2, shading is added to the route guide line on a road to create a three-dimensional map display. When a positional relationship occurs in which the route guide line is hidden by a building, the overlapped portion and the non-overlapped portion of the route guide line are displayed in different colors. In particular, the display color of the route guide line and the display color of buildings are drawn by a semi-transparency method of setting mutually different colors in the pixel units in VRAM (video RAM), and the relative positioning of buildings and the route guide line is clarified to enhance the visibility of the route guide line.
In order to create a display such as described above, building shape data and height information are stored in the map data in the map display device disclosed in Patent Document 2, and multi-level crossings and other road shape data are also stored. Based on this map data, buildings or multi-level road crossings are drawn in three dimensions in a drawing unit, and when the vehicle arrives at a position for which route guidance is to be provided, the desired image is drawn in the drawing unit, a predetermined sound is outputted by a voice output unit, and the user is guided to the destination.
A map is generally displayed in three dimensions using polygon data in the following manner. Specifically, the surfaces that form each building to be displayed in the map are prepared as polygon data, and the building polygon data are transmitted together with the map data to the map display device. The map display device displays the map and displays each building positions on the map in three dimensions using the polygon data. At this time, textures are applied to the surfaces of the buildings and displayed in order to more realistically display the buildings. The textures are data for showing patterns on the surfaces of the buildings, and include the shapes of windows in the buildings, wall designs, and the like.
For example, Patent Document 3 (Japanese Laid-open Patent Application No. 2000-259861) below discloses'a texture mapping device whereby textures are applied to polygon data and drawn. The invention disclosed in Patent Document 3 accurately and efficiently applies textures to polygons and facilitates subsequent editing. This texture mapping device designates square polygons P1 through P4 arranged in M lines and N columns on a wire frame, and textures applied thereto. The textures to be applied are divided according to the ratio of the edge lengths Lp1 through Lp12 of the designated plurality of polygons P1 through P4. A configuration is adopted in which the orientation of the polygons is taken into account to apply the divided textures to the corresponding polygons.
When buildings are displayed in three dimensions using polygon data and texture data in this manner, the polygon data indicating the surfaces of the buildings, and the texture data to be applied to the surfaces have a one-to-one correspondence, the texture data are accumulated in the server database along with the polygon data, and the polygon data and texture data of the buildings to be displayed on the map are transmitted to the map display device together with a predetermined range of map data.
According to such a method for creating a three-dimensional map display, since the user of the terminal device can see a display image that approximates the scenery actually being observed, advantages are gained in that the user can easily recognize the locations of right or left turns and the direction of travel. In order to obtain this advantage, however, map data for three-dimensional display separate from the plan-view map data are necessary in order to display the map in three dimensions. Three-dimensional map data include more data than plan-view'map data, and a large-capacity storage device is needed as the storage device for storing the map data. In a communication-type navigation system, the communication load between the terminal device and the route search server is also large, and there are also long communication times. Therefore, various problems arise such as the need for processing performance in the route search server sufficient to provide service to numerous terminal devices.
In order to overcome such problems, navigation devices have been proposed in Patent Document 4 (Japanese Laid-open Patent Application No. 2004-257979) and Patent Document 5 (Japanese Laid-open Patent Application No. 2006-162442), in which' an image of a scene in the forward direction is captured using a CCD camera or other imaging means, the captured peripheral image is displayed on a display screen, a forward direction guidance image of a right turn or a left turn is displayed superimposed on the peripheral image, and the guide route is displayed in a car-mounted navigation device.
Specifically, in the navigation device disclosed in Patent Document 4, an imaging means for imaging a scene in the forward direction, and a display means for displaying the image captured by the imaging means are provided, and when a branch point on a route detected by a route detection means, and the current position detected by a current position detection means are equal to or less than a predetermined distance from each other, an image for guiding in the recommended direction of travel at the branch point is superimposed on the image captured by the imaging means and displayed in the display means.
In the navigation device disclosed in Patent Document 5, a guide information specifying unit is provided with a guide information specifying unit for specifying a position on a map for presenting guide information, and the type of guide information presented; an obstacle detection unit for detecting the position and type of obstacles (pedestrians, vehicles, and the like) around the vehicle in which the device is used through the use of object recognition or the like; a display position determination unit for calculating a position that does not coincide with an obstacle detected in a display position of the guide information; and an image output unit for outputting guide information in a determined position within the windshield view of the vehicle, or a determined position within the frontal image display area; and the direction of travel along the route is superimposed and displayed on the actual scene viewed from the vehicle.
Navigation systems for use by a walker also include the mobile terminal device navigation device disclosed in Patent
Document 6 (Japanese Laid-open Patent Application No. 2005-241385) in which the route direction is superimposed and displayed on an image of the actual surroundings that is captured by a camera. In this mobile terminal device, the screen of the digital camera' is demarcated based on the current position of the user (destination guidance device), destination direction coordinates that correspond to the destination are determined for the screen, and an instructional display with respect to the destination direction coordinates is superimposed on the captured camera image on the screen on the basis of a predetermined reference point (the middle of the lower edge of the screen).
[Patent Document 1]: Japanese Laid-open Patent Application No. 2001-165681 (FIGS. 1 and 2)
[Patent Document 2]: Japanese Laid-open Patent Application No. 2001-27535 (FIGS. 1 and 4, paragraph [0054])
[Patent Document 3]: Japanese Laid-open Patent Application No. 2000-259861 (FIGS. 2 and 8)
[Patent Document 4]: Japanese Laid-open Patent Application No. 2004-257979 (FIG. 4)
[Patent Document 5]: Japanese Laid-open Patent Application No. 2006-162442 (FIGS. 4 and 5)
[Patent Document 6]: Japanese Laid-open Patent Application No. 2005-241385 (FIG. 9)
When a technique for displaying a peripheral image captured by a camera or other imaging means and displaying a forward direction or guide route on the peripheral image is used in a car-mounted navigation device, since the orientation of the camera for imaging the actual scene is fixed in the forward direction (forward direction) of the vehicle as a characteristic of automobile movement, the route essentially continues forward at all times during driving along the route. Even when there is a departure from the guide route as a result of a wrong turn, since the guide route is reset to the forward direction of the vehicle by a rerouting function (re-searching function) in the route search, the guide route always appears in front of the vehicle, i.e., in front of the imaging camera.
A walker navigation device differs from a vehicle-mounted navigation device in that the navigation device is not fixed in the forward direction, as in the case of a vehicle-mounted device, and the device can be oriented in any direction depending on the orientation of the user or the manner in which the user holds the device: For example, for navigation, the orientation of the terminal device when the map image or guide route is displayed is entirely arbitrary, and there is no guarantee that the guide route will be included in the image displayed at a given time. Since the horizontal field of view of the camera is adequately small in comparison to 360 degrees, the probability of the guide route being included in the screen display is somewhat low. The technique used for the vehicle-mounted navigation device disclosed in Patent Document 4 or 5 therefore cannot be applied to a walker navigation device without modification.
Consequently, in a pedestrian navigation system, several factors must be taken into account for the user when the guide route is not included in the peripheral image captured by the camera or other imaging means. In the mobile terminal for a walker according to Patent Document 6, an image of the scene in the forward direction is captured by a digital camera, and an instructional display for the destination direction is superimposed on the screen on the image captured by the camera. However, this mobile terminal device works on the assumption that the camera is oriented in the direction of the guide route, in basically the same manner as in a vehicle-mounted navigation device.
Specifically, this mobile terminal is configured so as to capture an image of the scene in the forward direction using a camera and display the guide route (forward direction) on the image, and has drawbacks in that no adaptation is made for cases in which the camera is not oriented in the forward direction. In such a mobile terminal, the guide route or guide displays for right and left turns can be displayed on the screen on which the image captured by the camera is displayed when the user is properly oriented in the forward direction with respect to the guide route at an intersection or the like, but when the camera orientation, i.e., the orientation of the mobile terminal, does not match the forward direction, drawbacks occur in that the guide route does not appear on the screen on which the image captured by the camera is displayed, and the guide route or forward direction cannot be indicated to the user on the image captured by the camera.
As a result of various studies of the problems described above, the inventors developed the present invention upon discovering that the problems described above can be overcome by providing a CCD camera or other imaging means and an orientation detection means composed of a magnetic bearing sensor, a tilt sensor, or the like to a mobile terminal device in a communication-type pedestrian navigation system, wherein the angle with respect to the horizon, or the orientation direction of the mobile terminal device is detected by the orientation detection means, and when the guide route is not included in the peripheral image captured by the imaging means, a display is provided indicating how the orientation of the imaging means, i.e., the mobile terminal device, should be corrected.
Specifically, an object of the present invention, which is aimed at overcoming the problems described above, is to provide a pedestrian navigation system for displaying a route guide image on a peripheral image captured by an imaging means, whereupon the system is guided toward an imaging direction in which a route image can be displayed in a case in which the route image cannot be displayed on the peripheral image.
A first aspect of the present invention for achieving the abovementioned objects is a navigation system comprising a route search server provided with route search means for referencing network data for route searching, and searching a guide route from a departure point to a destination; and a mobile terminal device having guide information storage means for storing guide information that includes guide route data, GPS receiving means for determining a current position, imaging means, display means for displaying a peripheral image captured by the imaging means, and drawing control means for displaying a guide route image or a forward direction guide image on the peripheral image displayed by the display means; wherein the navigation system is characterized in that the mobile terminal device has orientation detection means for detecting the orientation of the mobile terminal device when the imaging means has captured the peripheral image, and image layout means for determining a display position of the guide route image or the forward direction guide image on the basis of the orientation of the mobile terminal device as detected by the orientation detection means; and the drawing control means displays an imaging direction guide image for guiding the imaging means toward an orientation at which an image is to be captured by the imaging means on the peripheral image when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
In the first aspect, the orientation detection means preferably comprises bearing detection means for detecting the orientation of the imaging means when the imaging means has captured a peripheral image, and tilt angle detection means for detecting the tilt angle of the imaging means with respect to the horizontal direction.
In the first aspect, the image layout means preferably determines the display position of the guide route image or the forward direction guide image displayed on the peripheral image, the position being determined on the basis of the orientation of the mobile terminal device as detected by the orientation detection means, when the guide route is included in the peripheral image displayed by the display means.
In the first aspect, the drawing control means preferably displays an imaging direction guide image for guiding the imaging means toward a direction in which the guide route is present, as an orientation in which an image is to be captured by the imaging means on the peripheral image, when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
Furthermore, in the first aspect, the imaging direction guide image displayed by the drawing control means is preferably displayed using a different image than the forward direction guide image.
A second aspect of the present invention is a mobile terminal device connected via a network to a route search server provided with route search means for referencing network data for route searching, and searching a guide route from a departure point to a destination; the mobile terminal device comprising: guide information storage means for storing guide information that includes guide route data, GPS receiving means for determining a current position, imaging means, display means for displaying a peripheral image captured by the imaging means, and drawing control means for displaying a guide route image or a forward direction guide image on the peripheral image displayed by the display means; wherein the mobile terminal device is characterized in that the mobile terminal device further comprises orientation detection means for detecting the orientation of the mobile terminal device when the imaging means has captured the peripheral image, and image layout means for determining a display position of the guide route image or the forward direction guide image on the basis of the orientation of the mobile terminal device as detected by the orientation detection means; and the drawing control means displays an imaging direction guide image for guiding the imaging means toward an orientation at which an image is to be captured by the imaging means on the peripheral image when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
In the second aspect, the orientation detection means preferably comprises bearing detection means for detecting the orientation of the imaging means when the imaging means has captured a peripheral image; and tilt angle detection means for detecting the tilt angle of the imaging means with respect to the horizontal direction.
In the second aspect, the image layout means preferably determines the display position of the guide route image or the forward direction guide image displayed on the peripheral image, the position being determined on the basis of the orientation of the mobile terminal device as detected by the orientation detection means, when the guide route is included in the peripheral image displayed by the display means.
In the second aspect, the drawing control means preferably displays an imaging direction guide image for guiding the imaging means toward a direction in which the guide route is present, as an orientation in which an image is to be captured by the imaging means on the peripheral image, when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
Furthermore, in the second aspect, the imaging direction guide image displayed by the drawing control means is preferably displayed using a different image than the forward direction guide image.
Furthermore, a third aspect of the present invention is a guide display method for a mobile terminal device, the mobile terminal device being connected via a network to a route search server provided with route search means for referencing network data for route searching, and searching a guide route from a departure point to a destination; the mobile terminal device comprising guide information storage means for storing guide information that includes guide route data, GPS receiving means for determining a current position, imaging means; display means for displaying a peripheral image captured by the imaging means, and drawing control means for displaying a guide route image or a forward direction guide image on the peripheral image displayed by the display means; wherein the route guide display method is characterized in that the mobile terminal device further comprises orientation detection means for detecting the orientation of the mobile terminal device when the imaging means has captured the peripheral image, and image layout means for determining a display position of the guide route image or the forward direction guide image on the basis of the orientation of the mobile terminal device as detected by the orientation detection means; and the route guide display method comprises a step of the image layout means determining whether the guide route is included in the peripheral image displayed by the display means, and a step of the drawing control means displaying an imaging direction guide image for guiding the imaging means toward an orientation at which an image is to be captured by the imaging means on the peripheral image when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
In the third aspect, the orientation detection means preferably comprises bearing detection means for detecting the orientation of the imaging means when the imaging means has captured a peripheral image; and tilt angle detection means for detecting the tilt angle of the imaging means with respect to the horizontal direction; and the route guide display method preferably comprises a step of the image layout means determining, on the basis of detection output of the bearing detection means and tilt angle detection means, whether the guide route is included in the peripheral image displayed by the display means, and a step of the drawing control means displaying an imaging direction guide image for guiding the imaging means toward an orientation at which an image is to be captured by the imaging means on the peripheral image when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
In the third aspect, the route guide display method preferably further comprises a step of determining the display position of the guide route image or the forward direction guide image displayed on the peripheral image, the position being determined on the basis of the orientation of the mobile terminal device as detected by the orientation detection means, when the image layout means determines that the guide route is included in the peripheral image displayed by the display means, in the step of the image layout means determining whether the guide route is included in the peripheral image displayed by the display means.
Also in the third aspect, the route guide display method preferably further comprises a step of the drawing control means displaying an imaging direction guide image for guiding the imaging means toward a direction in which the guide route is present, as an orientation in which an image is to be captured by the imaging means on the peripheral image, when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means, in the step of the image layout means determining whether the guide route is included in the peripheral image displayed by the display means.
Furthermore, in the third aspect, the step of displaying the imaging direction guide image on the peripheral image preferably includes processing whereby the imaging direction guide image displayed by the drawing control means is displayed using a different image than the forward direction guide image.
In the first aspect of the present invention, the mobile terminal device comprises orientation detection means for detecting the orientation of the mobile terminal device when the imaging means has captured a peripheral image, and image layout means for determining the display position of the guide route image or the forward direction guide image on the basis of the orientation of the mobile terminal device as detected by the orientation detection means; and the drawing control means displays an imaging direction guide image for guiding the imaging means toward an orientation at which an image is to be captured by the imaging means on the peripheral image when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means.
According to such a configuration, even when the orientation of the camera as the imaging means, i.e., the orientation of the mobile terminal device, does not match the direction of the guide route, i.e., the forward direction, since the imaging direction guide image for guiding the camera toward the orientation in which an image is to be captured by the imaging means is displayed on the peripheral image, the user can correct the orientation of the mobile terminal device so that the guide route or forward direction is displayed on the peripheral image by changing the imaging direction in accordance with the imaging direction guide image.
In the first aspect of the present invention, the orientation detection means preferably comprises bearing detection means for detecting the orientation of the imaging means when the imaging means has captured a peripheral image; and tilt angle detection means for detecting the tilt angle of the imaging means with respect to the horizontal direction. Through this configuration, since it is possible to detect the orientation (bearing) and tilt angle of the optical axis of imaging by the imaging means, it is possible to determine the display position in which to display the guide route or the forward direction guide image with respect to the peripheral image.
In the first aspect of the present invention, the image layout means preferably determines the display position of the guide route image or the forward direction guide image displayed on the peripheral image, the position being determined on the basis of the orientation of the mobile terminal device as detected by the orientation detection means, when the guide route is included in the peripheral image displayed by the display means. Through this configuration, it is possible to determine the display position in which to display the guide route or the forward direction guide image with respect to the peripheral image in accordance with the orientation of the mobile terminal device.
In the first aspect of the present invention, the drawing control means preferably displays an imaging direction guide image for guiding the imaging means toward a direction in which the guide route is present, as an orientation in which an image is to be captured by the imaging means on the peripheral image, when the image layout means determines that the guide route is not included in the peripheral image displayed by the display means. Through this configuration, the imaging direction guide image indicating the direction in which the guide route is present is displayed as the orientation for imaging by the imaging means when the guide route is not included in the peripheral image displayed by the display means, and the user can therefore easily correct the orientation of the mobile terminal device so that the guide route or forward direction is displayed in the peripheral image, by changing the imaging direction in accordance with the imaging direction guide image.
Furthermore, in the first aspect of the present invention, the imaging direction guide image displayed by the drawing control means is preferably displayed using a different image than the forward direction guide image. Through this configuration, the user can easily distinguish between the forward direction guide image and the imaging direction guide image.
The second aspect of the present invention makes it possible to provide the mobile terminal device of the first aspect, and the second aspect also makes it possible to provide a route guide display method for implementing the mobile terminal device of the second aspect.
10 navigation system
12 network
20 mobile terminal device
201 control means
21 GPS receiving means
22 imaging means
23 orientation detection means
24 image layout means
25 drawing control means
26 communication means
27 guide information storage means
28 display means
29 operational input means
30 route search server
301 control means
31 communication means
32 guide information collecting means
33 route search means
34 map database
35 route search network database
Embodiments of the present invention will be described in detail below using embodiments and drawings. However, the embodiments described hereinafter are merely examples of the navigation system intended to specify the technical idea of the present invention, and are not intended to specify the present invention as a navigation system. The present invention has equivalent applicability to other embodiments of a navigation system included in the scope of the claims.
The route search server 30 is also provided with a control means 301, a communication means 31, and a guide information collecting means 32. The control means 301 is a microprocessor having RAM, ROM, and a processor (not shown), and controls the operation of each component through the use of a control program stored in the ROM. The communication means 31 is an interface for communicating with the mobile terminal device 20 via the network 12.
The route search server 30 references the route search network database 35 and searches for a guide route that matches the route search conditions when there is a route search request including a current position, a destination, a movement means (automobile, walking, a transportation facility, a combination of walking and a transportation facility, or the like) and other route search conditions from the mobile terminal device 20. The guide route data obtained as a result of the route search are collected in guide information (data) for transmission to the mobile terminal device 20 by the guide information collecting means 32, together with unit map data (map data divided into areas of predetermined size by latitude/longitude) of a predetermined range that include the current position of the mobile terminal device 20 and are selected from the map database 34. The collected guide information is delivered to the mobile terminal device 20.
The mobile terminal device 20 is provided with a control means 201, a GPS receiving means 21, an imaging means 22, an orientation detection means 23, an image layout means 24, a drawing control means 25, a communication means 26, a guide information storage means 27, a display means 28, an operational input means 29, and other components. The control means 201 is a microprocessor having RAM, ROM, and processor (not shown), and controls the operation of each component through the use of a control program stored in the ROM. The communication means 26 is an interface for communicating with the route search server 30 via the network 12.
The operational input means 29 is a means of operating/inputting composed of number keys or alphabet keys, and other function keys, selection keys, scroll keys, and the like, and various input operations can be performed by operating a key or selecting a desired menu from a menu screen displayed in the display means 28 that is an output means. The display means 28 therefore also functions as a component of the operational input means 29.
When a user requests a route search from the route search server 30, the user operates the operational input means 29 of the mobile terminal device 20, selects a route search from a service menu displayed in the display means 28, and inputs the current position, destination, movement means (automobile, walking, a transportation facility, a combination of walking and a transportation facility, or the like), and other route search conditions. The inputted route search conditions are collected in a delivery request to the route search server 30 and transmitted as a route search request to the route search server 30 via the communication means 26.
When the mobile terminal device 20 receives guide information that is the result of the route search from the route search server 30, the mobile terminal device 20 temporarily stores the guide information in the guide information storage means 27. The map image and the image of the guide route are usually displayed in the display means 28 with the current position of the mobile terminal device 20 at the center on the basis of the guide information. A current position mark is displayed in the corresponding position on the map image. When the current position is near an intersection node or the like on the guide route, a forward direction guide image is displayed that indicates a forward direction such as a right turn, a left turn, or straight ahead at the intersection node. When the mobile terminal device 20 is provided with a speaker or other output means, the mobile terminal device 20 may be configured so as to provide voice guidance for indicating the forward direction or distance to the intersection node by voice guidance.
A map display such as described above occurs in the case of a normal guidance mode. In the present embodiment, an image guide mode is provided in which a peripheral image is captured by the imaging means 22, the peripheral image is displayed in the display means 28 along with a guide route image or a forward direction guide image, and guidance is provided. This image guide mode will be referred to hereinafter as a live image mode. The switch to live image mode is preferably made by the orientation detection means 23 detecting the orientation of the mobile terminal device 20 as described hereinafter, and automatically switching to the live view mode when a predetermined state occurs. Of course, the user may also switch the mode through the use of the operational input means 29.
The focal length (ZOOM) of the imaging means 22 (CCD camera, for example) and the screen size of the display means 28 of the mobile terminal device 20 are fixed at predetermined set values. The GPS receiving means 21 measures the current position of the mobile terminal device 20 at prescribed time intervals, and it is therefore possible to specify the position at which the imaging means 22 is operated and the peripheral image captured. The measurements by the GPS receiving means 21 are made every 1 to 5 seconds, for example, when the mobile telephone is the mobile terminal device 20.
When the user holds the mobile terminal device 20 out forward, the orientation detection means 23 detects when the optical axis of the imaging means 22 is within a predetermined angle from horizontal, and the shutter of the imaging means 22 automatically operates to capture a peripheral image (still image) in the direction of the optical axis. Since the orientation detection means 23 is composed of a bearing sensor and a tilt sensor, the orientation (bearing) of the optical axis can be obtained from the bearing sensor output, and the angle formed by the optical axis and the horizontal direction can be obtained from the tilt sensor output. The orientation of the optical axis and the tilt angle are stored together with the peripheral image captured by the imaging means 22. The imaging angle of the imaging means 22 is generally tilted to a certain degree, and a captured peripheral moving image is captured so that the portions outside the display range also include an adequately wide range.
The peripheral image captured in this manner may be displayed without modification in the display means 28, but the corrections described below are also applied in the present embodiment. (1) Correction of rotation about the optical axis. Specifically, a correction is applied so that the upward direction of the image is vertical. (2) Correction of tilt in the vertical direction of the optical axis. The image is shifted by an amount that corresponds to the angle between the optical axis and the horizontal direction, and the image is centered in the horizontal direction. The height (distance) of the optical axis from the surface of the earth is also set to the eye height of a typical person.
Since the imaging means 22 captures the peripheral image at a predetermined focal length, the points P1, P2, and P3 in front of the imaging point P can each be computed from the angles θ1, θ2, and θ3 with the optical axis AX, and it is possible to use the abovementioned angles to geometrically compute how far in meters a position in the display image is in front of the imaging position. Specifically, the point P1 in
It is thereby possible to determine the display position and the position of the display image in which to display the guide route image by how far in meters the guide route is from the imaging point, and to provide route guidance in an image that is always stable, and the computational processing for determining the display position is simplified. Specifically, the position and direction of the guide route are calculated based on the position information (current position at the time of imaging) at the time the peripheral image is captured, i.e., the time the static image is acquired, and the guide route is drawn in a predetermined position on the display image with respect to the direction of the optical axis AX.
Adequate adaptation was not made in the prior art for the display in a case in which the route forward direction is in a direction outside the range shown on the screen. A walker is not necessarily oriented in the forward direction of the guide route when at an intersection or the like. Therefore, when the peripheral image captured by the imaging means 22 is displayed on the display screen, the guide route image GRI is not necessarily included in the display region and displayed together with the peripheral image, as in
When the walker is not oriented in the direction of the guide route or the forward direction of the guide route, and the peripheral image captured by the imaging means 22 is displayed on the screen of the display means 28, in a case in which the guide route is not included in the displayed peripheral image, the guide route is not in the display range, and therefore cannot be displayed even when there is a need to display the guide route image GRI or an arrow-shaped forward direction guide image 51 (see
In the present invention, the direction in which the guide route and the forward direction thereof are present with respect to the peripheral image displayed in the display screen is determined by geometric computation from the optical axis angle and the position at which the peripheral image is captured by the imaging means 22, and the angle of a link or position of a node of the guide route. An imaging direction guide image (e.g., a triangular mark) for guiding the imaging means toward the imaging direction in which the guide route or the forward direction thereof can be displayed is thereby displayed, and the user is prompted to change the imaging direction. The user changes the imaging direction in accordance with the imaging direction guide image, and the guide route image or forward direction guide image can thereby be displayed on the peripheral image that is displayed on the display screen of the display means 28.
For example, when the guide route is to the right of the peripheral image displayed on the display screen, an imaging direction guide image 52 is displayed that indicates moving the imaging direction to the right, as shown in
When the user changes the imaging direction in accordance with the displayed imaging direction guide image (triangular mark) 52 or 53, the guide route is then included in the peripheral image displayed on the. display screen, as shown in
The guide route image GRI is displayed (
An instruction (the imaging direction guide image 53 of
When the forward direction of the guide route GR from the imaging position P is not included in the field of view e, and is to the right as viewed from the optical axis AX, an instruction (the imaging direction guide image 52 of
First, in the routine of step S101, the tilt angle detected by the orientation detection means 23 is used as a basis to detect whether the mobile terminal device 20 (optical axis of the imaging means 22) is within a predetermined angle range with respect to the horizontal direction. The predetermined angle range may include average angles used when a typical person readies a camera to capture an image.
When the tilt angle of the mobile terminal device 20 with respect to horizontal is not within the predetermined angle range (NO), the process proceeds to the routine of step S109; a normal-mode display, i.e., a display using a plan-view map, is provided, and the process returns to the routine of step S101. When the tilt angle is within the predetermined angle range with respect to horizontal (YES), the shutter of the imaging means 22 is automatically operated in the routine of step S102, and a peripheral image in the optical axis direction is captured. The drawing control means 25 then switches to display control in the live view mode in the routine of step S103.
In the routine of step S102, the image data captured by the imaging means 22 are temporarily stored by an appropriate memory means in the routine of step S104 along with the current position at the time of imaging, the tilt angle detected by the orientation detection means 23, the bearing of the optical axis (orientation of the mobile terminal device 20), and other data. The process then proceeds to the routine of step S105, and as described with reference to
In the determination routine of step S105, the process proceeds to the routine of step S106 when the forward direction of the guide route as viewed from the current position, i.e., the imaging position, is within the field of view θ. The image layout means 24 calculates the display position of the guide route image and forward direction guide image, and the drawing control means 25 displays the guide route image and forward direction guide image in the predetermined display position of the peripheral image captured by the imaging means 22, on the basis of the computation results thereof, as described with reference to
In the determination routine of step S105, the process proceeds to the routine of step S108 when the forward direction of the guide route as viewed from the imaging position is not included in the field of view 49, and is to the left as viewed from the optical axis AX; and the drawing control means 25 displays an instruction (the imaging direction guide image 53 of
A route search in the route search server 30 will next be described. The route search network database 35 is provided with road network data and transportation network data. These network data are configured as described below. For example, when the roads are composed of roads A, B, and C as shown in
Specifically, in
When the data of such a road network is route-searched as a database for route searching, a link connected from the node of the departure point to the node of the destination point is traced, the link cost is accumulated, and the route having the smallest accumulated link cost is searched and used for guidance. Specifically, when a route search is performed using node AX in
Other routes from node AX to node CY are not shown in
In contrast, transportation network data for a route search of transportation facilities are configured as described below. For example, when the transportation network data are composed of transportation lines A, B, and C as shown in
However, the link costs in a transportation network are fundamentally different from those of a road network. Specifically, the link costs in the road network are fixed and static, whereas the transportation network has a plurality of trains or aircraft (trains, aircraft, and other routes are referred to hereinafter as modes of transportation) moving through the transportation lines, as shown in
The example of the transportation network shown in
All transportation means that can be used (ridden) to travel from a departure point to a destination point must be searched, and a transportation means that satisfies the search conditions must be specified in order to search the route from a certain departure point to a certain destination point using the type of transportation network data described above.
For example, in
A guide route searched with reference to such network data is delivered to the mobile terminal device 20 along with vector map data. The map data accumulated in the map database 34 is composed of unit map data that are divided into predetermined latitude/longitude ranges, as shown in
When the mobile terminal device 20 moves and there are insufficient map data, the movement direction of the mobile terminal device 20 is determined, and the route search server 30 delivers the needed portion of unit map data. The same procedure applies when the mobile terminal device 20 requests delivery of map data for a specific point or position of a POI (Point of Interest). The guide route data are delivered together with the vector map data to the mobile terminal device 20. The mobile terminal device 20 thus receives the map data and guide route data from the route search server 30 and displays the map and the guide route in the display means 28.
The present invention can be used as a method for indicating the, direction in which the guide route can be seen even when the walker strays slightly from the route, and the guide route disappears from the display screen.
In the present embodiment, a case was described in which the peripheral image captured by the imaging means 22 is still image, but a moving image may also be used if the image processing capability of the mobile terminal device 20 is adequately high.
In the present embodiment, when the optical axis of the imaging means 22 is within a predetermined angle from horizontal, the orientation detection means 23 detects this orientation, and the shutter of the imaging means 22 operates automatically. The switch to live view mode is therefore automatic, and the map display mode is returned to when the conditions described above are not satisfied. Consequently, the switch to the live view mode for displaying the peripheral image captured by the imaging means 22 does not occur during walking, and safety is maintained.
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2006/320461 | 10/13/2006 | WO | 00 | 4/10/2009 |
