Patent Application
20170308157
The present invention relates to a head-mounted display device, a display system, a control method for the head-mounted display device, and a computer program,
JP-T-2008-539493 (Patent Literature 1) describes a system that provides an individual shopper with a personal purchasing device in a shopping venue to support an individual purchasing activity of the shopper and provides the shopper with information using a consumer interface.
However, in the technique described in Patent Literature 1, although the shopper can receive support in the shopping venue, the shopper cannot receive support in a place away from the shopping venue. When the shopper is away from the shopping venue, the shopper cannot purchase commodities in a sense of actually performing shopping in the shopping venue. In this way, under the current situation, means for enabling the shopper to receive support of purchasing in a place away from the shopping venue has not been sufficiently contrived. Note that such a problem is not limited to the support of shopping and also occurs When a shopper desires to make some request from the outside of a facility taking into account a situation in the facility.
JP-A-2011-217098 and JP-A-2014-215748 are examples of related art.
An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms.
(1) According to a first aspect of the invention, a head-mounted display device including an image display section attached to a head is provided. The head-mounted display device includes: a specific-direction detecting section configured to detect a specific direction decided according to a direction of the image display section; a wireless communication section; and a processing section configured to perform presentation of information to an external information processing device via the wireless communication section. The processing section specifies a predetermined range including the specific direction detected by the specific-direction detecting section, performs communication with a short-range wireless communication terminal present in the specified predetermined range via the wireless communication section, receives data from the short-range wireless communication terminal, and transmits presentation information based on the received data to the information processing device. With the head-mounted display device according to this aspect, it is possible to transmit, to the information processing device, the presentation information based on the data received from the short-range wireless communication terminal present in the predetermined range including the specific direction decided according to the direction of the image display section. Therefore, an operator of the information processing device can receive, while staying in a position away from a user wearing the head-mounted display device, information it the periphery of the specific direction decided according to the direction of the image display section. Therefore, with the head-mounted display device according to this aspect, it is possible to appropriately and efficiently present information in the specific direction decided according to the direction of the image display section to the operator of the information processing device present in the position away from the user.
(2) In the head-mounted display device according to the aspect, the specific direction may be a direction that the user is viewing. With the head-mounted display device according to this aspect, it is possible to appropriately and efficiently present information in a line-of-sight direction of the user to the operator of the information processing device present in the position away from the user.
(3) In the head-mounted display device according to the aspect, the head-mounted display device may further include a camera configured to pick up an image in the direction that the user is viewing. The processing section may transmit at least a part of the image picked up by the camera to the information processing device via the wireless communication section as a picked-up image for transmission. With the head-mounted display device according to this aspect, it is possible to present an outside scene viewed by the user using the image picked up by the camera. Therefore, with the head-mounted display device according to this aspect, it is possible to more appropriately and efficiently present information to the operator of the information processing device.
(4) In the head-mounted display device according to the aspect, the processing section may specify a portion overlapping the predetermined range in the picked-up image for transmission and perform image processing concerning the overlapping portion. With the head-mounted display device according to this aspect, it is possible to clearly indicate the predetermined range to the operator of the information processing device present in the position away from the user.
(5) In the head-mounted display device according to the aspect, the data received from the short-range wireless communication terminal present in the predetermined range may be data for each of types of commodities. With the head-mounted display device according to this aspect, is possible to receive the data for each of the types of the commodities from the short-range wireless communication terminal disposed for each of the types of the commodities. Consequently, it is possible to present information for each of the types of the commodities to the external information processing device. Therefore, with the head-mounted display device according to this aspect, it is possible to support purchase of the commodities by the operator of the information processing device.
(6) In the head-mounted display device according to the aspect, the processing section may receive, from the external information processing device, an order instruction for a commodity selected out of the commodities, the information of which is presented. With the head-mounted display device according to this aspect, the operator of the information processing device present in the position away from the user can order the commodity from the position away from the user.
(7) In the head-mounted display device according to the aspect, the processing section may transmit start information for performing settlement processing to the external information processing device and may receive, from the external information processing device, the order instruction together with a notification to the effect that the settlement is completed. With the head-mounted display device according to this aspect, the operator of the information processing device present in the position away from the user can also perform settlement of purchase of the commodities from the position away from the user.
(8) In the head-mounted display device according to the aspect, the external information processing device may be another head-mounted display device different from the head-mounted display device. With the head-mounted display device according to this aspect, it is possible to provide an environment of use same as an environment of use by the user to a user wearing the other head-mounted display device.
(9) According to a second aspect of the invention, a display system including: a head-mounted display device including an image display section attached to a head; and an information processing device is provided. The head-mounted display device of the display system includes: a specific-direction detecting section configured to detect a specific direction decided according to a direction of the image display section; a wireless communication section; and a processing section configured to perform presentation of information to the information processing device via the wireless communication section. The processing section specifies a predetermined range including the specific direction detected by the specific-direction detecting section, performs communication with a short-range wireless communication terminal present in the specified predetermined range via the wireless communication section, receives data from the short-range wireless communication terminal, and transmits presentation information based on the received data to the information processing device. The information processing device of the display system receives the presentation information transmitted from the head-mounted display device and performs display on the basis of the received presentation information. With the display system according to this aspect, it is possible to transmit, to the information processing device, the presentation information based on the data received from the short-range wireless communication terminal present in the predetermined range including a line-of-sight direction of a user wearing the head-mounted display device. Therefore, an operator of the information processing device can receive, while staying in a position away from the user wearing the head-mounted ay device, information in the periphery of the specific direction decided according to the direction of the image display section. Therefore, the display system according to this aspect can appropriately and efficiently present information in the specific direction decided according to the direction of the image display section to the operator of the information processing device present in the position away from the user.
The invention can also be implemented in various forms other than the head-mounted display device. The invention can be implemented as, for example, a control method for the head-mounted display device, a computer program for realizing functions of the components included in the head-mounted display device, and a recording medium having the computer program recorded therein.
The invention will be described with reference to the accompanying drawings, wherein like numbers reference like elements.
Each of the two head-mounted display devices 100 and 400 is connected to an Internet INT by wireless communication via a communication carrier BS. The communication carrier ES includes a transmission and reception antenna, a wireless base station, and an exchange. The two head-mounted display devices 100 and 400 are capable of communicating with each other through the Internet INT. The configuration of one head-mounted display device 100, that is, the head-mounted display device 100 used by the shopping supporter in the store is explained first.
The head-mounted display device 100 is a display device mounted on the head of a shopping supporter (hereinafter referred to as “user” as well) and is called head mounted display (HMD) as well. The HMD 100 is a see-through type (transmission type) head-mounted display device in which an image emerges in an outside world visually recognized through glass.
The HMD 100 includes an image display section 20 that causes the user to visually recognize an image and a control device (a controller) 10 that controls the image display section 20.
The image display section 20 is a wearing body worn on the head of the user. In this embodiment, the image display section 20 has an eyeglass shape. The image display section 20 includes a right display unit 22, a left display unit 24, right light guide plate 26, and a left light guide plate 28 in a main body including a right holding section 21, a left holding section 23, and a front frame 27.
The right holding section 21 and the left holding section 23 respectively extend backward from both end portions of the front frame 27 and, like temples of glasses, hold the image display section 20 on the head of the user. Of both the end portions of the front frame 27, an end portion located on the right side of the user in the worn state of the image display section 20 is referred to as end portion ER. An end portion located on the left side of the user is referred to as end portion EL. The right holding section 21 is provided to extend from the end portion ER of the front frame 27 to a position corresponding to the right temporal region of the user in the worn state of the image display section 20. The left holding section 23 is provided to extend from the end portion EL to a position corresponding to the left temporal region of the user in the worn state of the image display section 20.
The right light guide plate 26 and the left light guide plate 28 are provided in the front frame 27. The right light guide plate 26 is located in front of the right eye of the user in the worn state of the image display section 20 and causes the right eye to visually recognize an image. The left light guide plate 28 is located in front of the left eye of the user in the worn state of the image display section 20 and causes the left eye to visually recognize the image.
The front frame 27 has a shape obtained by coupling one end of the right light guide plate 26 and one end of the left light guide plate 28 each other. A position of the coupling corresponds to the position of the middle of the forehead of the user in the worn state of the image display section 20. In the front frame 27, a nose pad section in contact with the nose of the user in the worn state of the image display section 20 may be provided in the coupling position of the right light guide plate 26 and the left light guide plate 28. In this case, the image display section 20 can be held on the head of the user by the nose pad section and the right holding section 21 and the left holding section 23. A belt in contact with the back of the head of the user in the worn state of the image display section 20 may be coupled to the right holding section 21 and the left holding section 23. In this case, the image display section 20 can be firmly held on the head of the user by the belt.
The right display unit 22 performs display of an image by the right light guide plate 26. The right display unit 22 is provided in the right holding section 21 and located near the right temporal region of the user in the worn state of the image display section 20. The left display unit 24 performs display of an image by the left light guide plate 28. The left display unit 24 is provided in the left holding section 23 and located near the left temporal region of the user in the worn state of the image display section 20. Note that the right display unit 22 and the left display unit 24 are collectively referred to as “display driving section” as well.
The right light guide plate 26 and the left light guide plate 28 in this embodiment are optical sections (e.g., prisms) formed of light transmissive resin or the like. The right light guide plate 26 and the left light guide plate 28 guide image lights output by the right display unit 22 and the left display unit 24 to the eyes of the user. Note that dimming plates may be provided on the surfaces of the right light guide plate 26 and the left light guide plate 28. The dimming plates are thin plate-like optical elements having different transmittances depending on a wavelength region of light. The dimming plates function as so-called wavelength filters. For example, the dimming plates are disposed to cover the surface (a surface on the opposite side of a surface opposed to the eyes of the user) of the front frame 27. By appropriately selecting optical characteristics of the dimming plates, it is possible to adjust the transmittances of lights in any wavelength regions such as visible light, infrared light, and ultraviolet light. It is possible to adjust light amounts of external lights made incident on the right light guide plate 26 and the left light guide plate 28 from the outside and transmitted through the right light guide plate 26 and the left light guide plate 28.
The image display section 20 guides image lights respectively generated by the right display unit 22 and the left display unit 24 to the right light guide plate 26 and the left light guide plate 28 and causes the user to visually recognize an image (an augmented reality (AR) image) with the image lights (this is referred to as “display an image” as well). When external lights are made incident or the eyes of the user through the right light guide plate 26 and the left light guide plate 28 from the front of the user, the image lights forming the image and the external lights are made incident on the eyes of the user. Therefore, visibility of the image in the user is affected by the intensity of the external lights.
Therefore, it is possible to adjust easiness of visual recognition of the image by, for example, attaching the dimming plates to the front frame 27 and selecting or adjusting the optical characteristics of the dimming plates as appropriate. In a typical example, it is possible to select the dimming plates having light transmissivity enough for enabling the user wearing the HMD 100 to visually recognize at least an outside scene. When the dimming plates are used, it is possible to expect an effect of protecting the right light guide plate 26 and the left light guide plate 28 and suppressing damage, adhesion of stain, and the like to the right light guide plate 26 and the left light guide plate 28. The ring plates may be detachably attachable to the front frame 27 or the respective right and left light guide plates 26 and 28. A plurality of kinds of dimming plates may be replaceable and detachably attachable. The dimming plates may be omitted.
A camera 61 is disposed in the front frame 27 of the image display section 20. On the front surface of the front frame 27, the camera 61 is provided in a position for not blocking the external lights transmitted through the right light guide plate 26 and the left light guide plate 28. In the example shown in
The camera 61 is a digital camera including an image pickup device such as a CCD or a CMOS and an image pickup lens. The camera 61 in this embodiment is a monocular camera. However, a stereo camera may be adopted. The camera 61 picks up an image of an outside scene (a real space) in a front side (forward) direction of the HMD 100, that is, a direction that the user is viewing in the state in which the image display section 20 is worn. In other words, the camera 61 picks up an image in a range including a line-of-sight direction of the user. The breadth of an angle of view of the camera 61, that is, the range can be set as appropriate. In this embodiment, the breadth of the angle of view of the camera 61 is set to pick up an image of the entire field of view of the user that the user can view (visually recognize) through the right light guide plate 26 and the left light guide plate 28. The camera 61 executes the image pickup according to control by a control function section 150 (
A distance measuring sensor 62 is disposed in an upper part of the camera 61 in a coupled portion of the right light guide plate 26 and the left light guide plate 28 of the front frame 27. The distance measuring sensor 62 detects the distance to a measuring target object located in a measuring direction set in advance. The measuring direction can be set in the front side direction of the HMD 100 (a direction overlapping the image pickup direction of the camera 61). The distance measuring sensor 62 can be configured by, for example, a light emitting section such as an LED or a laser diode and a light receiving section that receives reflected light of light emitted by a light source and reflected on the measurement target object. In this case, a distance is calculated by triangulation processing or distance measurement processing based on a time difference. The distance measuring sensor 62 may be configured by, for example, an emitting section that emits ultrasound and a receiving section that receives the ultrasound reflected on the measurement target object. In this case, the distance is calculated by the distance measurement processing based on a time difference. Like the camera 61, the distance measuring sensor 62 is controlled by the control function section 150 and outputs a detection result to the control function section 150.
As components for causing the right eye RE to visually recognize an image (an AR image), the right display unit 22 includes an OLED (Organic Light Emitting Diode) unit 221 and a right optical system 251. The OLED unit 221 emits image light. The right optical system 251 includes a lens group and the like and guides image light L emitted by the OLED unit 221 to the right light guide plate 26.
The OLED unit 221 includes an OLED panel 223 and an OLED driving circuit 225 that drives the OLED panel 223. The OLED panel 223 is a self-emitting display panel configured by light emitting elements that emit lights with organic electroluminescence and respectively emit color lights of R (red), G (green), and B (blue). In the OLED panel 223, a plurality of pixels, one pixel of which is a unit including one each of R, G, and B elements, are arranged in a matrix shape.
The OLED driving circuit 22 executes, according to the control by the control function section 150 (
The right optical system 251 includes a collimate lens that changes the image light L emitted from the OLED panel 223 to a light beam in a parallel state. The image light L changed to the light beam in the parallel state by the collimate lens is made incident on the right light guide plate 26. A plurality of reflection surfaces that reflect the image light L are formed in an optical path for guiding light on the inside of the right light guide plate 26. The image light L is guided to the right eye RE side through a plurality of times of reflection on the inside of the right light guide plate 26. On the right light guide plate 26, a half mirror 261 (a reflection surface) located in front of the right eye RE is formed. After being reflected on the half mirror 261, the image light L is emitted to the right eye RE from the right light guide plate 26. The image light L forms an image on the retina of the right eye RE to cause the user to visually recognize the image.
As components for causing the left eye LE to recognize an image (an AR image), the left display unit 24 includes an OLED unit 241 and a left optical system 252. The OLED unit 241 emits the image light L. The left optical system 252 includes a lens group and the like and guides the image light L emitted by the OLED unit 241 to the left light guide plate 28. The OLED unit 241 includes an OLED panel 243 and an OLED driving circuit 245 that drives the OLED panel 243. Details of the sections are the same as the details of the OLED unit 221, the OLED panel 223, and the OLED driving circuit 225. A temperature sensor 239 is mounted on a substrate fixed to the rear surface of the OLED panel 243. Details of the left optical system 252 are the same as the details of the right optical system 251.
With the configuration explained above, the HMD 100 can function as a see-through type display device. That is, the image light L reflected on the half mirror 261 and external light OL transmitted through the right light guide plate 26 are made incident on the right eye RE of the user. The image light L reflected on a half mirror 281 and the external light OL transmitted through the left light guide plate 28 are made incident on the left eye LE of the user. In this way, the HMD 100 superimposes the image light L of the image and the external light OL processed on the inside one on top of the other and makes the image light L and the external light OL incident on the eyes of the user. As a result, for the user, an outside scene (a real world) is seen through the right light guide plate 26 and the left light guide plate 28 and an image (an AR image) by the image light L visually recognized to overlap the outside scene.
Note that the half mirror 261 and the half mirror 281 function as an “image extracting section” that reflects image lights respectively output by the right display unit 22 and the left display unit 24 and extracts an image. The right optical system 251 and the right light guide plate 26 are collectively referred to as “right light guide section” as well. The left optical system 252 and the left light guide plate 28 are collectively referred to as “left light guide section” as well. The configuration of the right light guide section and the left light guide section is not limited to the example explained above. Any system can be used as long as the right light guide section and the left light guide section form an image in front of the eyes of the user by using image light. For example, as the right light guide section and the left light guide section, a diffraction grating may be used or a semitransparent reflection film may be used.
In
The connector 46 is a jack for connecting a stereo mini-plug. The connector 46 and the control device 10 are connected by, for example, a line for transmitting analog sound signal. In the example of this embodiment shown in
The microphone 63 is disposed such that a sound collecting section of the microphone 63 faces the line-of-sight direction of the user, for example, as shown in
The control device 10 is a device for controlling the HMD 100. The control device 10 includes a lighting section 12, a touch pad 14, a direction key 16, a determination key 17, and a power switch 18. The lighting section 12 notifies an operation state (e.g., ON/OFF of a power supply) of the HMD 100 with a light emitting form of the lighting section 12. As the lighting section 12, for example, an LED (Light Emitting Diode) can be used.
The touch pad 14 detects contact operation on an operation surface of the touch pad 14 and outputs a signal corresponding to detection content. As the touch pad 14, various touch pads of various types such as an electrostatic type, a pressure detection type, and an optical type can be adopted. The direction key 16 detects pressing operation on. keys corresponding to upward, downward, left, and right directions and outputs a signal corresponding to detection content. The determination key 17 detects pressing operation and outputs a signal for determining content of operation in the control device 10. The power switch 18 detects slide operation of the switch to switch a state of the power supply of the HMD 100.
As explained above, the camera 61 is disposed in the coupling section of the right light guide plate 26 and the left light guide plate 28 in the front frame 27 of the image display section 20. The camera 61 picks up an image in the line-of-sight direction of the user (the forward direction of the user) Therefore, the optical axis of the camera 61 is set in a direction including the line-of-sight direction of the right eye RE and the left eye LE. An outside scene that the user can visually recognize in a state in which the user wears the HMD 100 is not limited to infinity. For example, when the user gazes a target object OB with both the eyes, the line of sight of the user is directed to the target object OB as indicated by signs RD and LD in the figure. In this case, the distance from the user to the target object OB is often approximately 30 cm to 10 m and is more often 1 m to 4 m. Therefore, concerning the HMD 100, standards of an upper limit and a lower limit of the distance from the user to the target object OB during normal use may be decided. The standards may be calculated in advance and preset in the MMD 100 or the user may set the standards. The optical axis and the angle of view of the camera 61 are desirably set such that the target object OB is included in the angle of view when the distance to the target object OB during the normal use is equivalent to the set standards of the upper limit and the lower limit.
Note that, in general, the angular field of view of the human is approximately 200 degrees in the horizontal direction and approximately 125 degrees in the vertical direction. An effective field of view excellent in an information reception ability in the angular field of view is approximately 30 degrees in the horizontal direction and 20 degrees in the vertical direction. A stable gazing field in which a gazing point of the human can be quickly and stably seen is considered to be approximately 60 to 90 degrees in the horizontal direction and approximately 45 to 70 degrees in the vertical direction. In this case, when the gazing point is the target object OB (
The angle of view θ of the camera 61 in this embodiment is set such that the camera 61 can perform image pickup in a range wider than the field of view of the user. The angle of view θ of the camera 61 is desirably set such that the camera 61 can perform image pickup in a range wider than at least the effective field of view of the user and is more desirably set such that the camera 61 can perform image pickup in a range wider than the actual field of view. The angle of view θ of the camera 61 is more desirably set such that the camera 61 can perform image pickup in a range wider than the stable gazing field of the user and most desirably set such that the camera 61 can perform image pickup in a range wider than the angular field of view of both the eyes of the user. Therefore, the camera 61 may include a so-called wide angle lens as an image pickup lens to be capable of performing image pickup at a wider angle of view. The wide angle lens may include lenses called super wide angle lens and semi-wide angle lens. The camera 61 may include a single focus lens, may include a zoom lens, and may include a lens group consisting of a plurality of lenses.
The storing section includes a memory 118 and a nonvolatile storing section 121. The memory 118 configures a work area where computer programs executed the main processor 140 and data processed by the main processor 140 are temporarily stored. The nonvolatile storing section 121 is configured by a flash memory or an eMMC (embedded Multi Media Card) The nonvolatile storing section 121 stores computer programs executed by the main processor 140 and various data processed by the main processor 140. In this embodiment, the storing sections are mounted on the controller board 120.
The input/output section includes the touch pad 14 and an operation section 110. The operation section 110 includes the direction key 16, the determination key 17, and the power switch 18 included in the control device 10. The main processor 140 controls the input/output sections and acquires signals output from the input/output sections.
The sensors include a six-axis sensor 111, a magnetic sensor 113, and a GPS (Global Positioning System) receiver 115. The six-axis sensor 111 is a motion sensor (an inertial sensor) including a three-axis acceleration sensor and a three-axis gyro (angular velocity) sensor. As the six-axis sensor 111, an IMU (Inertial Measurement Unit) obtained by converting these sensors into a module may be adopted. The magnetic sensor 113 is, for example, a three-axis terrestrial magnetism sensor. The GPS receiver 115 includes a not-shown GPS antenna, receives a radio signal transmitted from the GPS satellite, and detects a coordinate of the present position of the control device 10. These sensors (the six-axis sensor 111, the magnetic sensor 113, and the GPS receiver 115) output detection values to the main processor 140 according to sampling frequency designated in advance. The sensors may output the detection values at timing corresponding to an instruction from the main processor 140.
The interface includes a wireless communication section 117, a sound codec 180, an external connector 184, an external memory interface 186, a USB (Universal Serial Bus) connector 188, a sensor hub 192, an FPGA 194, and an interface 196. These components function as interfaces with the outside. The wireless communication section 117 executes wireless communication between the HMD 100 and an external device. The wireless communication section 117 includes an antenna, an RF circuit, a baseband circuit, and a communication control circuits not shown in the figure. Alternatively, the wireless communication section 117 is configured as a device obtained by integrating these sections. The wireless communication section 117 performs, for example, wireless communication conforming to a wireless LAN including Wi-Fi (registered trademark) Bluetooth (registered trademark), and iBeacon (registered trademark).
The sound codec 180 is connected to the sound interface 182 and performs encoding and decoding of sound signals input and output via the sound interface 182. The sound interface 182 is an interface for inputting and outputting sound signals. The sound codec 180 may include an A/D converter that performs conversion from an analog sound signal into digital sound data and a D/A converter that performs conversion opposite to the conversion. The HMD 100 in this embodiment outputs sound from the right earphone 32 and the left earphone 34 and collects sound with the microphone 63. The sound codec 180 converts digital sound data output by the main processor 140 into an analog sound signal and outputs the analog sound signal via the sound interface 182. The sound codec 180 converts the analog sound signal input to the sound interface 182 into digital sound data and outputs the digital sound data to the main processor 140.
The external connector 184 is a connector for connecting an external device (e.g., a personal computer, a smartphone, or a game machine), which communicates with the main processor 140, to the main processor 140. The external device connected to the external connector 184 can be a supply source of contents and can be used for debugging of computer programs executed by the main processor 140 and collection of an operation log of the HMD 100. Various forms can be adopted as the external connector 184. As the external connector 184, for example, interfaces adapted to wired connection such as a USB interface, a micro USB interface, and an interface for a memory card and interfaces adapted to wireless connection such as a wireless LAN interface and a Bluetooth interface can be adopted.
The external memory interface 186 is an interface to which a portable memory device can be connected. The external memory interface 186 includes, for example, a memory card slot into which a card-type recording medium is inserted to perform reading and writing of data and an interface circuit. A size, a shape, a standard, and the like of the card-type recording medium can be selected as appropriate. The USB connector 188 is an interface to which a memory device, a smartphone, a cellular phone, a personal computer, and the like conforming to the USB standard can be connected. The USB connector 188 includes, for example, a connector conforming to the USB standard and an interface circuit. A size, a shape, a version of the USB standard, and the like of the USB connector 188 can be selected as appropriate.
The HMD 100 includes a vibrator 19. The vibrator 19 includes a motor and an eccentric rotor not shown in the figure and generates vibration according to control by the main processor 140. The HMD 100 causes the vibrator 19 to generate vibration in a predetermined vibration pattern, for example, when operation on the operation section 110 is detected and when the power supply of the HMD 100 is turned on and off.
The sensor hub 192 and the FPGA 194 are connected to the image display section 20 via the interface (I/F) 196. The sensor hub 192 acquires detection values of the various sensors included in the image display section 20 and outputs the detection value to the main processor 140. The FPGA 194 executes processing of data transmitted and received between the main processor 140 and the sections of the image display section 20 and transmission via the interface 196. The interface 196 is connected to the right display unit 22 and the left display unit 24 of the image display section 20. In the example in this embodiment, the connection cable 40 is connected to the left holding section 23. A wire connected to the connection cable 40 is laid inside the image display section 20. The right display unit 22 and the left display unit 24 are connected to the interface 196 of the control device 10.
The power supply section 130 includes a battery 132 and a power-supply control circuit 134. The power supply section 130 supplies electric power for the control device 10 to operate. The battery 132 is a rechargeable battery. The power-supply control circuit 134 performs detection of residual capacity of the battery 132 and control of charging to an OS 143. The power-supply control circuit 134 is connected to the main processor 140 and outputs a detection value of the residual capacity of the battery 132 and a detection value of the voltage of the battery 132 to the main processor 140. Note that electric power may be supplied from the control device 10 to the image display section 20 on the basis of the electric power supplied by the power supply section 130. A supply state of the electric power from the power supply section 130 to the sections of the control device 10 and the image display section 20 may be controllable by the main processor 140.
The right display unit 22 includes a display unit board 210, the OLED unit 221, the camera 61, an illuminance sensor 65, an LED indicator 67, and the temperature sensor 217. An interface (I/F) 211 connected to the interface 196, a receiving section (Rx) 213, and an EEPROM (Electrically Erasable Programmable Read-Only Memory) 215 are mounted on the display unit board 210. The receiving section 213 receives data input from the control device 10 via the interface 211. When receiving image data of an image displayed on the OLED unit 221, the receiving section 213 outputs the received image data to the OLED driving circuit 225 (
The EEPROM 215 stores various data in a form readable by the main processor 140. The EEPROM 215 stores, for example, data concerning emission characteristics and display characteristics of the OLED units 221 and 241 of the image display section 20 and data concerning sensor characteristics of the right display unit 22 or the left display unit 24. Specifically, the EEPROM 215 stores, for example, parameters related to gamma correction of the OLED units 221 and 241 and data for compensating for detection values of the temperature sensors 217 and 239 explained below. These data are generated by inspection during factory shipment of the HMD 100 and written in the EEPROM 215. After the shipment, the main processor 140 reads the data the EEPROM 215 and uses the data for various kinds of processing.
The camera 61 executes image pickup according to a signal input via the interface 211 and outputs a signal representing picked-up image data or an image pickup result to the control device 10. As shown in
The temperature sensor 217 detects temperature and outputs a voltage value or a resistance value corresponding to the detected temperature. The temperature sensor 217 is mounted on the rear surface side of the OLED panel 223 (
The left display unit 24 includes a display unit board 230, the OLED unit 241, and the temperature sensor 239. An interface (I/F) 231 connected to the interface 196, a receiving section (Rx) 233, a six-axis sensor 235, and a magnetic sensor 237 are mounted on the display unit board 230. The receiving section 233 receives data input from the control device 10 via the interface 231. When receiving image data of an image displayed on the OLED unit 241, the receiving section 233 outputs the received image data to the OLED driving circuit 245 (
The six-axis sensor 235 is a motion sensor (an inertial sensor) including a three-axis acceleration sensor and a three-axis gyro (angular velocity) sensor. As the six-axis sensor 235, an IMU obtained by converting the sensors into a module may be adopted. Since the six-axis sensor 235 is provided in the image display section 20, when the image display section 20 is worn on the head of the user, the six-axis sensor 235 detects a movement of the head of the user. The direction of the image display section 20 is seen from the detected movement of the head of the user. The magnetic sensor 237 is, for example, a three-axis terrestrial magnetism sensor. The temperature sensor 239 detects temperature and outputs a voltage value or a resistance value corresponding to the detected temperature. The temperature sensor 239 is mounted on the rear surface side of the OLED panel 243 (
The camera 61, the illuminance sensor 65, and the temperature sensor 217 of the right display unit 22 and the six-axis sensor 235, the magnetic sensor 237, and the temperature sensor 239 of the left display unit 24 are connected to the sensor hub 192 of the control device 10. The sensor hub 192 performs setting and initialization of sampling cycles of the sensors according to the control by the main processor 140. The sensor hub 192 executes energization to the sensors, transmission of control data, acquisition of detection values, and the like according to the sampling cycles of the sensors. The sensor hub 192 outputs, at timing set in advance, detection values of the sensors included in the right display unit 22 and the left display unit 24 to the main processor 140. The sensor hub 192 may include a cache function for temporarily retaining the detection values of the sensors. The sensor hub 192 may include a function of converting a signal format and a data format of the detection values of the sensors (e.g., a function of converting the signal format and the data format into unified formats). The sensor hub 192 starts and stops energization to the LED indicator 67 according to the control by the main processor 140 or extinguish the LED indicator 67.
Various data served for processing in the control function section 150 are stored in the storing function section 122. Specifically, setting data 123 and content data 124 are stored in the storing function section 122 in this embodiment. The setting data 123 includes various setting values related to the operation of the HMD 100. For example, the setting data 123 includes parameters, determinants, operational expressions, and LUTs (Look Up tables) used by the control function section 150 in controlling the HMD 100.
The content data 124 includes data (image data, video data, sound data, etc.) of contents including images and videos displayed by the image display section 20 according to the control by the control function section 150. Note that the content data 124 may include data of bidirectional contents. The bidirectional contents mean contents of a type for acquiring operation of the user with the operation section 110, executing, with the control function section 150, processing corresponding to acquired operation content, and displaying contents corresponding to processing content on the image display section 20. In this case, the data of the contents can include image data of a menu screen for acquiring the operation of the user and data for deciding processing corresponding to items included in the menu screen.
The control function section 150 executes various kinds of processing using the data stored by the storing function section 122 to thereby execute functions of the OS 143, an image processing section 145, a display control section 147, an image-pickup control section 149, an input/output control section 151, a communication control section 153, shopping-support processing section 155 In this embodiment, the functional sections other than the OS 143 are configured as computer programs executed on the OS 143.
The image processing section 145 generates, on the basis of image data of an image or a video displayed by the image display section 20, a signal transmitted to the right display unit 22 and the left display unit 24. The signal generated by the image processing section 145 may be a vertical synchronization signal, a horizontal synchronization signal, a clock signal, an analog image signal, or the like. Besides being realized by the main processor 140 executing a computer program, the image processing section 145 may be configured by hardware (e.g., a DSP (Digital Signal Processor)) separate from the main processor 140.
Note that the image processing section 145 may execute resolution conversion processing, image adjustment processing, 2D/3D conversion processing, and the like according to necessity. The resolution conversion processing is processing for converting the resolution of image data into resolution suitable for the right display unit 22 and the left ay unit 24. The image adjustment processing is processing for adjusting the luminance and the chroma of the image data. The 2D/3D conversion processing is processing for generating two-dimensional image data from three-dimensional image data or generating three-dimensional image data from two-dimensional image data. When executing these kind of processing, the image processing section 145 generates a signal for displaying an image on the basis of the image data after the processing and transmits the signal to the image display section 20 via the connection cable 40.
The display control section 147 generates a control signal for controlling the right display unit 22 and the left splay unit 24 and controls, with the control signal, generation and emission of image lights by the right display unit 22 and the left display unit 24 Specifically, the display control section 147 controls the OLED driving circuits 225 and 245 to execute display of an image by the OLED panels 223 and 243. The display control section 147 performs, on the basis of a signal output by the image processing section 145, control of timing of drawing on the OLED panels 223 and 243 by the OLED driving circuits 225 and 245, control of the luminance of the OLED panels 223 and 243, and the like.
The image-pickup control section 149 controls the camera 61 to execute image pickup, generates picked-up image data, and causes the storing function section 122 to temporarily store the picked-up image data. When the camera 61 is configured as a camera unit including a circuit that generates picked-up image data, the image-pickup control section 149 acquires the picked-up image data from the camera 61 and causes the storing function section 122 to temporarily store the picked-up image data.
The input/output control section 151 controls the touch pad 14 (
The shopping-support processing section 155 is a function realized according to an application program running on the OS 143. The shopping-support processing section 155 specifies a predetermined range including a detected line-of-sight direction of the user, performs communication with a BLE (Bluetooth Low Energy) terminal present in the specified predetermined range via the wireless communication section 117 (
As explained above, the HMD 100 is used by the shopping supporter in the store. The other HMD 400 is used by the shopping supported person in the home away from the store. In the following explanation, the HMD 100 used by the shopping supporter is referred to as “supporter HMD 100” and the HMD 100 used by the shopping supported person is referred to as “supported person HMD 400”. The supported person HMD 400 is the same model as the supporter HMD 100. Only a part of installed application programs is different. Since a part of the application programs is different, in the supported person HMD 400, the control function section of the control device 10 is different. Note that components of the supported person HMD 400 same as the components of the supporter HMD 100 are explained below using reference numerals and signs same as the reference numerals and signs of the supporter HMD 100.
The shopping processing section 455 is a function realized according to an application program running on the OS 143. The shopping processing section 455 demands shopping support and displays presentation information transmitted from the supporter HMD 100. Details of the shopping processing section 455 are explained below.
The store receiving the demand for the shopping support instructs the shopping supporter (hereinafter simply referred to as “supporter” as well) wearing the supporter HMD 100, who is on standby in advance in the store, to perform the shopping support. The instructed supporter establishes communication between the supporter HMD 100 and the supported person HMD 400 and then transmits, to the supported person HMD 400 worn by the shopping supported person, information indicating that the shopping support is entrusted (step P2). The communication between the supporter HMD 100 and the supported person HMD 400 includes not only data transfer but also a voice call.
Further, the supporter HMD 100 transmits a picked-up image picked up by the camera 61 to the supported person RED 400 (step P3).
The supported person HMD 400 receives the picked-up image transmitted from the supporter HMD 100 and displays the received picked-up image (step P4). Specifically, the supported person HMD 400 controls the image processing section 145 (
The supported person views the inside of the store in order while gazing at commodities. At this point, the supported person transmits an instruction to the supporter HMD 100 using the supported person HMD 400 (step P5). Specifically, the supported person transmits, with a voice call, information indicating that the supported person moves to a desired selling space to the supporter HMD 100. For example, the supported person transmits an instruction such as “I want fruits”.
The instructed supporter moves on the sis of the instruction (step P6). For example, the supporter moves to the fruit selling space. Subsequently, the supporter HMD 100 acquires presentation information for each of types of commodities around the line-of-sight direction of the supporter (step P7) and transmits the acquired presentation information to the supported person HMD 400 (step P8). It is explained below how the presentation information is acquired.
The supported person HMD 400 receives the presentation information transmitted from the supporter HMD 100 and displays the presentation information (step P9). Specifically, the supported person HMD 400 controls the image processing section 145 (
The supported person wearing the supported person HMD 400 views the displayed presentation information and orders a commodity of a favorite type. The supported person transmits, for example, with a voice call, the order of the commodity to the supporter HMD 100.
When the processing is started, first, the main processor 140 of the supporter HMD 100 establishes communication between the supporter HMD 100 and the supported person HMD 400 and then transmits, to the supported person HMD 400 worn by the shopping supported person, a message to the effect that the shopping support is entrusted (step S110). This processing is equivalent to step P2 in
Subsequently, the main processor 140 starts the camera 61 (step S120) and transmits an image picked up by the camera 61 to the supported person HMD 400 (step S130). As explained above, the image picked up by the camera 61 is obtained by performing image pickup in a range wider than the field of view of the supporter. Therefore, the main processor 140 deletes the periphery of the picked-up image to match the picked-up image to the field of view of the supporter and then transmits the picked-up image to the supported person HMD 400. That is, an image matching the field of view of the supporter is transmitted to the supported person HMD 400. The processing in step S130 is equivalent to step P3 in
Subsequently, when an instruction is transmitted from the supported person HMD 400, the main processor 140 receives the instruction (step S140). For example, the main processor 140 receives an instruction such as “I want fruits”. This processing is processing on the supporter HMD 100 side corresponding to step P5 in
Referring back to
Subsequently, the main processor 140 specifies a predetermined range including the line-of-sight direction calculated in step S150 (step S160). In this embodiment, the main processor 140 specifies one point in the line-of-sight direction and specifies a predetermined range centering on the one point. When an object is present in the line-of-sight direction, the “one point in the line-of-sight direction” is a point on the surface of the object crossing the line-of-sight direction. When an object is absent in the line-of-sight direction, the “one point in the line-of-sight direction” is a point away from the user by a predetermined distance (e.g., 5 m) on the line-of-sight direction.
Note that the predetermined range VA does not need to be limited to the predetermined range. For example, the predetermined range may be displayed on the image display section 20. The range can be enlarged and reduced or moved on the basis of an operation command of the user from the control device 10.
Referring back to
In the nonvolatile storing section 121 (
Subsequently, the main processor 140 starts the microphone 63 (
After the execution of step S200, the main processor 140 returns the processing to step S130 and repeatedly executes the processing in step S130 and subsequent steps. The main processor 140 executes the repetition until the shopping ends.
Further, in the field view VR2a, presentation information concerning the fruit groups C2 to C4 included in the predetermined range VA (see
Further, in the field of view VR2a, presentation information concerning golf club groups included in a predetermined range (not shown in the figure) including the line-of-sight direction, in the example shown in the figure, a group C12 of “irons” and a group C13 of “drivers” are displayed. The presentation information is created on the basis of data of a set transmitted from the supporter HMD 100. Specifically, the presentation information included in the data of the set is displayed in positions associated with the positions on a picked-up image of BLE terminal 602 and 603 included in the data of the set. Specifically, the position of a golf club group corresponding to the BLE terminal 602 is specified by pattern recognition of the image on the basis of the position on the picked-up image of the BLE terminal 602 of the group C12 of irons. Presentation image M12 is displayed with an upper part of the specified position set as an “associated position”. Presentation information M13 obtained from the BLE terminal 603 of the group C13 of drivers is also displayed in a position based on the position of the BLE terminal 603.
With the supporter HMD 100 in this embodiment configured as explained above, the presentation information based on the commodity attribute data received from the BLE terminals 502 to 504 present in the predetermined range VA including the one point VP in the line-of-sight direction of the user can be transmitted to the supported person HMD 400. Therefore, the supported person wearing the supported person HMD 400 can receive information around the line-of-sight direction of the user wearing the supporter HMD 100 while staying in a position away from the user. Therefore, the supporter HMD 100 in this embodiment can appropriately and efficiently present information in the line-of-sight direction to the supported person present in the position away from the user.
In this embodiment, since the information processing device used by the supported person is the HMD, the supported person can visually recognize an image coinciding with an image visually recognized by the supporter who uses the supporter HMD 100. Therefore, the supported person can purchase a commodity in feeling of actually performing shopping in the store. In particular, in this embodiment, since the HMD is a binocular type, the supported person can share, with the supporter, stereoscopic 3D feeling such as depth feeling.
In the HMD 100X, the telephotographic camera 710 is provided in the right holding section 21. Specifically, the telephotographic camera 71 is provided along the longitudinal direction of the right holding section 21. The telephotographic camera 710 can photograph at least a photographing range of the camera 61 provided on the front surface of the front frame 27. The telephotographic camera 710 can move and enlarge a focus area. The movement and the enlargement of the focus area are operated from a remote place by the HMD 400X. The telephotographic camera 710 may be provided in the left holding section 23 instead of the right holding section 21. The telephotographic camera 710 does not need to be limitedly provided in the right holding section 21 or the left holding section 23 and may be provided in any position of the image display section 20 as long as the telephotographic camera 710 can photograph the photographing range of the camera 61.
The MMD 400X used by the shopping supported person is different from the HMD 400 in the first embodiment in that the telephotographic camera 710 of the HMD 100X can be operated from a remote plate and order processing explained below can be executed. Otherwise, the MMD 400X is the same as the HMD 400. Components of the HMD 400X same as the components of the HMD 400 in the first embodiment are denoted by reference numerals and signs same as the reference numerals and signs of the components of the MMD 400. Explanation of the components is omitted.
In the second embodiment, the configurations of commodities displayed in a store are different from the configurations in the first embodiment. Specifically, QR codes (registered trademark) are attached to the commodities in the second embodiment. In the QR code, start information for starting a settlement application gram (e.g., a URL of a Web page for performing settlement processing) and commodity information including a price are coded and recorded.
When determining in step S310 that the notification of the shift to the order mode is received, the main processor receives a cursor position on the display screen in the supported person HMD 400X from the supported person HMD 400X (step S320). Subsequently, the main processor starts photographing by the telephotographic camera 710 and adjusts a focus area of the telephotographic camera 710 according to the received cursor position (step S330). That is, the focus area of the telephotographic camera 710 is remotely operated by the supported person who uses the supported person HMD 400X.
Subsequently, the main processor determines whether the QR code 800 can be read from a photographed image of the telephotographic camera 710 (step S340). When determining that the QR code 800 can not be read, the main processor returns the processing to step S320 and repeats the processing in steps S320 to S340. That is, the main processor waits for the QR code 800 to be pointed by the supported person who uses the supported person HMD 400X. When the QR code 800 can be read in step S340, the main processor advances the processing to step S350.
In step S350, the main processor performs processing for reading the QR code 800 from the photographed image obtained by the telephotographic camera 710. As a result, the main processor can acquire start information of a settlement application program and commodity information including a price recorded in QR code 800. Thereafter, the main processor transmits the acquired start information and commodity information to the supported person HMD 400X (step S360).
The supported person HMD 400X receives the start information and the commodity information transmitted from the supporter HMD 100X and performs settlement processing on the basis of these kinds of information. Specifically, the supported person HMD 400X starts the settlement application program on the basis of the start information and executes settlement according to credit card information registered in the settlement application program in advance. When the settlement is completed, the supported person HMD 400X transmits a notification to the effect that the settlement is completed to the supporter HMD 100X.
After the execution of step S360, the main processor of the supporter HMD 100X determines whether the settlement is completed on the supported person HMD 400X side (step S370) Specifically, the main processor performs the determination according to whether the notification to the effect that the settlement is completed is transmitted from the supported person HMD 400X. When determining that the settlement is not completed, the main processor returns the processing to step S320 and executes the processing again according to reception of a cursor position.
When determining in step S370 that the settlement is completed, the main processor causes the image display section 20 to display commodity information concerning a commodity for which the settlement is completed (step S380) Consequently, the supporter wearing the supporter HMD 100X can receive an order instruction for the commodity from the supported person HMD 400X.
After the execution of step S380, the main processor advances the processing to “return” and once ends the order processing. When determining in step S310 that the notification of the shift to the order mode is not received, the main processor also advances the processing to “return” and once ends the order processing.
With the supporter HMD 100X in the second embodiment configured as explained above, as in the first embodiment, it is possible to appropriately and efficiently present information in the line-of-sight direction to the supported person present in a position away from the user. Further, with the supporter HMD 100X in the second embodiment, it is possible to easily receive an order instruction for a commodity. On the other hand, with the supported person HMD 400X cooperating with the supporter HMD 100X, it is possible to easily perform processing up to settlement of the commodity.
Note that, in the order processing in the second embodiment, the settlement is performed for each of commodities. However, the settlement may be performed collectively for a plurality of commodities.
Note that the invention is not limited to the embodiments and modifications of the embodiments. It is possible to carry out the invention in various forms without departing from the spirit of the invention. For example, modifications explained below are also possible.
In the embodiments and the modifications, the specific-direction detecting section is configured by the six-axis sensor 235 equipped in the image display section 20. However, the specific-direction detecting section is not limited to this and can be variously modified. For example, the specific-direction detecting section may be a camera for eyeball photographing. The line-of-sight direction of the user is detected by picking up, using the camera for eyeball photographing, an image of the left and right eyeballs of the user in a state in which the head-mounted display device is mounted and analyzing an obtained image of the eyeballs. For example, the line-of-sight direction is detected by calculating a center position of the pupil from the eyeball image. In the analysis of the eyeball image, slight shakes and flicks of the eyes that involuntarily occur even if a human believes that the human is staring a gazing point may be taken into account.
In the embodiments and the modifications, the rectangular parallelepiped region centering on the one point VP (see
In the embodiments and the modifications, the predetermined range VA is specified on the basis of the line-of-sight direction. However, the predetermined range VA does not always need to be specified on the basis of the line-of-sight direction. Specifically, the predetermined range VA may be specified on the basis of a direction deviating from the line-of-sight direction. The predetermined range VA may be specified on the basis of any direction as long as the direction is a specific direction decided according to the direction of the image display section 20.
In the embodiments and the modifications, the display system is suitable for the use for supporting the shopping of fruits, golf clubs, and the like. However, the use of the display system is not limited to this and can be variously modified. For example, the use may be a use for borrowing articles and a use for discarding articles. In short, the display system can be applied when a user desires to make some request for articles from the outside of a shopping venue taking into account a state in the shopping venue. Further, a place of shopping is not limited to the shopping venue. The display system can also be applied when a user performs shopping in other places such as a stand and stall. For example, when a friend travels wearing an HMD and performs shopping in a souvenir shop, a stand, a stall, and the like, the invention can be applied assuming that the MMD worn by the friend is the supporter HMD.
In the embodiments and the modifications, the commodity name, the producing district, the price, the right time for eating, the characteristic, and the like are displayed as the presentation information. However, the presentation information is not limited to these kinds of information and can be variously modified. For example, in the case of foods, a calorie, an ingredient, a content, a unit price per one milliliter, traceability information, and the like may be displayed as the presentation information. Traceability means clarifying processes of cultivation and raising to processing, manufacturing, circulation, and the like in order to secure safety of foods. In the case of clothes and shoes, a size, a color, a stock, a target age group (e.g., married women or young people), and the like may be displayed as the presentation information. In the case of sporting goods, a target skill (e.g., experts, beginners, or ladies) may be added as the presentation information. Position information, store information, settlement information (a point card or a credit card), and the like may be added to the presentation information.
In the embodiments and the modifications, the image picked up by the camera 61 is transmitted to the supported person HMD 400 after deleting the periphery of the picked-up image to match the picked-up image to the field of view of the supporter. On the other hand, in a picked-up image obtained by deleting the periphery of the picked-up image (a picked-up image for transmission), a portion overlapping the predetermined range VA specified in step S160 may be specified. Processing for increasing brightness may be performed on the specified overlapping portion. An image after the processing may be transmitted to the supported person HMD 400. Note that processing of the image is not limited to the processing for increasing brightness and may be, for example, processing for providing a frame body around the image. With this configuration, it is possible to cause the supported person HMD 400 to display the predetermined range VA in the picked-up image for transmission while highlighting the predetermined range VA compared with other portions.
In the embodiments and the modifications, the supporter and the supported person respectively wear the HMD 100 and the HMD 400. However, the HMD 400 on the supported person side can also be information processing devices of other types such as a tablet computer, a smartphone, a personal computer, a projector, and a television.
In the embodiments and the modifications, the short-range wireless communication terminal is the BLE terminal. On the other hand, as a modification, the BLE terminal may be another short-range wireless communication terminal such as a wireless LAN terminal or an infrared communication terminal.
In the embodiments and the modifications, the wireless communication section 117 includes both of the function of performing wireless communication with the external HMD and the function of performing wireless communication with the BLE terminal functioning as the short-range wireless communication terminal. On the other hand, the functions of the wireless communication section may be performed in separate devices. With this configuration, the separate devices are equivalent to the “wireless communication section” in the first aspect of the invention described in the summary.
In the embodiments and the modifications, apart of the components realized by hardware may be replaced with software. Conversely, a part of the components realized by software may be replaced with hardware.
In the second embodiment and the modifications of the second embodiment, the HMD includes the telephotographic camera 710 separately from the camera 61 as the camera for QR code reading. However, the camera 61 may be replaced with a high-performance camera that can move and enlarge a focus area. The reading of the QR code may be performed by the camera 61. Further, a large number of telephotographic cameras may be set in advance in the shopping venue. Picked-up images may be read from the telephotographic cameras by radio and QR codes may be read from the picked-up images.
In the second embodiment and the modifications of the second embodiment, the telephotographic camera 710 is operated from the supported person HMD 400X side according to the movement of the cursor on the see-through display screen. On the other hand, as a modification, the position of the display screen may be tapped by a fingertip. The “tap” is operation for putting (placing) a fingertip of a hand on an image element and pointing the image element in such a manner as to press the image element. The movement of the fingertip can be detected from picked-up image obtained by controlling a camera to execute image pickup. With this configuration, it is also possible to remotely operate the telephotographic camera 710 from supported person HMD 400X side.
In the embodiments and the modifications, the supported person wearing the supported person HMD views the presentation information obtained from the BLE terminal and performs the processing for ordering a favorite commodity. On the other hand, the supported person may check a stock of a target commodity from the supported person HMD side before ordering the target commodity. For example, the supported person may ask the supporter HMD with voice such as “do you have a number 5 iron?” or “do you have a number 4 iron?”.
In the second embodiment and the modifications of the second embodiment, the settlement is performed using the QR code attached to the commodity. On the other hand, as a modification, settlement concerning a commodity may be performed on the supporter side by performing communication between an NFC (Near Field Communication) built in a commodity and the supporter HMD mounted with an NFC.
In the embodiments, the configuration of the HMD is illustrated. However, the configuration of the HMD can be optionally decided without departing from the spirit of the invention. For example, addition, deletion, conversion, and the like of components can be performed.
In the embodiments, the HMD 100 of a so-called transmission type is explained in which external light is transmitted through the right light guide plate 26 and the left light guide plate 28. However, the invention can also be applied to the HMD 100 of a so-called non-transmission type that displays an image in a state in which an outside scene cannot be transmitted. In these HMDs 100, besides the AR (Augmented Reality) display for displaying an image to be superimposed on the real space explained in the embodiments, MR (Mixed Reality) display for displaying a picked-up image of the real space and a virtual image in combination or VR (Virtual Reality) display for displaying a virtual space can also be performed.
In the embodiment, the functional sections of the control device 10 and the image display section 20 are explained. However, the functional sections can be optionally changed. For example, forms described below may be adopted. A form in which the storing function section 122 and the control function section 150 are mounted on the control device 10 and only a display function is mounted on the image display section 20. A form in which the storing function section 122 and the control function section 150 are mounted on both of the control device 10 and the image display section 20. A form in which the control device 10 and the image display section 20 are integrated. In this case, for example, all of the components of the control device 10 are included in the image display section 20. The image display section 20 is configured as a wearable computer of an eyeglass type. A form in which a smartphone or a portable game machine is used instead of the control device 10. A form in which the control device 10 and the image display section 20 are connected by wireless communication and the connection cable 40 is removed. In this case, for example, power feed to the control device 10 and the image display section 20 may also be carried out by radio.
In the embodiments, the configuration of the control device is illustrated. However, the configuration of the control device can be optionally decided without departing from the spirit of the invention.
For example, addition, deletion, conversion, and the like of components can be performed.
In the embodiments, an example of the inputting means included in the control device 10 is explained. However, the control device 10 may be configured by omitting an illustrated part of the inputting means. The control device 10 may include other inputting means not explained above. For example, the control device 10 may include an operation stick, a keyboard, and a mouse. For example, the control device 10 may include inputting means for interpreting a command associated with a movement of the body of the user or the like. The movement of the body of the user or the like can be acquired by, for example, line-of-sight detection for detecting a line of sight, gesture detection for detecting a movement of a hand, a footswitch for detecting a movement of a foot, and the like. Note that the line-of-sight detection can be realized by, for example, a camera that performs image pickup on the inner side of the image display section 20. The gesture detection can realized by, for example, analyzing images photographed by the camera 61 over time.
In the embodiments, the control function section 150 operates according to the execution of the computer program in the storing function section 122 by the main processor 140. However, the control function section 150 can adopt various configurations. For example, the computer may be stored in, instead of the storing function section 122 or in addition to the storing function section 122, the nonvolatile storing section 121, the EEPROM 215, the memory 118, and other external storage devices (including storage devices such as USB memories inserted into various interfaces and an external device such as a server connected via a network). The functions of the control function section 150 may be realized using ASICs (Application Specific Integrated Circuits) designed to realize the functions.
In the embodiments, the configuration of the image display section is illustrated. However, the configuration of the image display section can be optionally decided without departing from the spirit of the invention. For example, addition, deletion, conversion, and the like of components can be performed.
The OLED unit 241a corresponding to the left eye LE includes an OLED panel 243a that emits white light and the OLED driving circuit 245 that drives the OLED panel 243a to emit light. A modulating element 247 (a modulating device) is disposed between the OLED panel 243a and the left optical system 252. The modulating element 247 is configured by, for example, a transmissive liquid crystal panel and modulates the light emitted by the OLED panel 243a to generate the image light L. The image light L transmitted through the modulating element 247 to be modulated is guided to the left eye LE by the left light guide plate 28. The modulating elements 227 and 247 are connected to a not-shown liquid crystal driver circuit. The liquid crystal driver circuit (a modulating-device driving section) is mounted on, for example, a substrate disposed near the modulating elements 227 and 247.
In the image display section in the modification, the right display unit 22 and the left display unit 24 are respectively configured as video elements including the OLED panels 223a and 243a functioning as light source sections and the modulating elements 227 and 247 that modulate lights emitted by the light source sections and output image lights including a plurality of color lights. Note that the modulating devices that modulate the lights emitted by the OLED panels 223a and 243a are not limited to the configuration in which the transmissive liquid crystal panel is adopted. example, a reflective liquid crystal panel may be used instead of the transmissive liquid crystal panel. A digital micro-mirror device may be used. The HMD 100 may be the HMD 100 of a laser retinal projection type.
In the embodiment, the image display section 20 of the eyeglass type is explained. However, the form of the image display section 20 can be optionally changed. For example, the image display section 20 may be worn like a cap or may be incorporated in a body protector such as a helmet. The image display section 20 may be configured as a HUD (Head Up Display) mounted on vehicles such as an automobile and an airplane or other transportation means.
In the embodiment, as the optical system that guides the image light to the eyes of the user, the configuration is illustrated in which virtual images are formed by the half mirrors 261 and 281 in parts of the right light guide plate and the left light guide plate 28. However, this configuration can be optionally changed. For example, virtual images may be formed in regions occupying the entire surfaces (or most) of the right light guide plate 26 and the left light guide plate 28. In this case, an image may be reduced by operation for changing a display position of the image. The optical elements according to the invention are not limited to the right light guide plate 26 and the left light guide plate 28 including the half mirrors 261 and 281. Any form can be adopted as long as optical components (e.g., a diffraction grating, a prism, and holography) that make image light incident on the eyes of the user are used.
The invention is not limited to the embodiments, the examples, and the modifications explained above and can be realized in various configurations without departing from the spirit of the invention. For example, the technical features in the embodiments, the examples, and the modifications corresponding to the technical features in the aspects described in the summary can be substituted or combined as appropriate in order to solve a part or all of the problems explained above or achieve a part or all of the effects explained above. Unless the technical features are explained in this specification as essential features, the technical features can be deleted as appropriate.
The entire disclosure of Japanese Patent Application Nos. 2016-087032, filed Apr. 25, 2016 and 2016-241270, filed Dec. 13, 2016 are expressly incorporated by reference herein.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2016-087032 | Apr 2016 | JP | national |
| 2016-241270 | Dec 2016 | JP | national |
