Patent Application
20200097119
Advancements in computing-technologies have fueled tremendous development efforts to deploy immersive environments in various computing-applications, such as simulation, gaming, and entertainment applications. For instance, various online gaming platforms have integrated virtual-reality (VR) environments, augmented-reality (AR) environments, and/or mixed-reality (MR) environments into gameplay. Deploying such immersive environments enables users to view and perceive computer-generated three-dimensional (3D) objects, as if the objects were actually present within the users' perceived environments. Various head-mounted display (HMD) devices, such as VR and/or AR headsets, have been developed to deploy immersive environments. Such HMD devices generally provide users a display of a field-of-view (FOV) that includes computer-generated 3D objects. That is to say, HMD devices provide a display of an immersive environment.
However, to enhance the user's experience within the immersive environment, it is advantageous to enable realistic user-interactions with computer-generated 3D objects displayed via a HMD device. For instance, the immersive experience may be enhanced by enabling a user to select, control, edit, rotate, translate, or otherwise manipulate various computer-generated 3D objects within the FOV provided by a HMD device. Furthermore, the user may desire to alter various aspects or characteristics of the provided FOV. Thus, for a truly immersive experience, the ability for the user to provide input to interact with computer-generated 3D objects is of tantamount importance. It is for these concerns and other concerns that the following disclosure is provided.
Embodiments described herein provide methods and systems for providing realistic and intuitive user-interactions with virtual objects (VOs) included in immersive environments (IEs). More particularly, the various embodiments are directed towards commutatively coupling, via a communication session, a head-mounted display (HMD) device with an interaction-sensing (IS) device. The HMD device provides an IE to a wearer by displaying a field-of-view (FOV) that includes one or more VOs.
The user executes one or more gestures (i.e. user-interactions), such as but not limited to 2D and/or 3D hand gestures, fingertip gestures, multi-fingertip gestures, stylus gestures, hover gestures, and the like. The IS device detects, senses, and/or tracks the user-interactions. In response to such detections of user-interactions, the IS device generates interaction data, and provides at least a portion of the interaction data to the HMD device via the communication session. The FOV and/or one or more VOs included in the FOV are updated and/or modified in response to the interaction data received at the HMD device. Updating and/or modifying the FOV and/or the VO may indicate the user's intended user-interaction with the VO.
In some of the various embodiments, the IS device is separate from the HMD device, i.e. the IS device is not embedded within and/or not integrated with the HMD device. The IS device may include at least one of a hover-sensing (HS) device, a touch-and-hover (TAH) device, or a combination thereof. In other various embodiments, the IS device includes at least a 2D touch-sensitive device. In at least one of the various embodiments, the IS device includes multiple camera devices that detect and encode, via interaction data, the user-interactions.
In some embodiments, a physical overlay may be coupled with the IS device. For instance, when coupled to a HS device and/or a TAH device, one or more surfaces of the overlay (e.g. a protrusion) may be displaced from the active surface of the HS device and/or the TAH device. The overlay may be constructed from a material that is at least partially transparent to the hover-sensing capabilities of the HS and/or the TAH device. Thus, the user may touch the displaced surfaces of the overlay, and the hover-sensing capabilities of the HS device and/or the TAH device may detect the user's fingertips on the displaced surfaces. The displaced surfaces of overlay may provide one or more 2D and/or 3D shapes and/or protrusions. Such shapes or protrusions may include, but are not limited to curved bosses, parallelepipeds, cylinders, pyramids, and the like. Thus, the shapes and/or protrusions of displaced surfaces of the overlay provide tactile feedback for the user, when interacting with VOs.
The present invention is described in detail below with reference to the attached drawing figures, wherein:
As used herein, the term “virtualized object” (VO) may refer to any computer-generated object or any computer-generated visualization of a (physical or non-physical) object. Such computer-generated objects (or computer-generated visualizations thereof) may be one-dimensional (1D) objects, two-dimensional (2D), or three-dimensional (3D) objects (or visualizations thereof). As used herein, the term “immersive environment” (IE) may refer to any physical (i.e. “real”) environment, any non-physical (i.e. “computer-generated” and/or “virtual”) environment, or any combination thereof that includes a display of one or more VOs. Such IEs include, but are not otherwise limited to virtual-reality (VR) environments, augmented-reality (AR) environments, and/or mixed-reality (MR) environments.
As used herein, the term “interaction data” may refer to any data or information that indicates or encodes user-interactions (e.g. hand, fingertip, and multi-fingertip gestures). As used herein, an “interaction-sensing device,” or simply a “IS device” may be any device that detects, senses, and/or tracks user-interactions, such as but not limited to hand gestures, fingertip gestures, multi-fingertip gestures, stylus gestures, and the like. In response to such detections of user-interactions, an IS device may generate interaction data that encodes the user-interactions. The generated interaction data may encode 2D and/or 3D hand, fingertip, or multi-fingertip gestures executed by a user. In some embodiments, an IS device may encode gestures of a stylus or a stylus-tip executed by a user.
In various embodiments, a IS device includes one or more wired and/or wireless data transceivers to provide another device, via a communication session, at least a portion of the generated interaction data. Such wireless data transceivers include, but are not otherwise limited to one or more communication radios. In some embodiments, a IS device may include a haptic-feedback interface, similar to haptic-feedback interfaces frequently included in mobile devices such as tablets and smartphones.
Development efforts in computing technologies have recently enabled “hover”-sensing (HS) devices. Such HS devices generally detect “hover-gestures” of hands, fingers, and fingertips (i.e. extremities) of a user, as well as the gestures of a stylus held by a user. Typically, a HS device includes one or more (planar and/or non-planar) active surfaces and associated hover-gesture sensors. Hover-gesture sensors may include proximity sensors. The associated proximity sensors detect the presence (or non-presence) of user extremities (e.g. the user's hands, fingers, and fingertips) in proximity to the one or more active surfaces. For some HS devices, the proximity sensors can determine, sense, and/or detect at approximate least 3D positions or locations of the user extremities (relative to the associated active surface), while the extremities are hovering and/or in motion proximate the active surface.
More specifically, when the user extremities are proximate to the one or more surfaces, a HS device is enabled to generate interaction data that indicates or encodes the 3D location, position, and/or motion of user extremities relative to the one or more active surfaces. Although the detection and encoding of the location and/or motion of the user's extremities are discussed throughout, it should be understood that a HS device may be employed to detect and encode other user-interactions, such as those initiated via a stylus held be the user.
The proximity sensors may be implemented via capacitive-sensing technologies, similar to those employed in touch-sensitive display devices. Typically, a HS device can detect user extremities that are within a threshold distance (i.e. a proximate-distance threshold) from the active surface. Different HS devices are associated with various proximate-distance thresholds. For some HS devices, the proximate-distance threshold is between 2.0 cm and 20.0 cm. The proximate-distance threshold of some HS devices is less than 10.0 cm. For at least one HS device, the proximate-distance threshold is about 4.0 cm.
Thus, a HS device may detect (and encode via interaction data) hover-gestures, similar to fingertip and multi-fingertip gestures often associated with touch-sensitive display devices. However, in contrast to the 2D fingertip gestures associated with touch-sensitive display devices, such hover-gestures may be 3D fingertip and multi-fingertip gestures. For instance, for a planar active surface of a HS device, a Cartesian coordinate system may be employed. The x-y plane of the Cartesian coordinate system may be substantially co-planar with (and the z-axis of the Cartesian coordinate system may be substantially orthogonal to) the planar active surface of a HS device.
More particularly, capacitive proximity sensors embedded in and/or integrated with a planar active surface of a HS device detect the user's fingertips that are hovering and/or in motions proximate the active surface. In response to the detection, the HS device generates interaction data encoding the x-coordinate, y-coordinate, and z-coordinate of one or more of the user's fingertips (relative to the active surface), as long as the z-coordinate of fingertip is less than the proximate-distance threshold of the proximity sensors associated with the active surface. The resolution of the proximity sensors in each dimension may be similar or may be different, depending on the specific proximity sensors embedded in the HS device. For HS devices with non-planar active surfaces, other orthonormal 3D coordinate systems may be employed, such as but not limited to spherical coordinates or cylindrical coordinates, depending on the shape of the non-planar active surfaces.
As such, a HS device may be employed to detect and encode various user-interactions, such as but not limited to 3D-versions of any 2D gesture associated with touch-sensitive displace devices, such as but not limited to 3D versions of “pinch,” “pull,” “press and hold,” “single-finger tap,” “two-finger tap,” “single tap,” “double tap,” “swipe left,” “swipe right”, pressure-variance (e.g. 3D-touch) gestures, and the like. Furthermore, a HS device may be employed to detect and encode, hand-writing, notes, sketches, drawings, and other user-interactions initiated via the user's fingertips and/or a stylus held and guided by the user's hands.
The hover-sensing capabilities of a HS device may be integrated with a touch-sensitive device, such as but not limited to a touch-sensitive display device, to generate a touch-and-hover (TAH) device. The touch-sensitive device detects and encodes the 2D touch of the user's extremities, as well as the 3D hover-gestures of the user's extremities. That is, the active surface functions similar to the touch-sensitive surface of a touch-sensitive display device. As such, a TAH device may detect and encode all of the 2D fingertip, multi-fingertip, and stylus-tip gestures (on the active surface) associated with a touch-sensitive display device, as well as the 3D fingertip, multi-fingertip, and stylus-tip gestures in proximity to the active surface. Thus, a TAH device is a HS device. The touch-sensitive surface of a TAH device may also be a display device, but need not be.
Thus, in some embodiments, a IS device includes at least one of a HS device, a TAH device, or a combination thereof. In other various embodiments, a IS device includes at least a 2D touch-sensitive device. In some embodiments, the IS device may include a 2D touch-sensitive display device. In other embodiments, the IS device does not include a display device. For instance, an IS device may include a 2D touch-sensitive tracking pad or a TAH device that does not include a display device (i.e. the touch-sensitive device is not a display device). In at least one of the various embodiments, an IS device includes a gaming and/or an entertainment system that includes various combinations of proximity sensors, motion sensors, cameras, and the like. Such combinations of cameras and/or motion sensors may be enabled to detect and encode the user's movements through gameplay, via generated interaction data.
In the various embodiments, IS devices may include touch-sensitive device, such as a touch-sensitive display device, where the touch-sensitive device is a multi-touch display device. That is, the touch-sensitive device that detects and/or senses two or more points of contact. In addition to being touch-sensitive, an IS device may include a pressure-sensitive device, such as a pressure-sensitive display device. Such IS device may detect 3D gestures via multi-pressure touch or fingertip gestures. That is a user may provide 3D gestures by the pressure-sensitive device via pressure-variance gestures, e.g., 3D-touch. For instance, the 3D information provided via a HS may be similarly provided via pressure-variance gestures and a pressure-sensitive device.
In some embodiments, a IS device may include any combination of a touch-sensitive, pressure-sensitive device, hover-sensitive device. That is, a touch-sensitive device included in a IS device may also be pressure-sensitive and/or hover-sensitive. Thus, a IS device may detect 3D via touch, pressure, and/or hovering-like gestures. In other embodiments herein, 3D gestures may be detected via one or more devices, such as IS devices by employing one or more photon-detectors (e.g., a camera) to track the motion of fingertips hovering over a 2D touch-sensitive surface. For instance, a matrix of photon-detectors may be employed to detect gestures my detecting photons reflected from the users fingertips. Similarly, an IS device may track the user's gestures via acoustic and/or ultra-sound imaging. In some embodiments, an IS device may include magnetic sensors and the users wears magnetic rings. The IS device may detect the 3D gestures via the magnetic sensors and the magnetic rings. In other embodiments, user gestures may be detected the sensing of interaction with mechanical and/or active devices or parts, such as but not limited to switches, dials, levers, buttons, joysticks, trackballs, clickable devices, and the like. An IS device may include data gloves and other sensing devices, such as exoskeletons. Such IS devices may be enabled to detect gestures over a mobile device, such as a tablet. Any of these IS devices may also provide haptic feedback in the various embodiments.
In addition to HS devices and TAH devices, development efforts in computing technologies have also resulted in the development of various head-mounted display (HMD) devices. Such HMD devices generally enable the realistic perception of computer-generated two-dimensional (2D) and/or three-dimensional (3D) objects. More particularly, an HMD device provides, to a user wearing the HMD device, a visualization (or display) of a field-of-view (FOV) that includes one or more virtualized objects (VOs). As such, an HMD device provides a display or visualization of VOs within an immersive environment (IE).
A user may wish to interact with the VOs within an IE. For example, a user may wish to select, control, edit, rotate, translate, or otherwise manipulate a VO. Similarly, a user may wish to interact with various discrete features, structures, components, modules, controls, or other parts of a VO. Some HMD devices enable limited user-interactions with VOs. For instance, some HMD devices include embedded motion sensors (e.g. cameras) that detect broad user hand-gestures through free space. Upon detection of such a free-space gesture, the HMD device may modify a VO.
However, the spatial-resolution of such embedded motion sensors (and thus the user-interactions) may be limited. For instance, unless the user positions their hands in close proximity to HMD device, the angular resolution of optical systems associated with the embedded motion sensors may limit the ability to resolve relative small spatial-differences in the position of the user's hands. Such limited resolution may limit the user's ability to select, edit, or otherwise manipulate closely-spaced VOs or closely-spaced features of a VO. Additionally, such HMD devices may require a direct optical line-of-sight between the user's hands and the embedded motion sensors. For example, if an optically opaque surface (e.g. a desktop) is between the user's hands and the HMD device, or if the user positions their hands in proximity of their waist, the detection functionalities of embedded motion sensors may be limited.
In addition to spatial-resolution considerations, hand gestures through free space lack tactile and haptic feedback associated with other physical objects. Considering an example where a user desires to select and rotate a virtualized object (e.g. the user wants to select and rotate a virtual globe within an IE), a user employing free-space hand gestures does not experience the feedback of an associated physical object interacting with their hand or fingertips. Additionally, in some immersive applications, there may be limits to the allowed user-interactions. For example, within an immersive gaming-environment, there may be virtualized walls or other VOs that the user's hands (or a selected VO) may collide with. Free-space hand-gesture initiated user-interactions lack tactile and haptic feedback of such collision events. Additionally, such unrestrained free-space gestures may not provide intuitive user-interactions for the manipulations of VOs that are proxies to physical objects.
Furthermore, at least due to the significant adoption of touch-sensitive display devices, users have become accustomed to interacting with applications running on computing devices, via intuitive fingertip gestures. For instance, since the integration of touch-sensitive display devices within mobile devices (e.g. smartphones, tablets, smartwatches, and the like), users have become familiar with user-interactions within mobile applications via intuitive fingertip gestures. Such fingertip gestures may include multi-touch (or multi-fingertip) gestures, such as but not otherwise limited to “pinch,” “pull,” “press and hold,” “single-finger tap,” “two-finger tap,” “single tap,” “double tap,” “swipe left,” “swipe right”, pressure-variance (e.g. 3D-touch) gestures, and the like. Multi-touch gestures may even enable simultaneous interactions with multiple components of a computing application. For example, a user may simultaneously select multiple icons on a smartphone via multi fingertip touches to separate potions of the touch-sensitive display device.
Such heavily-adopted intuitive user-interactions may be difficult to implement via hand gestures though free space that are detected via motion sensors embedded in a HMD device. For instance, it may be difficult for resolution-limited HMD-device motion sensors to resolve the spatial resolution of multiple closely-spaced fingertips. That is to say, the spatial resolution of such a touch-sensitive device may be greater than that of motion sensors. Another issue with convention motion sensors is occlusions. For instance, some of the user's fingers may occlude other fingers of the user. Also, the detection of fingertip gestures that are based on physical contact with a touch-sensitive display device is a challenge. For example, hand gestures through free space may not realistically emulate the user-experience associated with “tap,” and “press and hold” fingertip gestures that are popular with touch-sensitive display devices. That is to say, free-space hand gestures lack the tactile feedback of tapping and swiping on a touch-sensitive display device.
Other limitations of free space user-interactions include that the user must suspend their hands in free space without support. The user's muscle may begin to tire after long usage. Additionally, the length of the user's arm is limited. In some situations, the arm of the user may be used as a pointer, at the expense of the resolution or accuracy of the gesture detecting. Such limitations of free space user-interactions, associated with motion sensors embedded in a HMD device, may decrease the “immersive experience” for the user interacting with VOs within an IE.
The various embodiments herein are directed to systems and methods that enable realistic and intuitive user-interactions with VOs included in an IE. As discussed throughout, the various embodiments herein address each of the above noted limitations of previously available methods and systems for interacting with VOs. More particularly, the various embodiments are directed towards commutatively coupling, via a communication session, a head-mounted display (HMD) device with an interaction-sensing (IS) device. The user executes one or more gestures, such as but not limited to 2D and/or 3D hand gestures, fingertip gestures, multi-fingertip gestures, stylus gestures, and the like. By executing such gestures, the user intends to interact with one or more of the VOs included in the IE (i.e. the user intends to execute user-interactions with a VO).
The IS device detects, senses, and/or tracks the user-interactions. In response to such detections of user-interactions, the IS device generates interaction data, and provides at least a portion of the interaction data to the HMD device via the communication session. The FOV and/or one or more VOs within the IE are updated and/or modified in response to the interaction data received at the HMD device. Updating and/or modifying the FOV and/or the VO may indicate the user's intended user-interaction with the VO.
In some of the various embodiments, the IS device is separate from the HMD device, i.e. the IS device is not embedded within and/or not integrated with the HMD device. The IS device may include at least one of a HS device, a TAH device, or a combination thereof. In other various embodiments, the IS device includes at least a 2D touch-sensitive device. In at least one of the various embodiments, the IS device includes multiple camera devices that detect and encode, via interaction data, the user-interactions.
In some embodiments, a physical overlay may be coupled with the IS device. For instance, when coupled to a HS device and/or a TAH device, one or more surfaces of the overlay may be displaced from the active surface of the HS device and/or the TAH device. The overlay may be constructed from a material that is at least partially transparent to the hover-sensing capabilities of the HS and/or the TAH device. Thus, the user may touch the displaced surfaces of the overlay, and the hover-sensing capabilities of the HS device and/or the TAH device may detect the user's fingertips on the displaced surfaces. The displaced surfaces of overlay may provide one or more 2D and/or 3D shapes and/or protrusions. Such shapes or protrusions may include, but are not limited to curved bosses, parallelepipeds, cylinders, pyramids, and the like. Thus, the shapes and/or protrusions of displaced surfaces of the overlay provide tactile feedback for the user, when interacting with VOs.
In various embodiments, the IS device may automatically determine an identifier associated with the overlay. A mode of the MHD device may be updated based on the identifier of the overlay. In some embodiments, the motion of the user's fingertips along the displaced surfaces of the overlay may be detected and encoded via the interaction data. The interaction data is employed to determine the shape and/or identify the protrusions provided by the displaced surfaces. The FOV and/or a VO may be updated and/or modified based on the determined shape.
In various embodiment, the 2D surface of the IS may be mapped to the 2D surface of a 3D virtual or real object in the environment. Moving a finger over the touch surface, will move a point on the surface of the object, where the mapping between the surface area and the IS surface is define the same way, as definition of texture mapping. Furthermore, hovering or elevating a finger above the IS surface, will move the corresponding 3D point, in the direction of the 3D surface normal, away from the object.
A haptic-feedback interface included in the IS device may be employed to provide haptic feedback to the user in accordance with events occurring within the FOV. Thus, the shapes and/or protrusions of an overlay, as well as the haptic-feedback interface of the IS device provide the user tactile and/or haptic feedback when interaction with VOs. When an overlay is not coupled with the IS device, the user may additionally interact with a VO, via 2D fingertip and multi-fingertip gestures on the touch-sensitive device of the IS device. Thus, the user may be provided with tactile feedback via the touch-sensitive surfaces of a IS device. Furthermore, as discussed throughout, hover gestures detected via the various IS devices may enable more precise and accurate user-interactions with VOs, given the increased spatial resolutions of hover-sensing and touch-sensing active surfaces of IS devices. Additionally, surfaces of an IS, such as but not limited to touch-sensitive surfaces and surfaces of an IS overlay provide support and friction for the user's fingertips. Such friction may enable precise movement of the user's fingers.
With reference now to
A communication network 160 communicatively couples IS device 110 and HMD device 140, via a communication session. In some embodiments, communication network 160 is peer-to-peer (P2P) network that enables the communication session between IS device 110 and HMD device 140. For instance, IS device 110 and HMD device 140 may be paired via a P2P communication session. In other embodiments, communication network 160 may include, without limitation, one or more local area networks (LANs) and/or wide area networks (WANs). In exemplary implementations, network 160 comprises the Internet and/or a cellular network, amongst any of a variety of possible public and/or private networks.
Although
IS device 110 may include a 2D touch-sensitive surface 112. Touch-sensitive surface 112 may include a touch-sensitive device, such as but not limited to a touch-sensitive display device similar to those frequently used in computing devices, such as smartphones and tablets. However, touch-sensitive surface 112 need not be a display device. As shown in
IS device 110 includes touch-gesture sensors 120 that sense or detect touch-gestures on touch-sensitive surface 112. More particularly, touch-gesture sensors 120 may sense and/or detect 2D touch-gestures associated with touch-sensitive devices. That us, touch-gesture sensors 120 may sense and/or detect hand, fingertip, multi-fingertip, and stylus gestures such as but not limited to “pinch,” “pull,” “press and hold,” “single-finger tap,” “two-finger tap,” “single tap,” “double tap,” “swipe left,” “swipe right”, pressure-variance (e.g. 3D-touch) gestures, and the like. In various embodiments, touch-gesture sensors 120 may be captivate-sensing sensors. Thus, touch-sensitive surface 112 may be a 2D capacitive-sensing surface.
In some embodiments, IS device 110 may be a hover-sensing (HS) device. In at least one embodiment, IS device 110 may be a touch-and-hover (TAH) device. In such embodiments, touch-sensitive surface 112 may additionally be an active surface of the HS device, i.e. IS device 110 may sense hover gestures in proximity to touch-sensitive surface 112. Accordingly, surface 112 may be both touch-sensitive and hover-sensitive. More particularly, IS device may include hover-gesture sensors 122, such as but not limited to proximity and/or motion sensors, that sense and/or detect hover-gestures in proximity to touch-sensitive surface 112. Such hover gestures include but are not limited to 3D hand, fingertip, multi-fingertip, and stylus gestures. For instance, hover-gesture sensors 122 may sense and/or detect 3D versions of fingertip gestures associated with touch-sensitive devices, such as but not limited to 3D versions of “pinch,” “pull,” “press and hold,” “single-finger tap,” “two-finger tap,” “single tap,” “double tap,” “swipe left,” “swipe right”, pressure-variance (e.g. 3D-touch) gestures, and the like.
In various embodiments, hover-gesture sensors 122 may be captivate-sensing sensors. Thus, touch-sensitive surface 112 may be a 3D capacitive-sensing surface. In at least one embodiment, IS device 110 does not include a touch-sensitive surface and/or touch-sensitive sensors. That is, surface 112 may be only a hover-sensing surface (i.e. surface 112 is not a touch-sensitive surface).
IS device 110 includes an interaction-data generator 124 that generates interaction data that encodes the touch and hover gestures sensed and/or detected via touch-gesture sensors 120 and hover-gesture sensors 122 respectively. Interaction-data generator 124 may process, package, encrypt, or other otherwise prepare the interaction data for transmission to HMD device 140. IS device 110 includes IS data-transceiver 128, which may be a wired or a wireless data transceiver. Such wireless data transceivers include, but are not otherwise limited to one or more communication radios. IS data-transceiver 128 is enabled to provide HMD device 140 the interaction data, via the communication session enabled by communication network 160.
In some embodiments, IS device 110 includes a haptic-feedback interface, such as those commonly integrated in smartphones, tablets, video-game controller devices, and the like. IS device 110 may include a plurality of camera devices that sense and/or detect free-space gestures of a user. Interaction-data generator 124 may generate interaction data encoding such free-space gestures. In various embodiments, IS device 110 includes one or more additional components included in a computing device. In at least one embodiment, IS device 110 device includes a IS computing device 130. Various embodiments of computing devices are discussed in conjunction with at least
HMD device 140 includes an immersive-environment (IE) display device 142 that is enabled to provide a display and/or a visualization of one or more virtual objects (VOs) to a user wearing HMD device 140. HMD device 140 also includes a wired and/or wireless data transceiver, such as HMD data-transceiver 114. HMD data-transceiver is enabled to receive the interaction data, provided via IS data-transceiver 128 of IS device 110. MHD device 140 may additionally include a computing device, such as HMD computing device 146. Other embodiments of a HMD device are discussed in conjunction with at least
IS device 110 may be paired with a HMD device, such as but not limited to HMD device 140 of
IS device 210 may generate interaction data that encodes the touch and hover gestures of a user. The interaction data may be provided to the HMD device, via the communication session. The HMD device may update and/or modify the FOV 250 and/or any VOs included in FOV 250 based on the interaction data. Thus, the user can interact with the FOV 250 and/or any VOs included in the FOV 250. For instance, the user may select, control, edit, rotate, translate, or otherwise manipulate a VO (or features of a VO). Similarly, a user may alter various aspects, characteristics, or properties the provided FOV 250.
More particularly,
Thus, virtually any user-interaction with a VO within an IE may be enabled via the communicatively coupling of IS device 210 with a HMD device. The spatial-resolution of the touch and hover gestures encoded in the interaction data may be greater than the spatial-resolution associated with detecting free-space gestures via motion sensors embedded within the HMD device. Accordingly, the spatial-resolution associated with user-interactions enabled via IS device 210 may be greater than the spatial-resolution associated with previously available systems and methods for enabling user-interactions with VOs. Additionally, IS device 210 enables interacting with VOs within an IE via 2D and 3D versions of pinch,” “pull,” “press and hold,” “single-finger tap,” “two-finger tap,” “single tap,” “double tap,” “swipe left,” “swipe right”, pressure-variance (e.g. 3D-touch) gestures, and other fingertip and multi-fingertip gestures commonly employed by users of 2D touch-sensitive display devices. The emulation of such 3D user-interactions may not be as readily accomplished via previously available methods of detecting free-space gestures.
In the various embodiments, touch and hover gestures may be combined to generate additional gestures. For instance, one finger may touch the display for support, while another finger is hovering to define position in space. In such embodiments, the hand is supported, as well as the user muscles are used to sense the distance from the IS in a very accurate and non-visual way. In at least one embodiment, the distance between the finger on the IS device and the hovering finger, can define a vertical scale of an object. The position of a second finger (such as the thumb) can change the meaning of the touch gesture (for example from dragging to selection).
Although
In the non-limiting embodiment shown in
Note, that the presentation of the layers of a multilayered object within a 3D FOV need not be “flat” as depicted in
The selection of the layer or planar slice of the VO, via a hover gesture detected by the IS device, may result in a “pinning” (or correlation) of the layer with a touch-sensitive surface of the IS device. That is to say, the plane of the selected layer is pinned or correlated with the plane of a touch-sensitive surface of the IS device. More specifically, once a layer or planar slice of a VO (e.g. Layer 3 of the presentation slide of
A hovering finger may select a plane above the IS device in a continues motion upward-downward. The selection of such a plane, may bring a cut-out of a 3D object presented above the IS device, to be displayed on the screen of the IS. In a similar manner, a hovering finger may move a 3D virtual object into and out of the FOV of the hover device of the IS. As the object is pushed into FOV of the IS, the IS may display different layers/cutouts of the object. In another embodiment the IS device can be hold in 3D space inside a 3D object, and by touching the IS, it may copy a slice of the 3D object onto the IS display. The user may hold, examine, annotate, or manipulated the object or slice of the object.
Fingers 204 are in physical contact with touch-sensitive surface 212. More specifically, via common 2D fingertip and multi-fingertip gestures (e.g. “pinch,” “pull,” “press and hold,” “single-finger tap,” “two-finger tap,” “single tap,” “double tap,” “swipe left,” “swipe right”, pressure-variance (e.g. 3D-touch) gestures, and the like), the user may re-size, rotate, translate, re-position, edit text, or otherwise interact with and manipulate any of VOs 272, 272, and 276.
The touch of the touch-sensitive surface on the user's fingertips may provide tactile feedback for the user, enabling more realistic, controlled, and precise user-interactions with various VOs included in the selected and pinned (or correlated) layer. Furthermore, the spatial resolution of the 2D touch-sensitive surface (i.e. the spatial resolution in the x-y plane) may be greater than the spatial resolution associated with 3D hover gestures. That is, the interaction data may more accurately encode the (x-axis and y-axis) positions and motions of the user's fingertips, when the fingertips are in contact with the touch-sensitive surface, as compared to when the fingertips are hovering above the touch-sensitive surface. Thus, by correlating a layer or planar-slice of a VO object to a touch-sensitive surface of the IS device, a user may more precisely and accurately interact with various VOs within an IE.
Overlay 320 is configured and arranged for mechanically coupling or otherwise interfacing with IS device 310. For instance, overlay 320 may be configured and arranged to “snap” onto or otherwise become fixably attached to IS device 310. Overlay 320 can include a 3D protrusion 322. In the non-limiting embodiment shown in
Overlay 320 may be constructed from plastic, a glass, or another material that is at least partially transparent to the hover-sensing abilities of IS device 310. For instance, overlay 320 may be constructed via molded plastic or a tempered glass. Plastics may include, but are not limited to a polyethylene (PET) plastic or a thermoplastic polyurethane (TPU) plastic. In at least one embodiments, overlay 320 may include capacitive sensors to detect and/or sense the user's touch of surfaces of overlay 320. In such embodiments, overlay may generate and provide interaction to IS device 320 via a wired and/or wireless data transceiver interface. Thus, overlay 320 may not be required to be transparent to the hover-sensing abilities of IS device 310.
As such, IS device 310 is enabled to detect (and generate interaction data in response to the detection of) when the user touches or positions one or more fingertips on the surfaces of the overlay 320, such as but not limited to the displaced surfaces (relative to active surface 312) of protrusion 322. As such, IS device 310 may sense the hover gestures of a user, while the surfaces of overlay 320, such as but not limited to the surfaces of protrusion 322, provide the user tactile feedback for user-interactions with a VO. That is, the user may experience tactile feedback while interacting with a VO via 3D hover gestures detected with IS device 310.
FOV 350 includes a VO 360, which is a hologram of Earth. The user may interact with VO 360 (or any other VO) via 3D hover gestures with fingertips 304 in contract with surfaces of protrusion 322. For instance, the user may manipulate, modify, or update a rotational orientation of VO 360 by gliding fingertips 304 or the palm of hand 302 over the hemispherical surfaces of protrusion 322. In another embodiment, the user may rotate a viewpoint of FOV 350 (i.e. rotate a position of viewer within the IE) based on hover gestures with their fingers in contact with the protrusion 322. In still other embodiments, the user may employ the hemispherical surfaces of protrusion 322 as a “trackball”-like control element for the interactions with VO 360. Thus, when interacting with VO 360, the user experiences tactile feedback via the physical sensation of fingertips 304 in physical contract with surfaces of protrusion 322.
In various embodiments, at least a portion of the surfaces of the overlay may be constructed from a softer material, such as a pliable or malleable material that the user may at least slightly deform via pressure from there touch. For instance, the upper portion of protrusion 322 may be constructed from such a softer material, while the lower portion is constructed from a harder or less pliable material, e.g., the upper portion may be overlaid with a putty or soft rubber-type material. The contrast in the hardness of the materials may provide additional tactile feedback, variances in friction, or otherwise constrain the movement of the fingers.
The constraints may be defined via furrows in the hard and/or soft materials. In at least one embodiment, at least a portion of the protrusion may be deformable, e.g., the protrusion may include an inflatable portion or otherwise be deformable via a mechanical device. As discussed herein, haptic feedback may be selectively provided via the harder and softer surfaces. For instance, an overlay may include one or more haptic feedback interface. The haptic feedback interface associated with the softer surfaces may be activated and/or operated separately from the haptic feedback interfaces of the harder surfaces.
Although not shown in
Although
The HMD device may update FOV 480 based on the determined shape of protrusion 432 (or another discrete physical object) and/or the interaction data that is used to encode the hover gestures of the user. In one non-limiting embodiment, FOV 480 may be updated to include a hologram or other visualization of a VO of the determined shape. For example, FOV 480 has been updated and/or modified to include VO 470, based on the determined pyramid shape of protrusion 432.
Furthermore, a unique identifier associated with overlay 430 may automatically be determined based on the determined shape of protrusion 432. In other embodiments, IS device 410 may be enabled to automatically determine and/or detect the unique identifier associated with overlay 430. For instance, IS device may include various sensors, such as but not limited to optical scanners, cameras, touch-sensitive sensors, radios, and the like that may automatically detect the mechanically coupling of overlay 430 with IS device 410. Such sensors may detect and optically scannable code, a radio-frequency identification (RFID) tag, or the like that is uniquely associated with the unique identifier associated with overlay 430.
In various embodiments, the unique identifier may be encoded in the interaction data provided to the HMD device. An operating mode of the HMD device may be updated based on the identifier. For instance, the various user-interactions provided to a user may be updated based on the identifier associated with the overlay. In other embodiments, the operating mode may be updated based on the automatically determined shape of the protrusion or the discrete physical object.
When overlay 520 is mechanically coupled to IS device 510, the plurality of capacitive couplers 530 can capacitively couple at least a portion of the points of the 3D surface of protrusion 530 with points on the 2D touch-sensitive surface 512. Thus, based on the 3D shape of protrusion 522 and the locations (on each of the protrusion 522 and touch-sensitive surface 512) of the capacitive couplers 530 and the locations of the corresponding points on the 2D touch-sensitive surface 512, a one-to-one correspondence map may be generated. That is, a one-to-one map between the points on the 3D surface of protrusion 522 and the corresponding points on the 2D touch-sensitive surface 512 may be generated. In some embodiments, the IS device is enabled to generate the one-to-one map. In at least one embodiments, the HMD device generates the one-to-one map.
Capacitive couplers 530 and the one-to-one correspondence map can enable the user performing 3D hover gestures along the surfaces of overlay 510 to 3D interact with a VO, employing an IS device that does not include hover-sensing capabilities. For example, a spherical overlay, may be used to rotate a 3D object in VR/AR while looking at it, and feeling the finger movement on the plastic protrusion.
Camera system 610 may be communicatively coupled, via a communication session, to HMD device 690. HMD device 690 may be an AR and/or a MR enabled HMD device, such as those discussed in conjunction with HMD device 140 of
Referring now to
Initially, at block 702, a communication session is established between an IS device and a HMD device. In some embodiments, the HMD device is enabled to initiate and establish the communication session. In other embodiments, the IS device initiates and establishes the communication session. In some non-limiting embodiments, the communication session is a peer-to-peer (P2P) communication session, but need not be. At any rate, via block 702, the IS device and the HMD device are communicatively coupled and/or are in communication.
The IS device may be a hover-sensing (HS) device, a touch and hover device, a touch sensing device, a camera system, or any of the other various embodiments of IS devices discussed herein. The HMD device may be an AR HMD device, a VR HMD device, a MR HMD device, or the like.
At block 704, the IS device detects one or more user gestures. For instance, the IS device may detect 2D and/or 3D gestures such as hover gestures, free-space hand gestures, fingertip gestures, multi-fingertip gestures, touch gestures, and the like. At block 706, and in response to the detected user gestures, the IS device generates interaction data encoding the user gestures.
At block 708, at least a portion of the interaction data is received at or by the HMD device. In some embodiments, the interaction data is received directly from the other IS device, via the communication session. That is, the IS device provides and/or communicates the interaction data to the HMD device via the communication session. In other embodiments, the interaction data is received via another computing device that is communicatively intermediate the IS device and the HMD device.
At block 710, a VO included in a FOV provided by the HMD device is modified and/or updated based on the generated interaction data. The modification to the VO may include virtually any modification to provide a visualization and/or other indication of the user interacting with the VO. For instance, the user may select, control, edit, rotate, translate, move, reposition or otherwise manipulate the VO. For instance, a rotational orientation or a position of the VO may be updated based on the interaction data. In some embodiments, the FOV may be updated and/or modified to change the color, contrast, or brightness of a background or one or more VOs included in the FOV. In various embodiments, the viewpoint (e.g. a position of the viewer's perspective) of the FOV may be updated at block 710. In some embodiments, the FOV may be updated to include the generation of new VOs and/or the multiplication and/or reproduction of VOs already included in the FOV.
In at least one embodiment, the interaction data encodes a selection of a planar slice or a layer of a 3D VO. At block 708 or block 710, the planar slice of the VO may be correlated with a touch-sensitive surface of the IS device. For instance, the selected planar slice may be “pinned” to the touch-sensitive surface. The IS device may generate additional interaction data, in response to the user executing touch gestures on the touch-sensitive device. Upon receiving the additional interaction data, the HMD device may modify and/or update the VO based on the additional interaction data.
At block 804, an identifier associated with the overlay may be automatically determined by at least one of the IS device or the HMD device. For instance, the identifier may be automatically determined via an RFID tag, an optically scannable code, object recognition features, or the like. In at least one embodiment, the identifier is automatically determined based on interaction data generated from the user gliding (or touching) their fingertips over one or more protrusions of the overlay.
At block 806, interaction data may be communicated to the HMD device. In addition to user gestures, the interaction data may encode the identifier of the overlay. An operating mode of the HMD device may be updated and/or modified based on the identifier. At block 808, a shape of a physical object is determined based on interaction data. For instance, the physical object may be a protrusion of the overlay and/or a discrete physical object that is separate from the overlay. The interaction data may be generated based on hover gestures of the user manipulating the physical object. The determination of the shape may be performed at the IS device or the HMD device.
At block 810, the FOV provided by the HMD device is updated and/or modified based on the determined shape. For instance, a VO may be generated within the FOV, wherein the shape, color, position, rotational orientation, or the like of the VO is based on the determined shape and/or generated interaction data. At block 812, an event is detected in the FOV. For instance, the event may include an event within a gaming or simulation application (e.g., a collision of a user controlled VO with another VO included in the IE). At block 814, and in response to the detected event, haptic feedback is provided at the IS device. For instance, a haptic-feedback interface of the IS device may be employed to provide haptic feedback to the user operating the IS device.
Turning to
Other arrangements and elements (e.g., machines, interfaces, functions, orders, and groupings of functions, etc.) can be used in addition to or instead of those shown, and some elements may be omitted altogether. Further, many of the elements described herein are functional entities that may be implemented as discrete or distributed components or in conjunction with other components, and in any suitable combination and location. Various functions described herein as being performed by one or more entities may be carried out by hardware, firmware, and/or software. For instance, various functions may be carried out by a processor executing instructions stored in memory.
Having described embodiments of the present invention, an exemplary operating environment in which embodiments of the present invention may be implemented is described below in order to provide a general context for various aspects of the present invention. Referring initially to
The invention may be described in the general context of computer code or machine-useable instructions, including computer-executable instructions such as program modules, being executed by a computer or other machine, such as a personal data assistant or another handheld device. Generally, program modules including routines, programs, objects, components, data structures, etc. refer to code that perform particular tasks or implement particular abstract data types. The invention may be practiced in a variety of system configurations, including hand-held devices, consumer electronics, general-purpose computers, more specialty computing devices, etc. The invention may also be practiced in distributed computing environments where tasks are performed by remote-processing devices that are linked through a communications network.
With reference to
Computing device 1000 typically includes a variety of computer-readable media. Computer-readable media can be any available media that can be accessed by computing device 1000 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer-readable media may comprise computer storage media and communication media.
Computer storage media include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computing device 1000. Computer storage media excludes signals per se.
Communication media typically embodies computer-readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media includes wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared and another wireless media. Combinations of any of the above should also be included within the scope of computer-readable media.
Memory 1012 includes computer storage media in the form of volatile and/or nonvolatile memory. The memory may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard drives, optical-disc drives, etc. Computing device 1000 includes one or more processors that read data from various entities such as memory 1012 or I/O components 1020. Presentation component(s) 1016 present data indications to a user or another device. Exemplary presentation components include a display device, speaker, printing component, vibrating component, etc.
I/O ports 1018 allow computing device 1000 to be logically coupled to other devices including I/O components 1020, some of which may be built in. Illustrative components include a microphone, joystick, game pad, satellite dish, scanner, printer, wireless device, etc.
Embodiments described in the paragraphs below may be combined with one or more of the specifically described alternatives. In particular, an embodiment that is claimed may contain a reference, in the alternative, to more than one other embodiment. The embodiment that is claimed may specify a further limitation of the subject matter claimed.
Accordingly, in one embodiment described herein, a method for enabling user-interactions with virtual objects (VOs). The method may include establishing, by a head-mounted display (HMD) device, a communication session with an interaction-sensing (IS) device. The HMD device displays, to a user, a field of view (FOV) that includes the virtualized object. The IS device is separate from the HMD device and is configured to detect user-interactions including a user extremity position relative to the IS device. The method may further include receiving from the IS device, via the established communication session, interaction data generated in response to a detected user extremity position relative to the IS device. Additionally, the method can include modifying the virtualized object included in the FOV based on the received interaction data.
The IS device may be a touch and hover (TAH) device. The TAH device is configured to detect a touch of the user extremity on a first surface of the TAH device. The TAH device is also configured to detect a distance between the first surface and the user extremity when the user extremity is displaced from the first surface. In other embodiments, the IS device is a hover-sensing (HS) device. In some embodiments, the IS device includes a touch-sensitive device, such as but not limited to a touch-sensitive display device.
The device may be further configured to mechanically couple with an overlay. When coupled, the overlay presents at least a portion of a second surface displaced from the first surface of the IS device. The IS device further generates the interaction data in response to a user extremity touch detected on the second surface. In at least one embodiment, the IS device and/or the HMD device is configured to automatically determine an identifier associated with the mechanically coupleable overlay for the IS device. The method may update an operating mode of the HMD device, based on the identifier. For instance, the identifier may be encoded in a portion of the interaction data provided to the HMD device.
The IS device may be configured to generate the interaction data in response to a motion of the user extremity detected along one or more surfaces of a physical object. For instance, the object may be a protrusion or shape included in the overlay. In other embodiments, the physical object may be another physical object that is not part of the over. A shape of the physical object, such as the protrusion or another discrete physical object, may be automatically determined (by either the IS device and/or the HMD device) based on interaction data generated in response to the motion of the user extremity detected along one or more surfaces of a physical object. The HMD device may update the FOV based on the determined shape. For instance, a hologram or other VO may be generated within the FOV that depicts a 2D or 3D visualization of the shape.
In some embodiments, the HMD device may update a rotational orientation of a VO included in the FOV based on interaction data that encodes detected motion of the user extremity along one or more surfaces of a physical object, such as but not limited to a protrusion of the overlay, or another separate physical object. The IS device may include multiple camera devices configured to detect at least a portion of the user-interaction.
The interaction data may encode a user-selection of a planar slice (or layer) of a VO. In such embodiments, the method further includes correlating, by the HMD device, the planar slice of the VO with a touch-sensitive surface of the IS device. The IS device is further configured to generate additional interaction data in response to a user extremity touch detected on the touch-sensitive surface. The HMD device receives, via the communication session, the generated additional interaction data communicated from the IS device. The HMD device modifies, the planar slice of the virtualized object included in the FOV based on the received additional interaction data.
In various embodiments, the IS device is configured to generate the interaction data in response to a motion of the user extremity detected along one or more surfaces of a protrusion included on an overlay that is mechanically coupled to the IS device. The HMD device is enabled to update a position of the virtualized object within in the FOV based on the received interaction data that encodes the motion of the user extremity.
In some embodiments, the IS device is configured to provide the user with haptic feedback in accordance with an event within the FOV. For instance, in gaming applications, the haptic-feedback interface included in the IS device may provide the user with haptic feedback.
In at least one embodiment, the IS device includes a two-dimensional (2D) capacitive-sensing surface. For instance, the IS device may include a 2D touch-sensitive surface such as but not limited to a touch-sensitive display device. Such IS devices are configured to interface with an overlay that includes a three-dimensional (3D) surface. For instance, the 3D surface may be a protrusion. The overlay may also include a plurality of capacitive couplers that capacitively couple portions of the 3D surface and portions of the 2D capacitive-sensing surface when the IS device interfaces with the overlay. A one-to-one map between the capacitively coupled portions of the 3D surface and the portions of the 2D capacitive-sensing surface may be generated. The IS device generates interaction data in response to a user extremity touch detected on the 3D surface. The HMD device updates a 3D aspect of the virtualized object based on the received interaction data and the generated one-to-one map.
The IS device or the HMD device may automatically determine a 3D contour of the protrusion or 3D surface of the overlay based on interaction data. The interaction data may be generated in response to the user touching or gliding their fingers along the 3D surfaces. The HMD device may generate another VO for display within the FOV. The shape of the VO is based on the determined 3D contour of the protrusion.
In another embodiment described herein, one or more computer storage media having computer-executable instructions embodied thereon that, when executed, by one or more processors, causing the one or more processors to execute operations for enabling user-interactions with virtual objects rendered in an immersive environment, is provided. For instance, the storage media may include non-transitory media. The operations include establishing, by a touch and hover (TAH) device, a communication session with a head-mounted display (HMD) device. The HMD device is configured to display, to a user, a field of view (FOV) that includes the virtualized object.
The operations may further include generating, by the TAH device, interaction data in response to three-dimensional (3D) motion of a user extremity detected by the TAH device. The generated interaction data may be communicated to the HMD device, via the established communication session. The HMD device is configured to modify the VO included in the FOV based on the communicated interaction data.
In some embodiments, the operations may further include receiving, via a mechanical coupling, an overlay for the TAH device. The overlay may include a protrusion. The TAH device is enabled to generate the interaction data in response to 3D motion of the user extremity detected along one or more surfaces of the overlay protrusion. The overlay protrusion includes at least one of a curved boss, a parallelepiped, a cylinder, or a pyramid.
In some embodiments, the HMD is configured to update a viewpoint of the FOV or a position of the VO based on a portion of the communicated interaction data that encodes the 3D motion of the user extremity detected along one or more surfaces of a protrusion coupled to the TAH device.
In various embodiments, the operations further include determining, by the TAH device, a shape of a protrusion coupled thereto. The determination of the shape is based on a portion of the generated interaction data that encodes at least a portion of the 3D motion of the user extremity detected along one or more surfaces of the protrusion. The HMD device may be configured to update the FOV based on the determined shape of the protrusion.
In yet another embodiment described herein, a system for enabling user-interactions with a VO in an immersive environment is provided. The system includes a physical overlay. The physical overlay presents a three-dimensional (3D) protrusion. The physical overlay is coupleable to an interaction-sensing (IS) device. The 3D protrusion is displaced from a two-dimensional (2D) interaction-sensing surface of the IS device. For instance, the 2D interaction-sensing surface may be a touch- or capacitive-sensing surface commonly employed in mobile devices such as but not limited to smartphones and tablets. The physical overlay is coupled to the IS device. The IS device is configured to generate interaction data in response to at least a user extremity touch detected on the 3D protrusion when coupled to the physical overlay. A head-mounted display (HMD) device is in communication with the IS device. The HMD device may be configured to update a field of view (FOV) displaying the virtualized object based on at least a portion of the generated interaction data received from the IS device.
In some embodiments, the IS device may be a touch and hover (TAH) device. The TAH device is configured to generate interaction data in further response to a user extremity hover-gesture detected over a portion of the 2D interaction-sensing surface. The physical overlay includes a plurality of capacitive couplers that are capacitively coupleable to portions of the 2D interaction-sensing surface when the physical overlay is coupled to the IS device. At least one of the IS device and/or the HMD device is configured to generate a one-to-one map between portions of the physical overlay and portions of the 2D interaction-sensing surface. The HMD device is further configured to update a 3D aspect of the virtualized object based on the received interaction data and the generated one-to-one map.
The subject matter of embodiments of the invention is described with specificity herein to meet statutory requirements. However, the description itself is not intended to limit the scope of this patent. Rather, the inventors have contemplated that the claimed subject matter might also be embodied in other ways, to include different steps or combinations of steps similar to the ones described in this document, in conjunction with other present or future technologies. Moreover, although the terms “step” and/or “block” may be used herein to connote different elements of methods employed, the terms should not be interpreted as implying any particular order among or between various steps herein disclosed unless and except when the order of individual steps is explicitly described.
For purposes of this disclosure, the word “including” has the same broad meaning as the word “comprising,” and the word “accessing” comprises “receiving,” “referencing,” or “retrieving.” In addition, words such as “a” and “an,” unless otherwise indicated to the contrary, include the plural as well as the singular. Thus, for example, the constraint of “a feature” is satisfied where one or more features are present. Also, the term “or” includes the conjunctive, the disjunctive, and both (a or b thus includes either a or b, as well as a and b).
For purposes of a detailed discussion above, embodiments of the present invention are described with reference to a head-mounted display unit; however, the head-mounted display unit depicted herein is merely exemplary. Components can be configured for performing novel aspects of embodiments, where configured for comprises programmed to perform particular tasks or implement particular abstract data types using code. Further, while embodiments of the present invention may generally refer to the head-mounted display unit and the schematics described herein, it is understood that the techniques described may be extended to other implementation contexts.
Embodiments of the present invention have been described in relation to particular embodiments which are intended in all respects to be illustrative rather than restrictive. Alternative embodiments will become apparent to those of ordinary skill in the art to which the present invention pertains without departing from its scope.
From the foregoing, it will be seen that this invention in one well adapted to attain all the ends and objects hereinabove set forth together with other advantages which are obvious and which are inherent to the structure.
It will be understood that certain features and sub-combinations are of utility and may be employed without reference to other features or sub-combinations. This is contemplated by and is within the scope of the claims.
The present application is a continuation of and claims priority from application Ser. No. 15/624,097, filed Jun. 15, 2017, which is assigned or under obligation of assignment to the same entity as this application, the entire contents of the application being herein incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15624097 | Jun 2017 | US |
| Child | 16695771 | US |
