Enhanced reality systems allow a user to become immersed in an enhanced reality environment wherein they can interact with the enhanced reality environment. For example, a head-mounted display, using stereoscopic display devices, allows a user to see, and become immersed in, any desired virtual scene. Such enhanced reality applications also provide visual stimuli, auditory stimuli, and can track user movement to create a rich immersive experience.
The accompanying drawings illustrate various examples of the principles described herein and are part of the specification. The illustrated examples are given merely for illustration, and do not limit the scope of the claims.
Throughout the drawings, identical reference numbers designate similar, but not necessarily identical, elements. The figures are not necessarily to scale, and the size of some parts may be exaggerated to more clearly illustrate the example shown. Moreover, the drawings provide examples and/or implementations consistent with the description; however, the description is not limited to the examples and/or implementations provided in the drawings.
Enhanced reality systems allow a user to become immersed in an enhanced reality environment wherein they can interact with the enhanced environment. For example, a head-mounted display, using stereoscopic display devices, allows a user to see, and become immersed in, any desired virtual scene. Such virtual reality applications also provide visual stimuli, auditory stimuli, and can track user movement to create a rich immersive experience.
Enhanced reality systems include virtual reality (VR) systems, augmented reality (AR) systems, and mixed reality (MR) systems. Such enhanced reality systems can include enhanced reality headsets to generate realistic images, sounds, and other human discernable sensations that simulate a user's physical presence in a virtual environment presented at the headset. A VR system includes physical spaces and/or multi-projected environments. AR systems may include those systems and devices that implement live direct and/or indirect displays of a physical, real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics and/or GPS data. MR systems merge real and virtual worlds to produce new environments and visualizations where physical and digital objects co-exist and interact in real time. For simplicity, VR systems, AR systems, and MR systems are referred to herein as enhanced reality systems.
While such enhanced reality systems have undoubtedly provided a valuable tool in many industries as well as a source of diversion for users, some characteristics impede their more complete implementation. For example, when interacting with the physical world, tactile or haptic feedback provides valuable sensory information to an individual. For example, as a user grasps a tool or other object in their hand, the mass of the tool indicates to the user information relating to the object they are grasping. Information such as its weight, density, and firmness. This information can provide a user with valuable sensory feedback, such as whether they have gripped the object sufficiently to pick it up. The sensory feedback can provide additional information on which a user can rely to process a situation. The replication of this sensory information in an enhanced reality environment is difficult to replicate.
Moreover, such devices, when power sources or fluid sources are located off the glove, may be cumbersome and may restrict movement. Both of these effects may impair the usability of the VR/AR/MR system.
Accordingly, the present specification describes a system that replicates, in the enhanced reality environment, the sensory information that a user receives when interacting with objects in the physical environment.
Specifically, the present specification describes a computing input device. The computing input device includes a glove to be worn by a user. The computing input device also includes a self-contained haptic feedback system disposed on the glove. The self-contained haptic feedback system includes a number of sensors disposed in the glove to collect information from an inserted hand. An inflatable haptic feedback device is positioned over a palm region of the glove and a control system provides fluidic and electronic control over the inflatable haptic feedback device.
The present specification also describes a method. According to the method, interaction data is received from a computing input device. The interaction data indicates interaction of a virtual hand of a user within an enhanced reality environment. The interaction data is converted into control data which activates an inflatable haptic feedback device. The inflatable haptic feedback device is located in a palm region of a glove. A pump that is also disposed on the glove is then actuated to inflate the inflatable haptic feedback device based on the control data.
The present specification also describes another example of a computing input device. In this example, the computing input device includes a glove to be worn by a user and a self-contained haptic feedback system disposed on the glove. The self-contained haptic feedback system includes the number of sensors. A first set of sensors generates position data. The control system of the self-contained haptic feedback system forms a representation of the inserted hand in the enhanced reality environment based on the position data. The computing input device also includes the inflatable haptic feedback device. In this example, the control system includes a pump disposed on the glove to inflate and deflate the inflatable haptic feedback device based on received control data. The self-contained haptic feedback system also includes a flexible battery disposed on a dorsal region of the glove to power the number of sensors, the inflatable haptic feedback device, and the pump. The computing input device also includes a number of additional haptic feedback devices disposed on the glove.
In summary, using such a computing input device 1) digitizes a user's hand thus making it visible in the enhanced reality environment; 2) provides a user with a sensation of physically holding an object represented in the enhanced reality environment; 3) creates a more immersive enhanced reality environment; and 4) is self-contained to enhanced user mobility. However, the devices disclosed herein may address other matters and deficiencies in a number of technical areas.
Turning now to the figures,
Disposed on the glove (102) is a haptic feedback system (104) that is self-contained. That is, the haptic feedback system (104) may reside entirely on the glove (102) with each component that makes up the haptic feedback system (104) disposed on the glove (102). In this fashion, the glove (102) may be a standalone unit that does not rely on any external mechanical components to provide the haptic feedback. For example, as will be described below a control system (110) or a battery for the haptic feedback system (104) are contained on the glove (102).
A haptic feedback system (104) provides a tactile, or touch sensation to the user. In some examples this feedback may be related to the enhanced reality environment in which a user is immersed. For example, a user may be immersed in an enhanced reality environment and in that environment the user may grab a tool. The haptic feedback system (104) of the present specification may provide haptic feedback to replicate the handle of the tool such that the user not only sees him/herself picking up the tool, but feels the tool in their hand thus enhancing the immersive experience.
To create such an effect, the haptic feedback system (104) includes various components. For example, the haptic feedback system (104) includes a number of sensors (106) disposed in, or on, the glove (102) to collect information from an inserted hand. In some examples, the sensors (106) may be motion sensors (106). In this example, the sensors can detect the position and/or motion of a hand within the glove (102) such that a reproduction of the position and/or movement is generated in the enhanced reality environment. The sensors (106) may also include heat sensors and/or pressure sensors to deliver additional information related to the hand. Such data can be used to 1) facilitate the generation of the digital representation of the hand, 2) facilitate immersion into the enhanced reality environment, and 3) select the form and/or characteristics of the haptic feedback.
The self-contained haptic feedback system (104) also includes an inflatable haptic feedback device (108). The inflatable haptic feedback device (108) may be positioned over a palm region of the glove (102). The inflatable haptic feedback device (108) can be inflated and deflated based on a corresponding action in the enhanced reality environment. For example, the inflatable haptic feedback device (108) may be a bladder that fills with a fluid such as air or water to replicate a virtual object in an enhanced reality environment.
As a specific example, a virtual hand may grip an object or may be placed against a representation of a surface such as a wall. In this example, the inflatable haptic feedback device (108) can be inflated to represent the physical contact with the object or placement of the hand against a physical wall. The inflatable haptic feedback device (108) may be formed of a variety of materials including urethane, flexible polyvinyl chloride (PVC), and nylon, among others.
In some examples, the inflatable haptic feedback device (108) has a shape to match the ergonomics of the hand, and more particularly to match the particular ergonomics of a particular user's hand. By using an inflatable haptic feedback device (108) that matches the ergonomics of the hand, a natural touch sensation can be passed to the user.
In some examples, the inflatable haptic feedback device (108) may be covered by a material that stretches based on the inflation and deflation of the inflatable haptic feedback device (108). Such a stretchable and flexible material allows the inflatable haptic feedback device (108) to inflate and deflate while providing protection against mechanical damage, such as puncture, of the inflatable haptic feedback device (108).
The haptic feedback system (104) also includes a control system (110) to provide a fluidic and electronic control over the inflatable haptic feedback device (108). That is, fluid such as air or water may be pumped into the inflatable haptic feedback device (108) during inflation. The fluid may be drawn from the inflatable haptic feedback device (108) during deflation. Accordingly, the control system (110) may include the pathways as well as the pump to facilitate the fluid transportation.
The control system (110) may also provide electronic control. That is, the control system (110) may manage the power delivery to a pump that moves the fluid. As described above, the haptic feedback system (104) may be self-contained on the glove (102). Accordingly, the control system (110), like the other components, may be disposed on the glove (102). In some examples, the control system (110) may be on a same side of the glove (102) as the inflatable haptic feedback device (108) and in other examples may be on an opposite side of the glove (102) as the inflatable haptic feedback device (108).
As described herein, the computing input device (100) that includes a glove (102) with a self-contained inflatable haptic feedback system (104) allows for a more immersive enhanced reality experience by generating haptic feedback that replicates physical contact. Such a system may be implemented in a variety of scenarios including gaming and business-related applications such as industrial design.
For example, the self-contained haptic feedback system (104), and specifically the control system (110), may include a pump (212). Like other components of the self-contained haptic feedback system (104), the pump (212) may be disposed entirely on the glove (102). The pump (212) operates to inflate and deflate the inflatable haptic feedback device (108). Inflating and deflating the inflatable haptic feedback device (108) may be based on control data from the control system (110). For example, within the enhanced reality environment a user may grasp a virtual object or otherwise interact with a virtual object. The control system (110) may receive information defining such contact. The control system (110) then converts this interaction data into pump (212) control data to inflate the inflatable haptic feedback device (108) to a certain degree to replicate that interaction.
The self-contained haptic feedback system (104) may also include a flexible battery (214). The flexible battery (214) provides operating power to the components of the self-contained haptic feedback system (104). In this example, the flexible battery (214) may be formed of a thin material that flexes, but does not plastically deform. That is, for a given amount of force the flexible battery (214) can conform to a shape. As will be described below, the flexible battery (214) may be positioned within the walls of the glove (102) in a location that corresponds to a neutral surface of the hand. In this context, neutral refers to a portion of the hand that does not move or flex much. Examples of such locations includes that region just below the wrist joint or the dorsal portion of the wrist.
The flexible battery (214) may be charged via a number of methods. In one example, the flexible battery (214) is charged via a tethering cable. In this example, the flexible battery (214) is coupled to an electrical outlet and charged. In another example, the flexible battery (214) is charged wirelessly. That is, the flexible battery (214) may include components that when proximate to a charging station, replenish the flexible battery (214). Examples of wireless charging methods include using wireless charging coils within the glove (102) that are placed in close proximity contact with coils in a charging station. The proximity of the coils triggers transfer of electrical power wirelessly. In this and other examples, the charging components of the glove (102) and the charging station do not have to be in direct contact, but can be within a specified range. In these examples, the wireless power can “jump” and charge the flexible battery (214).
Another example of wireless charging is optical wireless charging. In this example, the glove (102) includes a small window placed on top of the flexible battery (214). When placed in proximity to a wireless power emitter, the emitter distributes a focused electrical power stream through the window, thus charging the flexible battery (214). Other examples of charging the flexible battery (214) include via thermoelectric harnessing where a small thermoelectric generator harnesses electrical power from the skin while the user is wearing the glove (102). This energy is used to power the haptic feedback system (104) or to charge the flexible battery (214). In another example, the flexible battery (214) may be photovoltaic, meaning ambient light charges the flexible battery (214) or otherwise provides power to the components of the haptic feedback system (104).
By including the flexible battery (214) on the glove (102), the glove (102) operates as a standalone feedback device that both provides haptic feedback and powers the components that provide the haptic feedback. Doing so is advantageous as a user need not be tethered to a power source during use of the haptic feedback system (104). Such a tethered power system can impede the immersive nature of the experience. Moreover, by not having the haptic feedback system (104) on the glove (102) draw power from somewhere else on the body, for example, a battery disposed on an arm region, full mobility is possible.
In this example the haptic feedback system (104), and specifically the control system (110), also includes an input/output device (216) that allows the haptic feedback system (104) to communicate with other enhanced reality interaction components. For example, the self-contained haptic feedback system (104) may interact with other systems, such as other articles of clothing that similarly have sensors and feedback devices for immersing a user in an enhanced reality environment. As another example, the haptic feedback system (104) may interact with a computing input device that generates the enhanced reality environment, an application that uses the enhanced reality environment, and other computing input devices such as smartphones. The input/output device (216) facilitates such communication. In some examples, the input/output device (216) may include an antenna, such as a short distance antenna, to facilitate this communication. Other examples of devices that communicate with the self-contained haptic feedback system (104) on the glove (102) include an enhanced reality display device in the form of wearable goggles, an enhanced reality auditory device, and enhanced reality controllers.
In this example, the self-contained haptic feedback system (104) includes additional haptic feedback devices (218) disposed at different places on the glove (102). Such additional haptic feedback devices (218) provide feedback that can further enhance the experience. For example, heat feedback devices may generate a heat sensation when a user in the enhanced environment interacts with a hot object. In another example a user may receive alerts via vibration feedback devices. Another example of a feedback device is a pressure feedback device. While specific reference is made to particular feedback devices, any number of feedback devices may be implemented on the self-contained haptic feedback system (104) found on the glove (102).
Such additional haptic feedback devices (218) may be disposed at various locations on the glove (102). For example, additional haptic feedback devices (218) may be disposed at different points along the finger regions of the glove (102) to provide sensory feedback at those areas which are sensitive to touch. Such additional haptic feedback devices (218) provide additional opportunities for the user to feel completely entrenched in the enhanced reality environment.
The self-contained haptic feedback system (
The self-contained haptic feedback system (
This received data is converted (block 502) into control data. That is, the data indicative of a user interacting with the enhanced reality environment, along with additional control signals are used to operate the inflatable haptic feedback device (
Accordingly, the control system (
According to the method (600) interaction data indicating the user is interacting with the enhanced reality environment is received (block 603) and converted (block 604) into feedback data. This may be performed as described above in connection with
In this example, the self-contained haptic feedback system (
In summary, using such a computing input device 1) digitizes a users hand thus making it visible in the enhanced reality environment; 2) provides a user with a sensation of physically holding an object represented in the enhanced reality environment; 3) creates a more immersive enhanced reality environment; and 4) is self-contained to enhanced user mobility. However, the devices disclosed herein may address other matters and deficiencies in a number of technical areas.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/US2018/033649 | 5/21/2018 | WO | 00 |