Patent Application
20160357258
The present invention relates generally to an apparatus for providing haptic force feedback to a user interacting with a virtual object in a virtual space.
As 3D image-forming technology develops, research on virtual reality is being actively carried out. A technique for an interaction in virtual reality is developing continuously, and the most crucial factor is to implement a technique for creating a sense of realism in the virtual reality. The senses used in the interaction in the virtual space include sight, hearing, taste, smell, and touch, and haptic technology using the sensation of touch is the technology that is available to a variety of fields such as medicine, games, robots, etc.
The present invention is intended to propose an apparatus that is held on a portion of the user's body so as to provide force feedback arising from the interaction in a virtual space to the user.
Accordingly, the present invention has been made keeping in mind the above problems occurring in the related art, and an object of the present invention is to enable a user manipulating a 3D virtual object in a 3D virtual space to realize, from a virtual article, real sensation of touch that the user can feel as he/she touches or moves a certain real article in a real space.
Another object of the present invention is to enable 3D eyeglasses or a head-mounted device (HMD) interacting with a virtual article to obtain a direct sensation of touch for the virtual article that is similar to a real sensation of touch that is obtained when a certain real article is manipulated in a real space.
A further object of the present invention is to enable physical feedback in a virtual environment system so as to prevent a phenomenon that a 3D object separates a certain distance from a virtual article without touch, or penetrates through the virtual article.
In order to achieve the above objects, according to one aspect of the present invention, there is provided an apparatus for providing haptic force feedback to a user interacting with a virtual object in a virtual space, the apparatus including: a casing wearable around at least a portion of the user's body; a force-feedback provision unit coupled directly or indirectly to the casing and having a first end directly or indirectly touching at least a portion of the user's body and a second end coupled directly or indirectly to the casing through an elastic member to allow the first end to provide the sensation of touch arising from an interaction with the virtual object to the at least a portion of the user's body; a communication unit coupled directly or indirectly to the casing to communicate with an external terminal; and a processor unit coupled to directly or indirectly to the casing and configured to perform (i) a process of receiving at least one of: a first force-feedback control data generated based on vector information on a motion of the at least a portion of the user's body wearing the casing and information on the virtual object; and a second force-feedback control data generated based on image information on a motion of the at least a portion of the user's body wearing the casing and information on the virtual object, from the external terminal via the communication unit so as to control the force-feedback provision unit, or (ii) a process of generating at least one of the first and second force-feedback control data so as to control the force-feedback provision unit.
According to the foregoing, the present invention provides an effect of enabling a user manipulating a 3D virtual object in a 3D virtual space to realize, from a virtual article, real sensation of touch that the user can feel as he/she touches or moves a certain real article in a real space.
Further, the present invention provides an effect of enabling 3D eyeglasses or a head-mounted device (HMD) interacting with a virtual article to obtain a direct sensation of touch for the virtual article that is similar to a real sensation of touch that is obtained when a certain real article is manipulated in a real space, thereby improving an interaction and manipulation performance with respect to the virtual article.
Still further, the present invention provides an effect of enabling a user touching a virtual object during interacting with the virtual object using a 3D sub-visualization device to receive the touch feedback, thereby providing a real sensation of touch or impact for the virtual object and allowing the user to feel realism in manipulating the virtual object.
The above and other objects, features and advantages of the present invention will be more clearly understood from the following detailed description when taken in conjunction with the accompanying drawings, in which:
Reference will now be made in greater detail to exemplary embodiments of the invention, an example of which is illustrated in the accompanying drawings. These embodiments are described in detail such that a person skilled in the art can sufficiently implement those embodiments through the description. It is noted that, although being different from each other, various embodiments are not required to be necessarily mutually exclusive. For example, specified shapes, structures, and features of an embodiment described herein may be implemented into other embodiments without departing from the spirit and scope of the present invention. Further, it should be understood that positions or arrangements of individual elements in respective disclosed embodiments may be changed into other positions or arrangements without departing from the spirit and scope of the present invention. Accordingly, the following description is not intended to be limitative, and if appropriately described, the scope of the present invention is only defined by the appended claims and equivalents thereof. In the figures, like reference numerals refer to the same or similar functions throughout many aspects.
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to allow an ordinary skilled person in the art to which the present invention pertains to easily implement the invention.
As illustrated in
The force-feedback provision apparatus 100 according to embodiments of the present invention may be configured into a shape wearable around at least a portion of the user's body, so a user can manipulate a virtual object 300, wearing the force-feedback provision apparatus 100. Here, the virtual object 300 may be a virtual object output in a certain 3D space, but may also include a 2D virtual object.
In order to provide a user with force feedback arising from an interaction with the virtual object 300 through the force-feedback provision apparatus 100, there is a need to provide a process of checking a touch state between the force-feedback provision apparatus 100 worn by the user and the virtual object 300 and a process of generating data to control the force-feedback provision apparatus 100 according to the touch state. These processes may be carried out through the force-feedback provision apparatus 100 or the external terminal 200 operatively associated with the force-feedback provision apparatus 100. This will be described later. The external terminal 200 may include any of digital devices, such as desktop computers, notebook computers, workstations, PDAs, web pads, mobile phones, head-mounted devices (HMDs), etc., so long as they have a calculation capability through memories and microprocessors.
Referring to
Specifically, the force-feedback provision apparatus 100 according to embodiments of the present invention may be provided with a plurality of force-feedback provision units 120 for providing force feedback to the user's finger, so that sensation of touch such as muscular sensation, texture, a sense of direction, etc. can be provided to the user by controlling the force-feedback provision units 120. Although the force-feedback provision unit 120 is configured such that a first end thereof contacts the user's finger according to embodiments of the present invention, the first end may indirectly contact the user's finger, since a cover may be disposed above the force-feedback provision unit 120. The force-feedback provision unit 120 will be described in detail with reference to
As illustrated in
The casing 110 has a shape wearable around at least a portion of the user's body. For example, the casing may consist of an outer casing part 115 and an inner casing part 116 that can be coupled together. Here, the outer casing part 115 may consist of first and second outer casing parts 115a and 115b that can be coupled together, and the inner casing part 116 may consist of a first inner casing part 116a to be coupled to the first outer casing part 115a, and a second inner casing part 116b to be coupled to the second outer casing part 115b.
In the meantime, partition receptacles 111 may be provided between the outer casing part 115 and the inner casing part 116 to respectively accommodate the force-feedback provision units 120 therein. Further, although the communication unit 130, the force-feedback provision unit 120, the processor unit 140, and the power supply 150 may be installed between the outer casing part 115 and the inner casing part 116, the present invention is not limited thereto. For example, when a photovoltaic cell is used as the power supply 150, the photovoltaic cell may be installed directly or indirectly on an outer surface of the casing 110.
According to an embodiment, the partition receptacle 111 provided between the outer casing part 115 and the inner casing part 116 may be coupled to the inner casing part 116. Here, the partition receptacle 111 may have an opening (not shown) on an upper portion thereof, and the inner casing part 116 to be coupled to the upper portion of the partition receptacle 111 may have a hole 112 corresponding to the opening, so that the first end of the force-feedback provision unit 120 accommodated in a space of the partition receptacle 111 protrudes into the inner casing part 116 through the opening and the hole 112, thereby providing force feedback to the user's finger inserted into the inner casing part 116.
As illustrated in
Specifically, the pin section 121 is an element that serves to transfer the sensation of touch directly or indirectly to at least a portion of the user's body.
The actuator 122 may be installed such that an upper end thereof is operatively associated with at least a portion of a lower end of the pin section 121, and a lower end thereof is operatively associated with at least a portion of the elastic member 123 in order to vibrate the pin section 121 under the control of the processor unit 140.
Here, since at least a portion of the elastic member 123 is coupled to the outer casing part 115a, 115b, the elastic member 123 can support the actuator 122 such that vibrations generated by the actuator 122 can be transferred not towards the lower end (i.e., a direction towards the outer casing part), but towards the upper end of the actuator 122. Here, an upper end of the pin section 121 can protrude into the inner casing part 116 through the hole 112 shown in
The vibration-absorption member 124 may have a structure surrounding at least a portion of the pin section 121 and the actuator 122, so that the vibration-absorption member can prevent vibrations generated by the actuator 122 from being transferred to a portion other than the specified part of the user's body with which the pin section 121 facing the actuator 122 comes into contact. Here, the partition receptacle 111 coupled directly or indirectly to at least a portion of an outer surface of the vibration-absorption member 124 may also prevent vibrations generated by the force-feedback provision unit 120 from being transferred to a portion other than the specified part of the user's body. This will be described later with reference to
The structure shown in
A first space is formed between a lateral side of the actuator 122 and an inner side of the vibration-absorption member 124 so as to allow the actuator 122 to vibrate horizontally in the first space, and a second space is formed between the inner side of the vibration-absorption member 124 and an outer side of the pin section 121 so as to allow the pin section 121 to vibrate vertically in the second space by the action of the actuator 122.
When the force-feedback provision apparatus 100 is provided with the plurality of force-feedback provision units 120, the vibration-absorption member 124 can prevent horizontal vibrations generated in the first space by the actuator 122 from being transferred to another force-feedback provision unit 120 adjoining to the force-feedback provision unit 120 shown in
In the meantime, the vibration-absorption member 124 may be formed from a silicone rubber pad, the actuator 122 may be any one of a vibration motor, a linear motor, a VCM motor, a step motor, a servo motor, a solenoid valve, and a source of hydropneumatic pressure, and the elastic member 123 may be a spring. In the case of the force-feedback provision unit 120 shown in
Referring again to
The processes (i) and (ii) performed by the processor unit 140 will be described in detail.
First, the process (i) will now be described.
The force-feedback provision apparatus 100 may further include a sensor unit (not shown), such as a gyroscope sensor, an acceleration sensor, etc., which is coupled directly or indirectly to the casing 110 so as to sense vector information according to a motion of at least a portion of the user's body. In this case, the processor unit 140 is configured to acquire vector information from the sensor unit; transmit the acquired information to the external terminal 200 via the communication unit 130 so as to determine a touch state between the at least a portion of the user's body and the virtual object 300 based on the vector information and the information on the virtual object; and if it is determined that the at least a portion of the user's body and the virtual object 300 are in contact, receive the first force-feedback control data generated based on the vector information and the information on the virtual object via the communication unit 130 so as to control the force-feedback provision unit 120.
Further, the processor unit 140 may be configured to, when the external terminal 200 generates the second force-feedback control data based on image information of a motion of the at least a portion of the user's body and the information on the virtual object 300, receive the second force-feedback control data from the external terminal 200 so as to control the force-feedback provision unit 120. Here, the image information may be an image itself that is picked up by a camera operatively associated with the external terminal 200, or information on an image picked up by the camera operatively associated with the external terminal 200 and then image-processed.
Next, the process (ii) will be described in detail.
The processor unit 140 may perform the process of acquiring vector information from the sensor unit; acquiring information on the virtual object 300 from the external terminal 200 via the communication unit 130; and determining a touch state between the at least a portion of the user's body and the virtual object 300 based on the vector information and the information on the virtual object. If it is determined that the at least a portion of the user's body and the virtual object 300 are in contact, the processor unit 140 may generate the first force-feedback control data based on the vector information and the information on the virtual object so as to control the force-feedback provision unit 120. Further, the processor unit 140 may perform the process of acquiring image information of a motion of the at least a portion of the user's body and information on the virtual object 300 from the external terminal 200 via the communication unit 130, and determining whether the at least a portion of the user's body and the virtual object 300 are in contact or not. If it is determined that the at least a portion of the user's body and the virtual object 300 are in contact, the processor unit may generate the second force-feedback control data based on the image information and the information on the virtual object so as to control the force-feedback provision unit 120.
As illustrated in
Although
Referring to
In the meantime, in the case where the plurality of force-feedback provision units 120 is installed in the force-feedback provision apparatus 100 as shown in
If at least a portion of the user's body interacts with a specified portion of the virtual object 300 in a single contact state without a motion, the processor unit 140 may perform a control operation only on the force-feedback provision unit 120 corresponding to the interaction, thereby enabling the user to realize the sensation of touch occurring at a point at which the user touches a real object. For example, if the user's finger wearing the force-feedback provision apparatus 100 is brought into contact with a specified portion of the virtual object 300 without a change in direction of the finger, the processor unit may perform a control operation only on a single force-feedback provision unit 120 (e.g. the force-feedback provision unit on the lower side) of the plurality of force-feedback provision units 120 that correspond to the interaction with the virtual object 300, thereby providing the user with stationary sensation of touch. Here, the processor unit 140 may control the actuator 122 of the force-feedback provision unit 120 to provide a different level of intensity of vibrations according to an inclined angle of the user's body (e.g. finger) or a shape, texture, etc. of the virtual object.
Further, in a case where at least a portion of the user's body interacts with a specified portion of the virtual object 300 in a single contact state without a motion (assuming that the force-feedback provision unit 120 corresponding to the interaction with the virtual object 300 is one disposed on the lower side as illustrated in the above paragraph), and the at least a portion of the user's body is moved on the virtual object 300 while maintaining the single contact state (i.e. the state of the finger being maintained in a fixed direction), if the user holds that portion of the his or her body contacting the virtual object 300 at the same position (i.e. if even during moving, a lower side one is maintained as the force-feedback provision unit 120 corresponding to the interaction with the virtual object 300), the processor unit 140 can manipulate the virtual object 300 while maintaining a constant force or sensation of touch, since the actuator 122 to be controlled does not change.
Further, in a case where at least a portion of the user's body interacts with a specified region of the virtual object 300 in a multi-contact state while moving on the virtual object, the processor unit 140 may control the N force-feedback provision units sequentially in a direction opposite the direction in which the at least a portion of the user's body moves, thereby enabling the user to perceive a sense of direction occurring at a contact surface when the user moves while touching a real object. For example, when a user's finger wearing the force-feedback provision apparatus 100 is moved from the left side to the right side with a specified surface thereof brought into contact with the virtual object 300, 3 force-feedback provision units 120 of the plurality of force-feedback provision units 120 that correspond to the interaction with the virtual object 300 are controlled sequentially from the right side to the left side, thereby providing dynamic sensation of touch to the user. Here, the processor unit 140 may individually control the 3 force-feedback provision units 120 to generate a different level of intensity of vibrations from the actuator 122. Assuming that the 3 force-feedback provision units 120 are called A, B, and C from the right side, and a level of the intensity of the actuator 122 is put in parentheses as numbers like 4 (highest level), 3 (high level), 2 (middle level), and 1 (low level), the force-feedback provision units can be controlled sequentially from A so as to provide a sense of direction to the user as follows:
While the present invention has been described with reference to specific configurations, such as elementary components, exemplary embodiments, and drawings, the description is provided merely for assisting more generalized understanding of the invention, so the present invention is not limited to the above-mentioned embodiments. An ordinary person skilled in the art may appreciate that such embodiments can be changed and modified into a variety of other forms through the description.
Therefore, it should be understood that the idea of the present invention is not defined as being limited to the exemplary embodiments, and such changes and modifications fall within the scope of the present invention defined by following claims and equivalents thereof.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2015-0080185 | Jun 2015 | KR | national |
