Patent Application
20170075480
This application claims priority from European Application No. 15306415.9, entitled “Methods and Apparatus Of Transmitting A Rotation Angle Information to A Set Of At Least One Actuator Associated With A Surface,” filed on Sep. 15, 2015, the contents of which are hereby incorporated by reference in its entirety.
The present disclosure relates to rendering haptic properties of a texture on a surface when rubbing a surface, for example when the surface is a touch screen of a tablet or a smartphone.
Tablets and smartphones allow users to use their sense of touch to interact with media content such as video and pictures. For example, a user may “feel” the texture or the roughness of an image when he/she is touching a device by the means of haptic effects such as vibrations generated by actuators embedded in the end-user devices or roughness variations using dedicated “smart surfaces”.
Texture rendering on a surface can be achieved by the means of versatile renderers able to change their own surface properties depending on the underlying texture properties to render. One example of such renderers implements an array of pins going up and down (like a dynamic braille surface) and/or left and right. In this way, the surface granularity may be modulated. This approach has the disadvantage not to be adapted to touch screens as it disturbs the viewing conditions. Another approach makes use of embedded actuators (e.g. piezoelectric vibrators) to give vibrations as a feedback. Another approach addresses the problem of texture rendering by translating the screen in space. Some systems use horizontal translations (in the plane of the screen, left and right for example), others use backward and forward translations (the screen moves away and closer to the user's finger).
The purpose of the present disclosure is to overcome at least one of the disadvantages of the background. The present disclosure relates to a method of transmitting a rotation angle information to a set of at least one actuator associated with a surface, the method comprising:
According to a particular characteristic, the rotation angle information is equal to the first rotation angle.
According to a particular characteristic, the method further comprises:
Advantageously, the method further comprises:
In a variant, the translation vector information is equal to the first translation vector. In another variant, the translation vector information is an interpolation between a first and a second translation vector computed according to a first and a second rubbing contact and properties of the texture.
The present disclosure also relates to an apparatus configured for transmitting a rotation angle information to a set of at least one actuator associated with a surface, the apparatus comprising:
The present disclosure also relates to an apparatus configured for transmitting a rotation angle information to a set of at least one actuator associated with a surface, the apparatus comprising :
The present disclosure also relates to a computer program product comprising instructions of program code for executing, by at least one processor, the abovementioned method of transmitting a rotation angle information to a set of at least one actuator associated with a surface, when the program is executed on a computer.
The present disclosure also relates to a (non-transitory) processor readable medium having stored therein instructions for causing a processor to perform at least the abovementioned method of transmitting a rotation angle information to a set of at least one actuator associated with a surface.
The present disclosure will be better understood, and other specific features and advantages will emerge upon reading the following description, the description making reference to the annexed drawings wherein:
The subject matter is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the subject matter. It is understood that subject matter embodiments can be practiced without these specific details.
As used herein, a “texture” is an amount of data representative of the haptic properties of a surface of a real-world material. A texture comprises models describing, for example, friction, roughness and embossment of the surface of a material. For instance, rough stone is a coarse material with some granularity, velvet is smooth, sponge is soft and embossed and ice is flat and slippery. Texture may also comprise an image visually representative of the associated material.
As used herein, a “rubbing contact”, is a physical contact between a surface and at least one object, the continuous contact moving on the surface with a non-zero speed.
The present principles will be described in reference to a particular example of a method of rendering haptic texture properties on a surface associated with a set of at least one actuator when the user is rubbing the surface. A system is used to detect parameters of a rubbing contact between the surface and at least one object (e.g. fingers of the user or a stylus). To render the texture properties as the location of the contact, a rotation angle around an axis perpendicular to the direction of the rubbing contact is computed and transmitted to the at least one actuator. The at least one actuator makes the surface rotate around the provided axis. A slope is created in favour or in opposition to the current rubbing contact motion in order to simulate a lower or higher friction feedback depending on the underlying texture properties at the current contact position. The amplitude of the slope is a function of the friction level at the point of the rubbing contact and of parameters of the rubbing contact (e.g. the rubbing speed, the contact pressure, the contact area, . . . ). According to an optional variant, in order to avoid potential saccadic motions which could decrease the quality of experience, the present principles further comprise interpolating the rotation angle over the time.
γ=g({right arrow over (v)},f(xc,yc)) [E1]:
In the equation [E1]:
The function g determines the rotation angle in order to command to the set of at least one actuator to rotate the surface in the opposite or same direction than the direction of the rubbing contact. For instance, if in the direction the rubbing contact is moving, the friction parameter of the texture is high, the surface will incline oppositely to the movement of the rubbing contact in order to make a rising slope and to make the user's displacement harder. On the contrary, if the friction parameter of the texture is low (e.g. a slippery texture), the surface will incline according to the direction of the rubbing contact to create a falling slope and then to make the user's displacement easier.
Applied directly, a model like the equation [E1] could induce saccadic motions which could potentially decrease the quality of experience. To avoid such potential saccadic motions, the rotation angle may be filtered making use of quaternion interpolation, according to a non-limiting embodiment. The apparatus considers a first pose of the surface, which is the pose of the surface at the moment of the current rubbing contact. A first rotation angle information (i.e. an axis and an rotation angle) is computed according to an equation of the form of [E1]. The apparatus determines a second pose, which is the first pose rotated by the rotation angle information. The first pose is represented by a quaternion called qt−dt, and the second pose is represented by a quaternion called qt. A second rotation angle information is computed according to a quaternion intermediate between qt−dt and qt, determined by an equation like [E2] for example.
q=q
t−dt(qt−dt−1·qt)β with β∈[0,1] [E2]
The parameter β controls the smoothness of the pose trajectory. High values of β leading to brutal change while low values correspond to slight changes. The second rotation angle information is transmitted to the set of at least one actuator.
For the sake of clarity, the reference plane is considered horizontal. Each actuators may make the surface go up or down along the z-axis independently to cause a rotation of the surface. At rest position, the surface is horizontal and the four actuators lie at a rest elevation called zh. To induce friction, these elevations are shifted by a value dz1 for the actuator (21), dz2 for the actuator (22), dz3 for the actuator (23) and dz4 for the actuator (24) making the surface rotate. Setting the values dz1, dz2, dz3 and dz4 may consist in solving a non-linear least-square optimization at each time t as in equation [E3].
Where q(dz1,dz2,dz3) represents a quaternion associated with the pose of the surface computed according location (11), direction and speed (12) of the rubbing contact. The elevation shift dz4 is linearly correlated to the three previous elevation shifts due to the hyper-static configuration of the system illustrated in
A set of actuators as illustrated in
In another embodiment, the surface is nested in a box on which the actuators are fixed. The reference plane of the surface depends on the inclination of the box. In another embodiment, the at least one actuator is a robotic arm which can make the surface rotate and translate in every directions. The surface may translate in its own plane (e.g. horizontally, at the rest pose).
Advantageously, the device (30) is connected to one or more display devices (38) of display screen type directly to the graphics card (32) to display images calculated in the graphics card. In a particular embodiment, the contact receiver (35) is a touch screen and the display (38) is integrated to the apparatus (30).
The apparatus (30) is connected, for example via the link (36), to a set of at least one actuator (39).
It is noted that the word “register” used in the description of memories (321), (33) and (34) designates in each of the memories mentioned, both a memory zone of low capacity (some binary data) as well as a memory zone of large capacity (enabling a whole program to be stored or all or part of the data representative of data calculated or to be displayed).
When switched-on, the microprocessor (31) loads and executes the instructions of the program contained in the RAM (34).
The random access memory (34) notably comprises:
According to one particular embodiment, the algorithms implementing the steps of the method specific to the present disclosure and described hereafter are advantageously stored in the memory GRAM (321) of the graphics card (32) associated with the device (30) implementing these steps. When switched on and once the data (341) are loaded into the RAM (33), the graphic processors (320) of the graphics card (32) loads these data into the GRAM (521) and execute the instructions of these algorithms in the form of microprograms of “shader” type using HLSL (High Level Shader Language) language or GLSL (OpenGL Shading Language) for example.
The random access memory GRAM (321) notably comprises data representative of the texture and may contain an image representative of the texture (3210).
According to another variant, a part of the RAM (33) is assigned by the CPU (31) for storage of the algorithms. According to another variant, the power supply (37) is external to the device (30).
In an initialization step (41), the apparatus obtains data representative of the texture, stored these data in a register (341) and reset its computational values according to the rest pose (i.e. default values at the rest pose).
A step (42) consists in receiving parameters of a rubbing contact. In a first embodiment, the surface is a touch surface and the rubbing contact is detected on and by the touch surface. The data representative of the rubbing contact comprise parameters such as the touch area, the duration of the contact, the direction and the speed of the touch displacement and the pressure over the touch area. All these parameters are detected by the touch surface. In a second embodiment, the rubbing contact is detected by an external system such as a system of infra-red cameras or a pressure detection system. Parameters of the detected rubbing contact are transmitted to the apparatus (30) and received by the contact receiver (35). When a change is detected in at least one parameter of the rubbing contact, the step (43) is executed. If no change is detected, the surface does not need to rotate or to translate and the step (43) is not executed. According to a variant, the step (43) is executed at a constant rate even if no change is detected in the rubbing contact parameters.
At a step (43), rotation angle information and/or translation vector information is/are computed according to the properties of the texture (341) at the rubbing contact location and to the rubbing contact parameters, in particular direction and speed of the rubbing contact and according to the principles presented in this disclosure. The rotation angle information and/or the translation vector information is/are transmitted to the set of at least one actuator at a step (44) that is executed each time a new information is computed. In a variant, both rotation angle information and translation vector information are transmitted to the set of at least one actuator at a constant rate, even if they did not change.
The method (50) begins with the same steps (41), (42), (43), (44) than the method (40). A step (51) consists in obtaining a second set of parameters of the rubbing contact, an instant after the step (42). A step (52) is executed to compute a second rotation angle information and/or a second translation vector information the same way the first rotation angle information and/or first translation vector information are computed. At a step (53), a third rotation angle information is computed as an interpolation between the first and the second rotation angle information. According to a variant, the third rotation angle information is obtained by interpolating quaternions associated with resulting poses of the surface once the first rotation angle information and the second rotation information are applied to the surface by the set of at least one actuator. A third translation vector information is computed as an interpolation between the first and the second translation vector information. In a particular embodiment, the rotation angle information and the translation vector information are combined in one motion information. The computed information is transmitted to the set of at least one actuator at a step (54).
At a step (55), the parameters of the second rubbing contact replace the parameters of the first rubbing contact in the registers of the apparatus (30). The same way, the second rotation angle information and the second translation vector information are stored in place of the first rotation angle information and the first translation vector information. In a variant, the second motion information is stored in place of the first motion information. At this step (55), the apparatus (30) is ready to receive new rubbing contact parameters which are stored as second rubbing contact parameters at step (51).
Naturally, the present disclosure is not limited to the embodiments previously described. In particular, the present disclosure is not limited to a method of transmitting a rotation angle information to a set of at least one actuator associated with a surface but also extends to any method of rendering any texture property (e.g. roughness, embossment, granularity) implementing this rotating and/or translating rendering method. The implementation of calculations necessary to render the texture properties is not limited either to an implementation in shader type microprograms but also extends to an implementation in any program type, for example programs that can be executed by a CPU type microprocessor.
The implementations described herein may be implemented in, for example, a method or a process, an apparatus, a software program, a data stream, or a signal. Even if only discussed in the context of a single form of implementation (for example, discussed only as a method or an apparatus), the implementation of features discussed may also be implemented in other forms (for example a program). An apparatus may be implemented in, for example, appropriate hardware, software, and firmware. The methods may be implemented in, for example, an apparatus such as, for example, a processor, which refers to processing devices in general, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. Processors also include communication devices, such as, for example, smartphones, tablets, computers, mobile phones, portable/personal digital assistants (“PDAs”), and other devices.
Implementations of the various processes and features described herein may be embodied in a variety of different equipment or applications, particularly, for example, equipment or applications associated with data encoding, data decoding, view generation, texture processing, and other processing of images and related texture information and/or depth information. Examples of such equipment include an encoder, a decoder, a post-processor processing output from a decoder, a pre-processor providing input to an encoder, a video coder, a video decoder, a web server, a set-top box, a laptop, a personal computer, a cell phone, a PDA, and other communication devices. As should be clear, the equipment may be mobile and even installed in a mobile vehicle.
Additionally, the methods may be implemented by instructions being performed by a processor, and such instructions (and/or data values produced by an implementation) may be stored on a processor-readable medium such as, for example, an integrated circuit, a software carrier or other storage device such as, for example, a hard disk, a compact diskette (“CD”), an optical disc (such as, for example, a DVD, often referred to as a digital versatile disc or a digital video disc), a random access memory (“RAM”), or a read-only memory (“ROM”). The instructions may form an application program tangibly embodied on a processor-readable medium. Instructions may be, for example, in hardware, firmware, software, or a combination. Instructions may be found in, for example, an operating system, a separate application, or a combination of the two. A processor may be characterized, therefore, as, for example, both a device configured to carry out a process and a device that includes a processor-readable medium (such as a storage device) having instructions for carrying out a process. Further, a processor-readable medium may store, in addition to or in lieu of instructions, data values produced by an implementation.
As will be evident to one of skill in the art, implementations may produce a variety of signals formatted to carry information that may be, for example, stored or transmitted. The information may include, for example, instructions for performing a method, or data produced by one of the described implementations. For example, a signal may be formatted to carry as data the rules for writing or reading the syntax of a described embodiment, or to carry as data the actual syntax-values written by a described embodiment. Such a signal may be formatted, for example, as an electromagnetic wave (for example, using a radio frequency portion of spectrum) or as a baseband signal. The formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information that the signal carries may be, for example, analog or digital information. The signal may be transmitted over a variety of different wired or wireless links, as is known. The signal may be stored on a processor-readable medium.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made. For example, elements of different implementations may be combined, supplemented, modified, or removed to produce other implementations. Additionally, one of ordinary skill will understand that other structures and processes may be substituted for those disclosed and the resulting implementations will perform at least substantially the same function(s), in at least substantially the same way(s), to achieve at least substantially the same result(s) as the implementations disclosed. Accordingly, these and other implementations are contemplated by this application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 15306415.9 | Sep 2015 | EP | regional |
