The present disclosure relates generally to improved calibration techniques for haptic-based systems.
A continuous distribution of sound energy, referred to as an “acoustic field”, may be used for a range of applications including haptic feedback in mid-air.
By defining one or more control points in space, the acoustic field can be controlled. Each point can be assigned a value equating to a desired amplitude at the control point. A physical two-dimensional array of transducers can then be controlled to create an acoustic field exhibiting the desired amplitude at the control points.
However, because complete control of space is not possible, controlling the acoustic field at given points yields erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area. The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. One of the ways to change how the space is controlled is by rearranging the transducer elements. By arranging the two-dimensional array of transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
How to best approach designing an array of transducers that generates control points in an acoustic field with the minimum of erroneous maxima is not trivial.
Further, in prior applications, transducer layouts that minimize the formation of these erroneous maxima were described. In reality, however, due to physical manufacturing and production constraints inherent in the process of electronic design and layout these ideal configurations are often not achievable. As a result, a method to create configurations that maintain the required beneficial properties of the ideal physical arrangement while accounting for such design limitations is needed. Although this process could be performed manually this would be time consuming and prone to human error and as such an automated approach would be preferable.
Finally, in order to create an appropriate acoustic field that responds to input, a sensor must be attached to the system. This sensor is required for interaction, but the correspondence between the coordinate space of the acoustic field and the coordinate space of the sensor, must be established first. A human can perform this initial calibration step, by actuating the device, prompting for a corresponding behavior and then measuring the output of the sensor in this situation. This provides the correspondence between the two vector spaces. However, this requires human interaction and so is subject to human error, as well as requiring human intervention in a more general sense.
The accompanying figures, where like reference numerals refer to identical or functionally similar elements throughout the separate views, together with the detailed description below, are incorporated in and form part of the specification, and serve to further illustrate embodiments of concepts that include the claimed invention, and explain various principles and advantages of those embodiments.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the dimensions of some of the elements in the figures may be exaggerated relative to other elements to help to improve understanding of embodiments of the present invention.
The apparatus and method components have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
Described herein are certain techniques to enhance calibration techniques in haptic systems. Some or all of these techniques may be used at the same time or one after the other in order to improve such calibration.
An acoustic field may be controlled by defining one or more control points in space. Each control point can be assigned a value equating to a desired amplitude at that control point. A physical set of transducers may then be controlled to create an acoustic field exhibiting the desired amplitude at the control points.
Because complete control of space is not possible, controlling the acoustic field at given points may yield erroneous local maxima in the acoustic field levels at other related positions. In relation to mid-air haptic feedback, these can interfere in interactions with the space by creating secondary effects and ghost phenomena that can be felt outside the interaction area.
The level and nature of the secondary maxima in the acoustic field is determined by how the space is controlled. One of the ways to change how the space is controlled is by rearranging the transducer elements. By arranging the transducer elements in different ways, unwanted effects on the acoustic field can be limited and controlled.
How to best approach designing an array of transducers that generates control points in an acoustic field with the minimum of erroneous maxima is not trivial. Existing array geometries will be reviewed to consider their drawbacks. New array geometries will then be reviewed to remedy these drawbacks.
1. Figure Definitions
2. Existing Array Configurations
Since the rectilinear structure is the source of these four ghost images 220230240250, a potential solution is to break up the rectilinear structure. As shown in
As shown by the hexagonal array simulation 400 in
As shown in
3. Phyllotactic Spiral Transducer Array
The issues outlined in the previous section necessitate the creation of a new type of array that has both uncorrelated secondary maxima that fall away from the control points and uniform density to avoid unexpected power fall off. A transducer arrangement that achieves both of these goals is desired with a geometry that has both uniform or predictable density and non-uniform sampling in any given direction to avoid ghost images.
The phyllotactic spiral pattern is common in nature where a dense packing of leaves to absorb the maximum amount of sunlight is needed. Its construction is based on angular proportions of the golden ratio (an irrational number) to produce alternating curved packing of elements that—while uniform and regular—can never result in a periodic sampling in any given direction. Two quantities are in the golden ratio (an irrational number, approximately 1.618) if their ratio is the same as the ratio of their sum to the larger of the two quantities.
As shown in
As shown in
The extra images in the regular array arrangements shown in
As shown, these images can be removed by creating a system wherein collinear groupings are minimized or there are no uniform separations. In these situations, the groupings do not have the same sampling approach as a discrete Fourier transform and so will not exhibit these effects. A random or Poisson disk sampling may be effective to eliminate these effects, but this has a downside of reducing the ability to pack the elements. Packing techniques that prevent uniform separation or collinearity are non-trivial as crystal structures in nature can attest. But the phyllotactic spiral design fulfills all these criteria by placing the transducer elements along curves that are constructed and governed by irrational numbers, minimizing the effects of collinearity.
Instead of using a square root distance (raising by a power of 0.5) that distributes transducers equally in array area, the exponent may be increased (raising by a power of the golden ratio minus one, 0.618) to spread the transducers more towards the edges. As shown in
By using the higher power in the exponent, the phyllotactic spiral relaxes the packing of the elements, making it more similar to the natural arrangement found in sunflowers. Here, the distribution of the transducers in a phyllotactic spiral pattern is sparser toward the edges of the phyllotactic spiral pattern. As the elements get further apart as they move away from the center, they may produce reduced haptic effects at the edge. However, these sparse transducers at the edge remain able to suppress excess noise, at the cost of a larger footprint and some extra transducers. This effect may justify the use of higher exponents in the production of the spiral.
The phyllotactic spiral array need not actually be in a complete spiral formation. The spiral may be cut into shapes that are more suitable for array footprints such as in a rectilinear, square or rectangle format. Thus one or more partial phyllotactic spiral patterns may be used to generate the desired effects. The use of the term “partial phyllotactic spiral pattern” herein may mean either a portion of a complete phyllotactic spiral pattern or a complete phyllotactic spiral pattern.
As shown in
Alternatively, the phyllotactic spiral may be cut into a frame for a device that gives mid-air haptic capabilities. As shown in
Moving the transducers in the z-direction (the direction orthogonal to the two-dimensional transducer array) and rotating them 45 degrees into a beveled “picture frame” arrangement verifies this result. In the arrangement shown in
Counter-intuitively, rotating each transducer further to point directly at the center of the shape actually degrades focusing performance, as the lack of resolving power perpendicular to the four sides of the array creates more powerful secondary maxima in the plane of the shape. In the arrangement shown in
The transducers need not all point in the same direction or be fixed at the same orientation in the z-axis.
While these transducer arrangements have been shown to use circular transducer elements the designs described are also applicable to differently shaped transducer elements, including square, rectangle or ellipse or other shapes.
4. Importance of Reduction and Decorrelation of Noise
Noise in the array output can cause a number of different phenomena. The first and most obvious side effect is that a high level of noise may interfere with the mid-air haptic qualities of the array. They may cause phantom points and low level stimulation that can drown out the intended signal. Another side effect is that since the array produces sound as a side effect, stronger random secondary maxima may result in more audible noise outside the working volume.
Reducing nearby secondary maxima will reduce the level of points of feedback that can interfere in the haptic sensation. De-correlating the noise with the feedback will smooth out noise more effectively by making it destructively interfere with itself spatially and temporally.
If the physical limitation of the placing of each electronic component is applied to the ideal layout by pruning transducers that are inappropriately placed, the design becomes sparse and inefficient.
1. Application of Physical Constraints
Transducers must be contained within the footprint of the printed circuit board.
2. Physical Constraint Optimization
Transducer layout design may be better optimized for physical constraints through the use of a simulation of two-dimensional physics. Each electrical component in the simulation is comprised of multiple layers. In each layer, an object interacts with other physical bodies that reside on the same layer. For example, a transducer body may be modeled as a primitive shape on one layer. A different but rigidly-connected shape on a separate layer represents the electrical pins connection beneath the transducer. In the above example, the transducer bodies interact with each other and with the edges of the printed circuit board. In contrast, the shapes that model the transducer pins interact only with the physical board connectors to ensure there is enough space to physically place the connector. Thus, the location of the component as a whole is optimized using the simulation of a physical process.
3. Simulation Behaviors that Enhance Design
A simulator (whether in hardware, software or both) configured to provide a physical simulation of such a constrained system may be used to provide an intuitive design tool for transducer placement tasks. The challenge is that the fulfillment of positive constraints is required in the face of a desired placement of components which violates negative constraints. Such an approach yields readily intuited results satisfying these constraints while attempting to give the closest placement to the ideal of any given component. The end goal of this effort is to design and deliver an efficient real-world layout of transducers on a circuit board.
The physical process may be envisioned as, for example, a repulsive force field between elements that spreads the components evenly and/or each component represented as a rigid body. These elements may be connected to their preferred locations by constraints that correspond to an interpretation of physical springs. In this way, the physical configuration in which the “springs”, positive constraints, are most relaxed while continuing to obey other physical negative constraints such as shape occlusion provides a comprehensive metaphor for component placement on a circuit board.
A rigid body simulation is one example of a physical simulation system that can be configured to yield metaphors for these constraints. The simulation of each component as a rigid body then prevents shape occlusion, as two rigid bodies may not occupy the same space at the same time. The simulator may also be used to apply a restraining force to each component in the design. This is intended to keep the components as close to the pre-calculated and ideal positions as possible. Deviation from the ideal positions results in an increased restoring force. This may be envisioned and modeled as a spring pulling the component towards the ideal location. At a predefined point, the spring may be configured to break, resulting in the deletion and possible relocation of the component.
Once the initial placement of the components, which may be automated, is complete the simulation is moved forward in time. As the simulation moves forwards in time, springs will constrict, which may pull components into better positions, while other constraints may push components away from their various preferred locations. Over time, the actions of these opposing forces will allow the board design to settle into a more optimized configuration that can be exported from the process. This may be repeated multiple times, or as a single step in a longer design process in order to obtain an effective circuit board design.
In the right layout 2130, the left-most transducer 2115 is removed because the expanded rectangular component-free zone 2132 forces the distance from the leftmost initial transducer placement 2116 to the simulated final transducer placement 2115 to be too large. This causes the spring to break 2131 and thus the simulated final transducer placement 2115 is removed from the layout.
As shown in the right layout 2240, the former middle-left transducer 2212 is moved to position 2222 on the other side of the array and a spring 2231 is used to model its placement with respect to the component free zone 2235. The rest of the elements in the right layout 2240 are the same as the center layout 2220.
Abruptly applying constraints to the component layout may create uncertainty due to physical constraints being immediately and sharply violated. To alleviate this, constraints and components may be made to move and expand slowly into position. This physical interpretation allows components to slowly adopt a position as a natural result of the simulation.
The simulation may also be interactive by allowing for the dynamic creation and deletion of components. These components are also physically simulated and this interactivity enables an element of human control over the placement process. In these cases, both the immediate component position and the anchoring location of its associated spring constraint may be modified.
An example of a layout improved using the foregoing methods is shown in
1. Levitation as an Input Calibration Step
The levitation of objects such as polystyrene beads, is possible using the acoustic field produced above the array. It has been shown that an optimization may be employed to generate a levitation point at which a small object can be trapped. (Marzo et al., Holographic acoustic elements for manipulation of levitated objects, Nature Communications 6:8661 (2015)). A levitation point may be created in a manner similar to a control point and at a predictable sub-wavelength offset from an equivalent control point that can be created at the same position. The object can then be sensed using the same sensor as the input, which could for example be a camera. This object may be then used as a fiducial marker floating in free space. This fiducial object may provide a correspondence between the input coordinate space of a sensing device capable of spatial tracking and the acoustic output coordinate space.
Once a bead is trapped at a levitation point, the levitation point may be slowly moved in a similar fashion to a control point. As the levitation point moves around the acoustic volume above the array, the correspondence between the acoustic output space and sensor input space is refined. In order to provide sufficient sensor calibration, the object must move through all three dimensions, registering correspondences between input and output space through time. An example path for the object might be between the vertices of a tetrahedron floating above the array. The system may be operated without human intervention if an acoustically transparent structure holds the fiducial in a predefined initial position so that the array can grip and levitate the object.
2. Levitation as an Output Calibration Step
If the correspondence between the input sensor space and the objective measurement of position is known, then the levitating object may be used to calibrate the output space of the array. As the formation of a levitation point is similar to the formation of a control point, the former may be used to measure the way in which the acoustic assumptions hold across the acoustic volume. This may be used to ascertain whether the positions to which the array focuses are correct and enable the computation of a correction factor in the event that the focus has been linearly or non-linearly transformed in space. Both the input and the output calibration steps may be performed at the same time.
The various features of the foregoing embodiments may be selected and combined to produce numerous variations of improved haptic systems.
In the foregoing specification, specific embodiments have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of present teachings.
The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
Moreover, in this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” “has”, “having,” “includes”, “including,” “contains”, “containing” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “comprises . . . a”, “has . . . a”, “includes . . . a”, “contains . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises, has, includes, contains the element. The terms “a” and “an” are defined as one or more unless explicitly stated otherwise herein. The terms “substantially”, “essentially”, “approximately”, “about” or any other version thereof, are defined as being close to as understood by one of ordinary skill in the art. The term “coupled” as used herein is defined as connected, although not necessarily directly and not necessarily mechanically. A device or structure that is “configured” in a certain way is configured in at least that way, but may also be configured in ways that are not listed.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.
This application claims the benefit of the following three U.S. Provisional Patent Applications, all of which are incorporated by reference in their entirety: 1. Ser. No. 62/193,180, filed on Jul. 16, 2015. 2. Ser. No. 62/275,206, filed on Jan. 5, 2016. 3. Ser. No. 62/275,030, filed on Jan. 5, 2016.
Number | Name | Date | Kind |
---|---|---|---|
4218921 | Oran et al. | Aug 1980 | A |
4771205 | Mequio | Sep 1988 | A |
4881212 | Takeuchi | Nov 1989 | A |
5329682 | Thurn | Jul 1994 | A |
5426388 | Flora et al. | Jun 1995 | A |
5511296 | Dias | Apr 1996 | A |
6503204 | Sumanaweera et al. | Jan 2003 | B1 |
6647359 | Verplank | Nov 2003 | B1 |
6772490 | Toda | Aug 2004 | B2 |
6800987 | Toda | Oct 2004 | B2 |
7109789 | Spencer | Sep 2006 | B2 |
7225404 | Zilles et al. | May 2007 | B1 |
8269168 | Axelrod | Sep 2012 | B1 |
8279193 | Birnbaum | Oct 2012 | B1 |
8607922 | Werner | Dec 2013 | B1 |
8833510 | Koh | Sep 2014 | B2 |
8884927 | Cheatham, III | Nov 2014 | B1 |
9208664 | Peters et al. | Dec 2015 | B1 |
9612658 | Subramanian | Apr 2017 | B2 |
9662680 | Yamamoto | May 2017 | B2 |
9841819 | Carter et al. | Dec 2017 | B2 |
9945818 | Ganti | Apr 2018 | B2 |
10101811 | Carter | Oct 2018 | B2 |
20020149570 | Knowles | Oct 2002 | A1 |
20030024317 | Miller | Feb 2003 | A1 |
20030144032 | Brunner et al. | Jul 2003 | A1 |
20040014434 | Haardt | Jan 2004 | A1 |
20040091119 | Duraiswami et al. | May 2004 | A1 |
20040226378 | Oda | Nov 2004 | A1 |
20040264707 | Yang | Dec 2004 | A1 |
20050212760 | Marvit | Sep 2005 | A1 |
20060085049 | Cory et al. | Apr 2006 | A1 |
20060090955 | Cardas | May 2006 | A1 |
20060091301 | Trisnadi | May 2006 | A1 |
20070177681 | Choi et al. | Aug 2007 | A1 |
20080012647 | Risbo et al. | Jan 2008 | A1 |
20080273723 | Hartung et al. | Nov 2008 | A1 |
20080300055 | Lutnick | Dec 2008 | A1 |
20090093724 | Pernot et al. | Apr 2009 | A1 |
20100013613 | Weston | Jan 2010 | A1 |
20100085168 | Kyung | Apr 2010 | A1 |
20100103246 | Schwerdtner | Apr 2010 | A1 |
20100109481 | Buccafusca | May 2010 | A1 |
20100262008 | Roundhill | Oct 2010 | A1 |
20110051554 | Varray et al. | Mar 2011 | A1 |
20110199342 | Vartanian et al. | Aug 2011 | A1 |
20110310028 | Camp, Jr. | Dec 2011 | A1 |
20120063628 | Rizzello | Mar 2012 | A1 |
20120223880 | Birnbaum | Sep 2012 | A1 |
20120229400 | Birnbaum | Sep 2012 | A1 |
20120229401 | Birnbaum et al. | Sep 2012 | A1 |
20120236689 | Brown | Sep 2012 | A1 |
20120299853 | Dagar | Nov 2012 | A1 |
20120307649 | Park et al. | Dec 2012 | A1 |
20120315605 | Cho | Dec 2012 | A1 |
20130035582 | Radulescu | Feb 2013 | A1 |
20130094678 | Scholte | Apr 2013 | A1 |
20130100008 | Marti | Apr 2013 | A1 |
20130101141 | McElveen | Apr 2013 | A1 |
20140027201 | Islam | Jan 2014 | A1 |
20140168091 | Jones | Jun 2014 | A1 |
20140265572 | Siedenburg | Sep 2014 | A1 |
20150002477 | Cheatham, III | Jan 2015 | A1 |
20150007025 | Sassi | Jan 2015 | A1 |
20150006645 | Lin et al. | Mar 2015 | A1 |
20150066445 | Lin | Mar 2015 | A1 |
20150070147 | Cruz-Hernandez et al. | Mar 2015 | A1 |
20150070245 | Han et al. | Mar 2015 | A1 |
20150110310 | Minnaar | Apr 2015 | A1 |
20150013023 | Harris et al. | May 2015 | A1 |
20150130323 | Harris | May 2015 | A1 |
20150168205 | Lee | Jun 2015 | A1 |
20150192995 | Subramanian et al. | Jul 2015 | A1 |
20150220199 | Wang | Aug 2015 | A1 |
20150226537 | Schorre | Aug 2015 | A1 |
20150226831 | Nakamura et al. | Aug 2015 | A1 |
20150248787 | Abovitz | Sep 2015 | A1 |
20150277610 | Kim | Oct 2015 | A1 |
20150304789 | Babyoff | Oct 2015 | A1 |
20160019879 | Daley | Jan 2016 | A1 |
20160044417 | Clemen | Feb 2016 | A1 |
20160124080 | Carter | May 2016 | A1 |
20160189702 | Blanc et al. | Jun 2016 | A1 |
20160242724 | Lavallee | Aug 2016 | A1 |
20160246374 | Carter | Aug 2016 | A1 |
20160249150 | Carter et al. | Aug 2016 | A1 |
20160320843 | Long | Nov 2016 | A1 |
20170004819 | Ochiai | Jan 2017 | A1 |
20170018171 | Carter | Jan 2017 | A1 |
20170123499 | Eid | May 2017 | A1 |
20170193768 | Long | Jul 2017 | A1 |
20170193823 | Jiang | Jul 2017 | A1 |
20180039333 | Carter | Feb 2018 | A1 |
20180304310 | Long | Oct 2018 | A1 |
20180361174 | Radulescu | Dec 2018 | A1 |
Number | Date | Country |
---|---|---|
102591512 | Jul 2012 | CN |
103797379 | May 2014 | CN |
103984414 | Aug 2014 | CN |
1911530 | Apr 2008 | EP |
2464117 | Apr 2010 | GB |
2513884 | Nov 2014 | GB |
2513884 | Nov 2014 | GB |
2530036 | Mar 2016 | GB |
2010109579 | May 2010 | JP |
2011172074 | Sep 2011 | JP |
201248378 | Mar 2012 | JP |
2016035646 | Mar 2016 | JP |
20120065779 | Jun 2012 | KR |
20130055972 | May 2013 | KR |
20160008280 | Jan 2016 | KR |
199118486 | Nov 1991 | WO |
9639754 | Dec 1996 | WO |
2005017965 | Feb 2005 | WO |
2013179179 | Dec 2013 | WO |
2014181084 | Nov 2014 | WO |
2015006467 | Jan 2015 | WO |
2015039622 | Mar 2015 | WO |
2016007920 | Jan 2016 | WO |
2016132141 | Aug 2016 | WO |
2016132144 | Aug 2016 | WO |
2016132144 | Aug 2016 | WO |
2016137675 | Sep 2016 | WO |
2016162058 | Oct 2016 | WO |
Entry |
---|
Search report and Written Opinion of ISA for PCT/GB2015/050417 dated Jul. 8, 2016. |
Search report and Written Opinion of ISA for PCT/GB2015/050421 dated Jul. 8, 2016. |
Martinez-Graullera et al., 2D array design based on Fermat spiral for ultrasonic imaging, Ultrasonics 50 (2010) 280-89. |
Search Report for PCT/GB2017/050012 dated Jun. 8, 2017. |
Pompei, F.J. (2002), “Sound from Ultrasound: The Parametric Array as an Audible Sound Source”, Massachusetts Institute of Technology. |
Hasegawa, K. and Shinoda, H. (2013) “Aerial Display of Vibrotactile Sensation with High Spatial-Temporal Resolution using Large Aperture Airbourne Ultrasound Phased Array”, University of Tokyo. |
Hoshi, T. et al. (2010), “Noncontrast Tactile Display Based on Radiation Pressure of Airbourne Ultrasound”, IEEE Transactions on Haptics, vol. 3, No. 3. |
Yoshino, K. and Shinoda, H. (2013), “Visio Acoustic Screen for Contactless Touch Interface with Tactile Sensation”, University of Tokyo. |
Kamakura, T. and Aoki, K. (2006) “A Highly Directional Audio System using a Parametric Array in Air” WESPAC IX 2006. |
Alexander, J. et al. (2011), “Adding Haptic Feedback to Mobile TV”. |
Carter, T. et al. (2013) “Ultrahaptics: Multi-point Mid-Air Haptic Feedback for Touch Surfaces” UIST. |
Gavrilov, L.R. (2008) “The Possibility of Generating Focal Regions of Complex Configurations in Application to the Problems of Stimulation of Human Receptor Structures by Focused Ultrasound” Acoustical Physics, vol. 54, No. 2, pp. 269-278. |
Search Report for GB1308274.8 dated Nov. 11, 2013. |
Search Report for PCT/GB2014/051319 dated Jul. 28, 2014. |
Gavrilov L R Et al (2000) “A theoretical assessment of the relative performance of spherical phased arrays for ultrasound surgery” Ultrasonics, Ferroelectrics, and Frequency Control, IEEE Transactions on (vol. 47, Issue: 1), pp. 125-139. |
Mingzhu Lu et al. (2006) Design and experiment of 256-element ultrasound phased array for noninvasive focused Ultrasound surgery, Ultrasonics, vol. 44, Supplement, Dec. 22, 2006, pp. e325-e330. |
E.S. Ebbini et al. (1991), A spherical-section ultrasound phased array applicator for deep localized hyperthermia, Biomedical Engineering, IEEE Transactions on (vol. 38 Issue: 7), pp. 634-643. |
Iwamoto et al. (2006), Two-dimensional Scanning Tactile Display using Ultrasonic Radiation Pressure, Symposium of Haptic Interfaces for Virtual Environment and Teleoperator Systems, pp. 57-61. |
Iwamoto et al. (2008), Non-contact Method for Producing Tactile Sensation Using Airborne Ultrasound, EuroHaptics, pp. 504-513. |
Search report for PCT/GB2015/052578 dated Oct. 26, 2015. |
IPRP for PCT/GB2014/051319 dated Nov. 10, 2015. |
Search Report for GB1415923.0 dated Mar. 11, 2015. |
Marshall, M ., Carter, T., Alexander, J., & Subramanian, S. (2012). Ultratangibles: creating movable tangible objects on interactive tables. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems. (pp. 2185-2188). |
Obrist et al., Talking about Tactile Experiences, CHI 2013, Apr. 27-May 2, 2013. |
Freeman et al., Tactile Feedback for Above-Device Gesture Interfaces: Adding Touch to Touchless Interactions ICMI'14, Nov. 12-16, 2014, Istanbul, Turkey. |
Obrist et al., Emotions Mediated Through Mid-Air Haptics, CHI 2015, Apr. 18-23, 2015, Seoul, Republic of Korea. |
Wilson et al., Perception of Ultrasonic Haptic Feedback on the Hand: Localisation and Apparent Motion, CHI 2014, Apr. 26-May 1, 2014, Toronto, Ontario, Canada. |
Phys.org, Touchable Hologram Becomes Reality, Aug. 6, 2009, by Lisa Zyga. |
Iwamoto et al., Airborne Ultrasound Tactile Display: Supplement, The University of Tokyo 2008. |
Hoshi, T., Development of Aerial-Input and Aerial-Tactile-Feedback System, IEEE World Haptics Conference 2011, p. 569-573. |
EPSRC Grant summary Ep/J004448/1 (2011). |
Hoshi, T., Handwriting Transmission System Using Noncontact Tactile Display, IEEE Haptics Symposium 2012 pp. 399-401. |
Takahashi, M. et al., Large Aperture Airborne Ultrasound Tactile Display Using Distributed Array Units, SICE Annual Conference 2010 p. 359-62. |
Hoshi, T., Non-contact Tactile Sensation Synthesized by Ultrasound Transducers, Third Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 2009. |
Light, E.D., Progress in Two Dimensional Arrays for Real Time Volumetric Imaging, 1998. |
Casper et al., Realtime Control of Multiple-focus Phased Array Heating Patterns Based on Noninvasive Ultrasound Thermography, IEEE Trans Biomed Eng. Jan. 2012; 59(1): 95-105. |
Hoshi, T., Touchable Holography, SIGGRAPH 2009, New Orleans, Louisiana, Aug. 3-7, 2009. |
Marshall et al., Ultra-Tangibles: Creating Movable Tangible Objects on Interactive Tables, CHI'12, May 5-10, 2012, Austin, Texas. |
Marzo et al., Holographic acoustic elements for manipulation of levitated objects, Nature Communications DOI: 10.1038/ncomms9661 (2015). |
Search Report for PCT/GB2017/052332 dated Oct. 10, 2017. |
A. Sand, Head-Mounted Display with Mid-Air Tactile Feedback, Proceedings of the 21st ACM Symposium on Virtual Reality Software and Technology, Nov. 13-15, 2015. |
M. Toda, New Type of Matching Layer for Air-Coupled Ultrasonic Transducers, IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, vol. 49, No. 7, Jul. 2002. |
E. Bok, Metasurface for Water-to-Air Sound Transmission, Physical Review Letters 120, 044302 (2018). |
Long et al. Rendering Volumetric Haptic Shapes in Mid-Air using Ultrasound, ACM Transactions on Graphics (Proceedings of SIGGRAPH Asia), vol. 33, No. 6, Article 181 (Nov. 2014). |
Xin Cheng et al., Computation of the Aoustic Radiation Force on a Sphere Based on the 3-D FDTD Method, IEEE Symposium 2010. |
K. Jia, Dynamic properties of micro-particles in ultrasonic transportation using phase-controlled standing waves, J. Applied Physics 116, n. 16 (2014). |
Yang Ling, Phase-coded approach for controllable generation of acoustical vortices, J. Applied Physics 113, No. 15 (2013). |
M. Barmatz, Acoustic radiation potential on a sphere in plane, cylindrical, and spherical standing wave of fields, J. Acoustical Socirty, 77 No. 3 (1985). |
M. A.B. Andrade, Matrix method for acoustic levitation simulation, IEEE Transactions on Ultrasonics Ferroelectrics and Frequency Control, 58 n. 8 (2011). |
Search Report for PCT/GB/2017/053729 dated Mar. 15, 2018. |
Search Report for PCT/GB/2017/053880 dated Mar. 21, 2018. |
“Welcome to Project Soli” video, https://atap.google.com/#project-soli Accessed Nov. 30, 2018, 2 pages. |
Colgan, A., “How Does the Leap Motion Controller Work?” Leap Motion, Aug. 9, 2014, 10 pages. |
Damn Geeky, “Virtual projection keyboard technology with haptic feedback on palm of your hand,” May 30, 2013, 4 pages. |
Definition of “Interferometry”according to Wikipedia, 25 pages., Retrieved Nov. 2018. |
Definition of “Multilateration” according to Wikipedia, 7 pages., Retrieved Nov. 2018. |
Definition of “Trilateration” according to Wikipedia, 2 pages., Retrieved Nov. 2018. |
EPO Office Action for EP16708440.9 dated Sep. 12, 2018 (7 pages). |
Ex Parte Quayle Action dated Dec. 28, 2018 for U.S. Appl. No. 15/966,213 (pp. 1-7). |
Gokturk, et al., “A Time-of-Flight Depth Sensor-System Description, Issues and Solutions,” Published in: 2004 Conference on Computer Vision and Pattern Recognition Workshop, Date of Conference: Jun. 27-Jul. 2, 2004, 9 pages. |
Iddan, et al., “3D Imaging in the Studio (and Elsewhwere . . . ” Apr. 2001, 3DV systems Ltd., Yokneam, Isreal, www.3dvsystems.com.il, 9 pages. |
International Preliminary Report on Patentability and Written Opinion issued in corresponding PCT/US2017/035009, dated Dec. 4, 2018, 8 pages. |
International Preliminary Report on Patentability for Application No. PCT/EP2017/069569 dated Feb. 5, 2019, 11 pages. |
Japanese Office Action (with English language translation) for Application No. 2017-514569, dated Mar. 31, 3019, 10 pages. |
Kolb, et al., “Time-of-Flight Cameras in Computer Graphics,” Computer Graphics forum, vol. 29 (2010), No. 1, pp. 141-159. |
Krim, et al., “Two Decades of Array Signal Processing Research—The Parametric Approach”, IEEE Signal Processing Magazine, Jul. 1996, pp. 67-94. |
Lang, Robert, “3D Time-of-Flight Distance Measurement with Custom Solid-State Image Sensors in CMOS/CCD—Technology”, A dissertation submitted to Department of EE and CS at Univ. of Siegen, dated Jun. 28, 2000, 223 pages. |
Li, Larry, “Time-of-Flight Camera—An Introduction,” Texas Instruments, Technical White Paper, SLOA190B—Jan. 2014 Revised May 2014, 10 pages. |
Meijster, A., et al., “A General Algorithm for Computing Distance Transforms in Linear Time,” Mathematical Morphology and its Applications to Image and Signal Processing, 2002, pp. 331-340. |
Notice of Allowance dated Dec. 19, 2018 for U.S. Appl. No. 15/665,629 (pp. 1-9). |
Notice of Allowance dated Dec. 21, 2018 for U.S. Appl. No. 15/983,864 (pp. 1-7). |
Notice of Allowance dated Feb. 7, 2019 for U.S. Appl. No. 15/851,214 (pp. 1-7). |
Notice of Allowance dated May 30, 2019 for U.S. Appl. No. 15/966,213 (pp. 1-9). |
Office Action dated Apr. 18, 2019 for U.S. Appl. No. 16/296,127 (pp. 1-6). |
Office Action dated Apr. 4, 2019 for U.S. Appl. No. 15/897,804 (pp. 1-10). |
Office Action dated Feb. 20, 2019 for U.S. Appl. No. 15/623,516 (pp. 1-8). |
Office Action dated May 16, 2019 for U.S. Appl. No. 15/396,851 (pp. 1-7). |
PCT Partial International Search Report for Application No. PCT/GB2018/053404 dated Feb. 25, 2019, 13 pages. |
Péter Tamás Kovács et al, “Tangible Holographic 3D Objects with Virtual Touch”, Interactive Tabletops & Surfaces, ACM, 2 Penn Plaza, Suite 701 New York NY 10121-0701 USA, (Nov. 15, 2015), ISBN 978-1-4503-3899-8, pp. 319-324. |
Schmidt, Ralph, “Multiple Emitter Location and Signal Parameter Estimation” IEEE Transactions of Antenna and Propagation, vol. AP-34, No. 3, Mar. 1986, pp. 276-280. |
Search report for PCT/GB2018/051061 dated Sep. 26, 2018 (17 pages). |
Search report for PCT/US2018/028966 dated Jul. 13, 2018 (43 pages). |
Sixth Sense webpage, http://www.pranavmistry.com/projects/sixthsense/ Accessed Nov. 30, 2018, 7 pages. |
Steve Guest et al., “Audiotactile interactions in roughness perception”, Exp. Brain Res (2002) 146:161-171, DOI 10.1007/s00221-002-1164-z, Accepted: May 16, 2002/ Published online: Jul. 26, 2002, Springer-Verlag 2002, (11 pages). |
Takahashi Dean: “Ultrahaptics shows off sense of touch in virtual reality”, Dec. 10, 2016 (Dec. 10, 2016), XP055556416, Retrieved from the Internet: URL: https://venturebeat.com/2016/12/10/ultrahaptics-shows-off-sense-of-touch-in-virtual-reality/ [retrieved on Feb. 13, 2019] 4 pages. |
Teixeira, et al., “A brief introduction to Microsoft's Kinect Sensor,” Kinect, 26 pages., retrieved Nov. 2018. |
Xu Hongyi et al, “6-DoF Haptic Rendering Using Continuous Collision Detection between Points and Signed Distance Fields”, IEEE Transactions on Haptics, IEEE, USA, vol. 10, No. 2, ISSN 1939-1412, (Sep. 27, 2016), pp. 151-161, (Jun. 16, 2017). |
Zeng, Wejun, “Microsoft Kinect Sensor and Its Effect,” IEEE Multimedia, Apr.-Jun. 2012, 7 pages. |
Corrected Notice of Allowability dated Oct. 31, 2019 for U.S. Appl. No. 15/623,516 (pp. 1-2). |
Extended European Search Report for Application No. EP19169929.7, dated Aug. 6, 2019, 7 pages. |
International Search Report and Written Opinion for Application No. PCT/GB2019/050969, dated Jun. 13, 2019, 15 pages. |
Office Action dated Aug. 22, 2019 for U.S. Appl. No. 16/160,862 (pp. 1-5). |
Office Action dated Oct. 17, 2019 for U.S. Appl. No. 15/897,804 (pp. 1-10). |
Office Action dated Oct. 31, 2019 for U.S. Appl. No. 15/671,107 (pp. 1-6). |
Office Action dated Oct. 7, 2019 for U.S. Appl. No. 15/396,851 (pp. 1-9). |
Corrected Notice of Allowability dated Jun. 21, 2019 for U.S. Appl. No. 15/966,213 (2 pages). |
International Search Report and Written Opinion for Application No. PCT/GB2018/053739, dated Jun. 4, 2019, 16 pages. |
Notice of Allowance dated Jul. 31, 2019 for U.S. Appl. No. 15/851,214 (pp. 1-9). |
Notice of Allowance dated Jul. 31, 2019 for U.S. Appl. No. 16/296,127 (pp. 1-9). |
Office Action dated Jul. 26, 2019 for U.S. Appl. No. 16/159,695 (pp. 1-8). |
Number | Date | Country | |
---|---|---|---|
20170018171 A1 | Jan 2017 | US |
Number | Date | Country | |
---|---|---|---|
62193180 | Jul 2015 | US | |
62275206 | Jan 2016 | US | |
62275030 | Jan 2016 | US |