Patent Grant
9300430
1. Statement of the Technical Field
The inventive arrangements relate to teleoperation systems and methods for the electronic remote control of unmanned machines. More particularly, the invention concerns teleoperation systems and methods for smoothing unpredictable latencies (or time delays) experienced in communications links (or data links) between controllers and remote unmanned machines.
2. Description of the Related Art
Teleoperation systems typically comprise hand controller systems for remotely controlling movement of an unmanned slave device via a wired or wireless communications link (or data link). In some scenarios, the remote control of the slave device is an Internet based control. In all cases, the slave device can include, but is not limited to, a robot, a robotic manipulator, an Unmanned Aerial Vehicle (“UAV”), an Unmanned Ground Vehicle (“UGV”), and/or an Unmanned Underwater Vehicle (“UUV”). During operation, a hand controller system (sometimes in the form of a joy stick or a pistol grip) senses the movement by a user's hand. Thereafter, the hand controller system generates one or more control messages. The control messages are sent over a wired or wireless data link to the slave device. At the slave device, the control messages are used for actuating various motion operations of the slave device. The control system of the slave device will respond to these control messages by activating certain servo-motors, solenoids, or other devices within the slave device to produce the desired action. Video cameras can be used to provide visual information to a remote operator regarding a work piece and/or the environment surrounding the slave device. Haptic technology can also be employed to provide tactile feedback by the teleoperation system to a user. The tactile feedback is typically provided by applying forces, vibrations and/or motions to one or more portions of the hand controller.
Notably, unpredictable inherent latency can be experienced in the data links between the hand control system and the slave device. For example, the slave device is a UGV with a radio. The UGV drives over a hill, whereby the remotely located human operator controlling the UGV loses Line Of Sight (“LOS”) thereof. Also, the quality of communications link degrades. In this case, the inherent latency continuously varies unpredictably within, for example, a range of ten milliseconds to hundreds of milliseconds. The operator controlling the UGV is aware of the inherent latency between the time a control message is sent from the hand controller system and the time the control message is received by slave device. Consequently, the operator tries to mentally compensate for the inherent latency. In this regard, the operator tries to compensate for a relatively low inherent latency when the inherent latency is actually a relatively high inherent latency, or vice versa. In effect, the operator commands a movement of the slave device that is incorrect because the slave device is actually in a different location than that thought by the operator. As a result, the slave device may undershoot or overshoot a point or a target, thereby making the teleoperation system unpredictable. In the UAV scenarios, the unpredictable natural latency leads to unstable control of the UAV by the operator. The unstable control may cause the UAV to crash. One can appreciate that such unpredictable and unstable control of slave devices is undesirable, especially in a military and rescue contexts.
Many conventional systems do not implement any solutions to the variable inherent latency problem. In these systems, teleoperation commands are executed when they are received by the slave device. Also, video feedback is shown to the operator as it is received. Other conventional systems employ stabilization algorithms to address the variable inherent latency problem. In these systems, the stabilization algorithms prevent the systems from becoming dangerously unstable. For example, a system comprises a remotely controlled UAV. If the operator is providing incorrect commands as a result of the variable inherent latency, then a control system on the UAV rejects or modifies the commands received thereat so to prevent crashing of the UAV. Despite the advantages of the stabilization algorithms, they do not address the issue of predictability of the teleoperation systems.
The present invention concerns systems and methods for increasing a predictability of telematic operations of a teleoperation system. The methods comprise measuring an inherent latency of a communications link of the teleoperation system which varies unpredictably over at least a first window of time. The inherent latency is then analyzed to determine a first reference value useful for increasing the predictability of the telematic operations. The first reference value can include, but is not limited to, a peak value of a plurality of measured values for the inherent latency, one of the plurality of measured values which falls within a pre-defined threshold range, and/or an average of the plurality of measured values for the inherent latency.
Subsequently, the first reference value is used to select an amount of controlled latency to be added to the communications link at each of a plurality of time points. The amount of controlled latency to be added to the communications link at each time point can be: selected based on a difference between the first reference value and a measured value for the inherent latency at that time point; and/or selected to produce a constant overall latency for the communications link.
Next, the selected amount of controlled latency is added to the communications link at each of the plurality of time points so as to increase the predictability of the telematic operations. Notably, the amount of controlled latency added at a first time point is different than the amount of controlled latency added at a second time point. The controlled latency can be added to the communications link by time delaying a received message by an amount determined based on the first reference value and a current level of inherent latency of the communications link.
In some scenarios, the communications link comprises a forward communications link and the controlled latency is added to the communications link by a slave device of the teleoperation system. Alternatively or additionally, the communications link is a reverse communications link and the controlled latency is added to the communications link by a control unit of the teleoperation system operative to control a remotely located slave device.
Embodiments will be described with reference to the following drawing figures, in which like numerals represent like items throughout the figures, and in which:
It will be readily understood that the components of the embodiments as generally described herein and illustrated in the appended figures could be arranged and designed in a wide variety of different configurations. Thus, the following more detailed description of various embodiments, as represented in the figures, is not intended to limit the scope of the present disclosure, but is merely representative of various embodiments. While the various aspects of the embodiments are presented in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
The present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. The described embodiments are to be considered in all respects as illustrative. The scope of the invention is, therefore, indicated by the appended claims. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Reference throughout this specification to features, advantages, or similar language does not imply that all of the features and advantages that may be realized with the present invention should be or are in any single embodiment of the invention. Rather, language referring to the features and advantages is understood to mean that a specific feature, advantage, or characteristic described in connection with an embodiment is included in at least one embodiment of the present invention. Thus, discussions of the features and advantages, and similar language, throughout the specification may, but do not necessarily, refer to the same embodiment.
Furthermore, the described features, advantages and characteristics of the invention may be combined in any suitable manner in one or more embodiments. One skilled in the relevant art will recognize, in light of the description herein, that the invention can be practiced without one or more of the specific features or advantages of a particular embodiment. In other instances, additional features and advantages may be recognized in certain embodiments that may not be present in all embodiments of the invention.
Reference throughout this specification to “one embodiment”, “an embodiment”, or similar language means that a particular feature, structure, or characteristic described in connection with the indicated embodiment is included in at least one embodiment of the present invention. Thus, the phrases “in one embodiment”, “in an embodiment”, and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment.
As used in this document, the singular form “a”, “an”, and “the” include plural references unless the context clearly dictates otherwise. Unless defined otherwise, all technical and scientific terms used herein have the same meanings as commonly understood by one of ordinary skill in the art. As used in this document, the term “comprising” means “including, but not limited to”.
Embodiments will now be described with respect to
In some scenarios, the methods generally involve: continuously measuring an inherent latency of a communications link of the teleoperation system which varies unpredictably over a given period or window of time; analyzing the plurality of measured values for the inherent latency of the communications link to identify a first value thereof which constitutes a peak value, to identify at least one second value thereof which falls within a given threshold range, and/or to determine a third value which constitutes an average of the plurality of measured values for the inherent latency; using the first, second and/or third value(s) to select or determine an amount of controlled latency to be added to the communications link at each of a plurality of time points; and adding the amount of controlled latency to the communications link at each time point so as to provide a constant or substantially constant latency for the communications link in a forward and/or reverse direction. The amount of controlled latency added at each time point to the communications link is generally selected or determined based on the difference between the first, second and/or third value(s) and the measured value for the inherent latency at that time point. The above described process may be repeated for each of a plurality of periods or windows of time.
Notably, it is generally understood in the art that latency is undesirable in communication systems. As such, the present method of adding controlled latency to the communications link is a somewhat counter-intuitive way of addressing the variable inherent latency problem of teleoperation systems. However, by adding the additional controlled latency to the communications link, the teleoperation system has more predicable overall latency as compared to conventional teleoperation systems, such as those described in the background section of this document. Consequently, the teleoperation system of the present invention has superior overall telematic performance as compared to that of the conventional teleoperation systems.
Referring now to
UGVs, UAVs and/or UUVs are often used by first responders or military personnel for search and rescue operations in hazardous and dangerous environments. For example, a UGV may be employed to find and disable an explosive device. In this regard, the slave device 102 may have a manipulator 104 with a camera 106 coupled to a distal end 112 thereof. In this example, the manipulator 104 comprises a robot arm extending from a body 130. The robot arm has a gripper 124 disposed at a distal end 112 thereof for contact, grasping or otherwise manipulating external objects 122 (e.g., an explosive device at least partially hidden under dirt or cloth). The body 130 includes hardware and/or software for controlling the operations of the slave device 102 such that it performs moving actions, manipulating actions, and/or grasping actions based on telematic commands received thereat. The telematic commands are generated by a control unit 110 in response to operator inputs thereto. The telematics commands are communicated from the control unit 110 to the slave device 102 via a wireless communications link 120 as shown in
The control unit 110 may be a mobile unit or a fixed unit capable of communicating with the slave device 102 via the wired and/or wireless communications link. In either scenario, the control unit 110 comprises a video display unit 114 for displaying a video image obtained from the camera 106. A Hand Control Unit (“HCU”) 116 senses movements of an operator and communicates input signals specifying the sensed movements to the processor 118. The processor 118 processes each input signal to determine if a sensed movement corresponds to an element of a predetermined movement set of the teleoperation system 100. If the sensed movement corresponds to an element of the predetermined movement set, then the processor 118 generates a control message including a command for controlling the slave device 102 in a specified manner (e.g., move the manipulator up, down, forward, backward, to the right, and/or to the left). Thereafter, the control message is communicated from the control unit 110 to the slave device 102 via the wired and/or wireless communications link.
In response to the reception of the control message, the slave device 102 acts in the manner specified therein (e.g., moves the manipulator up, down, forward, backward, to the right, and/or to the left). Consequently, the manipulator 104 may physically engage an object 122. Force sensors (not shown) of the manipulator 104 detect forces exerted thereon as a result of the physical engagement. The information relating to the detected forces is communicated back to the control unit 110 via the wired and/or wireless communications link.
At the control unit 110, the information is then used to generate suitable tactile or haptic feedback for the user. The tactile or haptic feedback allows the physical engagement to be felt by the operator's hand, through the HCU 116. Accordingly, the tactile or haptic feedback may include linear forces and/or rotational forces applied to the HCU 116. Advantageously, the linear forces and/or rotational forces are directional forces such that the operator's sensation is more realistic than in conventional tactile or haptic feedback systems.
Notably, unpredictable inherent latency can be experienced in the wired and/or wireless communications link(s) between the HCU 116 and the slave device 102. For example, the slave device 102 drives over a hill, whereby the remotely located operator controlling the salve device 102 loses LOS thereof. Also, the quality of the wired and/or wireless communications link(s) degrade(s). In this case, the inherent latency of a communications link varies unpredictably within a range of, for example, ten milliseconds to hundreds of milliseconds. As such, the teleoperation system 100 employs a novel method for solving the unpredictable inherent latency issue. An example of such a novel method will be described below in relation to
The implemented method generally involves: continuously or periodically measuring an inherent latency of a communications link (e.g., a forward communications link and/or a reverse communications link) which varies unpredictably over a given period of time or window of time; analyzing a plurality of measured values for the inherent latency of the communications link to identify a first value thereof which constitutes a peak value, to identify at least one second value thereof which falls within a given threshold range, and/or to determine a third value which constitutes an average of the plurality of measured values for the inherent latency; using the first, second and/or third value(s) to select or determine an amount of controlled latency to be added to the communications link at each of a plurality of time points; and adding the amount of controlled latency to the communications link at each time point so as to provide an overall constant or substantially constant latency for the communications link. The amount of controlled latency added to the communications link at each time point is generally selected or determined based on the difference between the first, second and/or third value(s) and the measured value for the inherent latency thereof at that time point. The above described process may be repeated for each of a plurality of periods of time or windows of time. Therefore, the first, second and/or third value(s) can be variable values, i.e., different for at least two periods or windows of time. Also, the plurality of periods or windows of time can include overlapping and/or non-overlapping periods or windows of time.
As noted above, the amount of controlled latency added to the communications link is generally selected or determined based on the difference between the first, second and/or third value(s) and the measured value for the inherent latency at that time point. For example, a peak latency of 1.0 second was detected by the slave device 102 for a plurality of ping signals transmitted over a communications link at a regular time interval during a 10.0 second window of time. As such, the slave device 102 performs operations to provide a constant 1.0 second overall latency for command or control messages received thereat (e.g., telematics command messages). In this regard, an additional 0.5 seconds of controlled latency is added by the slave device 102 to the communications links at a first time point when it has an inherent latency of 0.5 seconds. In effect, a first command or control message is delayed by an amount selected or determined based on the current level of inherent latency of the communications link (e.g., delay=0.5 seconds). In contrast, no additional controlled latency is added by the slave device 102 to the communications link at a second time point when it has an inherent latency of 1.0 second or approximately 1.0 second (e.g., an inherent latency between 0.8 seconds and 1.0 seconds). Therefore, a second command or control message is delayed by an amount selected or determined based on the current level of inherent latency of the communications link (e.g., delay=0.0 seconds). Embodiments of the present invention are not limited to the particularities of this example.
Notably, it is generally understood in the art that latency is undesirable in communication systems. As such, the present method of adding controlled latency is a somewhat counter-intuitive way of addressing the variable inherent latency problem of teleoperation systems. However, by adding the additional controlled latency to the communications link, the teleoperation system has more predicable overall latency as compared to conventional teleoperation systems, such as those described in the background section of this document. Consequently, the teleoperation system of the present invention has superior overall telematic performance as compared to such conventional teleoperation systems.
The present invention will now be described in more detail in relation to the scenario where the novel method is implemented in both the HCU 116 and the slave device 102. The present invention is not limited in this regard, as recited above. For example, the novel method can be implemented in only of the HCU 116 or only the slave device 102.
Referring now to
The controller 204 is configured to receive ping signals and/or control messages from the control unit 110. The ping signals are provided to facilitate the continuous and/or periodic measurement of an inherent latency of a communications link in at least a forward direction (i.e., a direction from the control unit 110 to the salve device 102). The control messages can also facilitate the measurement of the inherent latency. In this regard, the ping signals and/or the control messages include timestamps indicating the time that they were sent from the control unit 110. The timestamps allow the slave device 102 to determine measured values for the inherent latency of the communications link which varies unpredictably over a given period or window of time. Once a pre-defined number of measured values for the inherent latency of the communications link have been obtained, the controller 204 analyzes the measured values to: (1) identify the peak value(s) thereof; (2) identify value(s) thereof which fall(s) within a given threshold range; and/or (3) to determine an average thereof. Next, the controller 204 performs operations to provide a constant overall latency or a substantially constant overall latency (e.g., a latency of ≦1 second) for the communications link in a forward direction. These operations involve adding an amount of controlled latency to the communications link at each time point, where the amount of controlled latency is selected or determined based on the difference between the first, second and/or third value(s) and the measured value for the inherent latency at that time point. Stated differently, a received control message is delayed by an amount selected or determined based on the current latency level of the communications link.
In some scenarios, the controller 204 generates a response message for each received ping signal and/or control message. The response message is then sent in a reverse direction over the wired and/or wireless communications link (i.e., in a direction from the slave device 102 to the control unit 110). The response message can include one or more timestamps. For example, the response message includes a first timestamp indicating the time that the respective ping signal or control message was received by the slave device 102 and/or a second timestamp indicating the time that the response message was sent from the slave device 102. The first and/or second timestamps facilitate various operations by the control unit 110 for solving the unpredictable inherent latency issue with regard to the communications link in the reverse direction. Such operations will be described in detail below in relation to
The time delayed control messages are then further processed by the controller 204 to obtain motion control commands therefrom. The controller 204 then causes the slave device 102 to perform certain actions in accordance with the motion control commands. For example, the slave device 102 may be caused to move the body 120 in a forward direction, a backward direction, a right direction and/or a left direction. Additionally or alternatively, the slave device 102 may be caused to move the manipulator 104 in a linear motion and/or a rotational motion. The manipulator 104 can be a robot arm including one or more mechanical joints. Robot arms of this type are well known in the art, and therefore will not be described in detail herein. However, a brief description of one example of a manipulator 104 is provided to assist the reader in understanding the present invention.
Referring now to
At least one sensor 3081, 3082, . . . , 308m may be provided for detecting forces exerted upon the manipulator 104. In this regard, the sensors 3081, 3082, . . . , 308m are configured to detect forces (i.e., linear and/or rotational forces) exerted upon the manipulator 104 and the gripper 124. In
The data from the sensors 3061, 3062, . . . , 306n, 3081, 3082, . . . , 308m is communicated to the controller 204, which performs any necessary data formatting before communicating the data to transmit/receive circuitry 206. The information is then transmitted from the antenna 208 to the control unit 110 in one or more haptic feedback messages. The haptic controls messages can each include one or more timestamps, as described above.
Referring now to
The control unit 110 is configured to generate ping signals. Each ping signal includes at least one timestamp. The timestamp can indicate the time that the ping signal was sent from the control unit 110 to the slave device 102. The ping signals are then communicated from the control unit 110 to the slave device 102 via a wired or wireless communications link. As noted above, the timestamps of the ping signals can be used by the slave device 102 to provide a constant or substantially constant overall latency for the communications link in at least the forward direction. As also noted above, the slave device 102 may generate and send response messages to the control unit 110. In this regard, the control unit 100 is configured to receive the response messages and process the same. Time stamps of the response messages can be used by the control unit 110 to provide a constant or substantially constant overall latency for the communications link in at least the reverse direction, as described below.
As shown in
As described above, the slave device 102 processes the control messages to improve the overall latency of the communications link in at least the forward direction prior to acting in the manner specified by the motion control commands. In response to the motion control commands, the slave device 102 can travel in a forward direction, a backward direction, a right direction, and/or a left direction. Additionally or alternatively, the manipulator 104 of the slave device 102 may be caused to move in a linear or rotational motion. In some scenarios, the manipulator 104 may physically engage an object 122. Force sensors 3081, 3082, . . . , 308m of the manipulator 104 detect forces exerted thereon as a result of the physical engagement. The information relating to the detected forces is communicated back to the control unit 110 within haptic feedback messages. The haptic feedback messages are received by the control unit 110 via the receive circuitry 436, 434, 430 thereof. The haptic feedback messages can each comprise one or more timestamps. For example, each haptic feedback message can include a first timestamp indicating that time that a particular ping signal or control message was received by the slave device 102 and/or a second timestamp indicating the time that the haptic feedback message was sent from the slave device 102.
At the control unit 110, the timestamp(s) of the response messages and/or haptic feedback messages are then used to provide a constant overall latency or a substantially constant overall latency (e.g., latency ≦1 second) for the communications link in at least a reverse direction. In this regard, the control unit 110 is configured to perform one or more of the following operations: continuously measure an unpredictable inherent latency of the communications link in at least the reverse direction over a given period of time or window of time; analyze the measured values for the latency to identify a first value thereof which constitutes a peak value, to identify at least one second value thereof which falls within a given threshold range, and/or to determine a third value which constitutes an average of the measured values for the inherent latency; use the first, second and/or third value(s) to select or determine an amount of controlled latency to be added to the communications link (in at least the reverse direction) at each of a plurality of time points; and add the amount of controlled latency to the communications link at each time point. The amount of controlled latency added to the communications link at each time point is generally selected or determined based on the difference between the first, second and/or third value(s) and the measured value for the inherent latency of the communications link at that time point. Stated differently, a received message (e.g., a haptic feedback message) is delayed by an amount selected or determined based on the current level of inherent latency of the communications link (e.g. delay=difference between the first, second and/or third value(s) and the measured value for the inherent latency of the communications link at that time point). The above described process may be repeated for each of a plurality of periods of time or windows of time.
The delayed received messages (e.g., haptic feedback messages) are then further processed by the control unit 110 to obtain the information contained therein. The information is then used to generate suitable tactile or haptic feedback for the user. The tactile or haptic feedback allows the physical engagement to be felt by the user's hand, through the handle portion of the movable component 462. Accordingly, the tactile or haptic feedback may include linear forces and/or rotational forces applied to the handle portion of the movable component 462.
In some scenarios, the tactile or haptic feedback is directional such that the user's sensation is more realistic (i.e., the force applied to the slave device is exactly or substantially duplicated such that the user will feel the same or similar force). For example, if the manipulator 104 squeezes an object, then the force is applied to the user's hand via the handle portion of the movable component 462 such that the user experiences the same or similar sensation as the manipulator 104 as a result of said squeezing. In some scenarios, the duplicated “squeeze-resulting” force is applied to at least one of the user's finger via a trigger mechanism. Similarly, if a right surface of manipulator 104 rubs against a ruff surface of an object, then an internal component of the movable component 462 is actuated such that the handle portion thereof shakes in the right direction so as to mimic the vibration experienced by the manipulator 104 as a result of said rubbing.
Accordingly, one or more vibrational elements 472 can be provided internal to the HCU 116. The vibrational elements 472 can include, but are not limited to, a first vibrational element that can cause the HCU 116 to shake in the left and/or right direction, a second vibrational element that can cause the HCU 116 to shake in the up and/or down direction, and/or a third vibrational element that can cause the HCU 116 to shake in a front and/or back direction. Any combination of the vibrational elements can be actuated to shake the HCU 116 is one or more directions at any given time. A determination as to which vibrational element(s) to actuate at a particular time can be made based on results of a translation process. The translation process involves translating a spatial direction associated with the force applied to the slave device 102 to a coordinate direction associated with the handle portion of the HCU 116. The coordinate direction is determined based on an orientation of the user's hand relative to an orientation of the handle portion. One or more orientations can be detected using any known or to be known sensors which are configured to detect an orientation of an object relative to a pre-defined reference orientation, point, object and/or axes.
Although the control unit 110 was described above to measure inherent latency of signals sent in only one direction (i.e., a reverse direction). The present invention is not limited in this regard. For example, in some scenarios, the control unit 110 may additionally or alternatively be configured to continuously/periodically detect and measure a inherent latency of a communications link in a forward direction and/or an inherent cumulative latency of the communications link in both the forward and reverse directions. The measured values for the inherent cumulative latency can then be used to provide a constant overall latency for signals communicated within the teleoperation system 100.
Referring now to
Graph 600 of
Referring now to
The first reference value is then used in step 708 to select an amount of controlled latency to be added to the communications link at each of a plurality of time points. The amount of controlled latency to be added to the communications link at each time point can be: selected based on a difference between the first reference value and a measured value for the inherent latency at that time point; and/or selected to produce a constant overall latency for the communications link.
Subsequently, the selected amount of controlled latency is added to the communications link at each of the plurality of time points so as to increase the predictability of the telematic operations, as shown by step 710. Notably, the amount of controlled latency added at a first time point of the plurality of time points is different than the amount of controlled latency added at a second time point of the plurality of time points. The controlled latency can be added to the communications link by time delaying a received message by an amount determined based on the first reference value and a current level of inherent latency of the communications link.
In some scenarios, the communications link comprises a forward communications link and the controlled latency is added to the communications link by a slave device of the teleoperation system. Alternatively or additionally, the communications link is a reverse communications link and the controlled latency is added to the communications link by a control unit of the teleoperation system operative to control a remotely located slave device.
Although the invention has been illustrated and described with respect to one or more implementations, equivalent alterations and modifications will occur to others skilled in the art upon the reading and understanding of this specification and the annexed drawings. In addition, while a particular feature of the invention may have been disclosed with respect to only one of several implementations, such feature may be combined with one or more other features of the other implementations as may be desired and advantageous for any given or particular application. Thus, the breadth and scope of the present invention should not be limited by any of the above described embodiments. Rather, the scope of the invention should be defined in accordance with the following claims and their equivalents.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5266875 | Slotine | Nov 1993 | A |
| 5548516 | Gudat et al. | Aug 1996 | A |
| 5581666 | Anderson | Dec 1996 | A |
| 6144884 | Niemeyer et al. | Nov 2000 | A |
| 7676570 | Levy | Mar 2010 | B2 |
| 7747729 | Levy | Jun 2010 | B2 |
| 7957744 | Oesterling | Jun 2011 | B2 |
| 8125907 | Averi | Feb 2012 | B2 |
| 8171152 | White | May 2012 | B2 |
| 8190292 | Niemeyer | May 2012 | B2 |
| 8274891 | Averi | Sep 2012 | B2 |
| 8644164 | Averi | Feb 2014 | B2 |
| 8661218 | Piszczek | Feb 2014 | B1 |
| 8804794 | Malkin | Aug 2014 | B2 |
| 8885498 | Choi | Nov 2014 | B2 |
| 8918215 | Bosscher | Dec 2014 | B2 |
| 8966109 | White | Feb 2015 | B2 |
| 20060109376 | Chaffee et al. | May 2006 | A1 |
| 20060161647 | Wojtkiewicz | Jul 2006 | A1 |
| 20070058929 | Chaffee | Mar 2007 | A1 |
| 20070156286 | Yamauchi | Jul 2007 | A1 |
| 20080144563 | Hart | Jun 2008 | A1 |
| 20080276408 | Gilbert, Jr. et al. | Nov 2008 | A1 |
| 20090106491 | Piszczek | Apr 2009 | A1 |
| 20090157233 | Kokkeby et al. | Jun 2009 | A1 |
| 20090323657 | Gandham | Dec 2009 | A1 |
| 20100070079 | Mangaser et al. | Mar 2010 | A1 |
| 20100321203 | Tieman | Dec 2010 | A1 |
| 20110087371 | Sandberg et al. | Apr 2011 | A1 |
| 20130244670 | Biermann | Sep 2013 | A1 |
| 20130329584 | Ghose | Dec 2013 | A1 |
| 20140095944 | Alexander | Apr 2014 | A1 |
| 20140181334 | Jeyaseelan | Jun 2014 | A1 |
| 20140204743 | Garcia | Jul 2014 | A1 |
| 20140219284 | Chau | Aug 2014 | A1 |
| 20140236518 | Hergesheimer | Aug 2014 | A1 |
| 20140236519 | Hergesheimer | Aug 2014 | A1 |
| 20140340257 | Basnayake | Nov 2014 | A1 |
| 20140376427 | Hui | Dec 2014 | A1 |
| 20150298318 | Wang | Oct 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 2861191 | Apr 2005 | FR |
| 2009091536 | Jul 2009 | WO |
| Entry |
|---|
| Information about Related Patents and Patent Applications, see section 6 of the accompanying Information Disclosure Statement Letter, which concerns Related Patents and Patent Applications. |
| Number | Date | Country | |
|---|---|---|---|
| 20150117466 A1 | Apr 2015 | US |
