Patent Application
20090135133
A portion of the disclosure of this patent document contains material which is subject to copyright protection. This patent document may show and/or describe matter which is or may become trade dress of the owner. The copyright and trade dress owner has no objection to the facsimile reproduction by anyone of the patent disclosure as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright and trade dress rights whatsoever.
1. Field
This disclosure relates to man-in-the-loop control systems and methods based on sensing three-dimensional motion of a human body.
2. Description of the Related Art
Systems controlled by a human operator are ubiquitous in modern society, and can be found in activities ranging from driving an automobile to playing video games. Within this description, such control systems will be termed “man-in-the-loop” control system to distinguish from fully automated closed loop control systems. Man-in-the-loop control systems generally utilize motions of a person's hands/arms and feet, such as turning the steering wheel and pressing the gas and break pedals when operating an automobile. Current man-in-the-loop control systems generally do not utilize movements of the operator's hips, pelvis, and torso for control purposes.
Some existing simulation systems, such as arcade motorcycle simulators are believed to sense the operator's weight distribution and thus effectively sense the position of the operator's body as a whole. However, it is believed that such simulation systems do not sense the movements or the positions of individual elements of the operator's hips, pelvis, and torso for control purposes.
Throughout this description, elements appearing in figures are assigned three-digit reference designators, where the most significant digit is the figure number and the two least significant digits are specific to the element. An element that is not described in conjunction with a figure may be presumed to have the same characteristics and function as a previously-described element having a reference designator with the same least significant digits.
The midsagittal plane (also called the median plane) is defined (Merriam Webster Medical Dictionary) as a vertical longitudinal plane that divides a bilaterally symmetrical animal, such as a person, into right and left halves. A coronal plane (also called a frontal plane) is defined as a plane parallel to the long axis of a body and at right angles to the midsagittal plane.
Some elements of the control system described herein are symmetrical about the midsagittal plane. The reference designators applied to these elements in the figures may include the suffixes “L” and “R” to indicate mirror-imaged left-side and right-side elements having the same function.
Within this description, the term “elastically coupled” will be used to indicate that a first element is joined to a second element with a flexible connection that defines and tends to restore a nominal positional relationship between the elements but allows relative motion in at least one direction.
Description of Apparatus
Referring now to
The sensors 140 may include any apparatus or device that measures the instantaneous position, instantaneous motion, rate of motion, cumulative motion, or other parameter of a portion of the operator's body. The sensors 140 may include conventional control devices such as the steering wheel (which measures the cumulative motion of the operator's hands) and pedals (which measure the instantaneous position of the operator's foot or feet) of an automobile, or a computer mouse (which measures the rate of linear motion of the operator's hand). The sensors 140 may also include known devices such as accelerometers, gyros, potentiometers, and photosensors to measure the position of the operator's body and limbs and/or the angles of the operator's joints.
The device under control may be a real apparatus such as a vehicle. The term vehicle, as used herein, includes automobiles, aircraft, motorcycles, boats, construction equipment or any other apparatus that moves under control of an operator. The device under control may be a computing device running an application program. The application program may be as a simulation or some other program. The application program may simulate a physical system or a physical activity such as driving an automobile or skiing. The objective of the simulation may be entertainment (such as a video game), training, testing, exercise, rehabilitation, or some other objective.
In
The seat 220 may include a lower back support such as lower back support 226. The lower back support 226 may be elastically coupled to the left seat 222L and the right seat 222R by flexible elements or some other mechanism that allows the lower back support to support the operator without inhibiting or preventing movement of the operator's 210 pelvis. The lower back support 226 may have a variety of shapes other than that illustrated in
The seat 220 may include an upper back support 228. The upper back support 228 may be movably coupled to the lower back support 226. The movable coupling of the upper back 228 support to the lower back support 226 may allow free angular and linear motion of the upper back support 228 with respect to the lower back support 226. For example, the upper back support 228 may be movably coupled to the lower back support 226 by a telescoping ball-and-socket joint 230. The telescoping ball-and-socket joint may include a bushing/retainer 232 attached to the lower back support 226. A ball 234 may be free to rotate within the bushing/retainer 232. A shaft 236 may be attached to the upper back support 228. The shaft 236 may be free to move linearly through a bushing within the ball 234. The shaft 236 may move through the ball 234 when, for example, the operator 210 leans forward and backward. The telescoping ball and socket joint 230 may include soft or hard stops (not shown) to limit the rotation of the ball 234 and to limit the linear motion of the shaft 236. The telescoping ball and socket joint 230 may also include springs and/or damping mechanisms affecting either or both the rotational or linear motion.
The upper back support 228 may include side extensions that wrap, at least partially, around an occupant's torso under their arms. The upper back support may include upper extensions that may wrap, at least partially, over the top of the occupant's shoulders. The upper back support 228 may have a variety of shapes other than that illustrated in
The control system 200 may include a pelvis angle sensor 242 to estimate the angle of the operator's 210 pelvis in a frontal plane (as indicated by arrow 243). The pelvis angle sensor 242 may measure the angle between the left and right seats 222L/R using an angle sensor such as a potentiometer or rotary optical encoder linked to the left and right seats 222L/R. The pelvis angle sensor may measure the vertical positions of the left and right seats 222L/R using a linear differential transformer, a linear optical encoder, or other position sensors and estimate the angle of the operator's pelvis from the difference in the vertical positions of the left and right seats 222L/R. The pelvis angle sensor 242 may use one or more accelerometers, gyros, linear differential transformers, optical sensors, acoustic sensors, or other sensors to estimate the operator' pelvis angle 243.
The control system 200 may include a torso angle sensor 244 to estimate the angle of the operator's 210 torso in a frontal plane (as indicated by arrow 245). The torso angle sensor may also measure the operator's torso angle in the sagital plane, which would be motion of the torso essential normal to the plane of
The positional information measured by the pelvis angle sensor 242 and the torso angle sensor 244 may be communicated to the device under control 260. The positional information may be communicated as analog signals, parallel digital signals, or serial digital signals. The communication path may be wired or wireless. The positional information may be communicated continuously or periodically (as is done for a conventional joystick or mouse). If the positional information is communicated periodically, the period must be sufficiently short to allow stable control of the device under control 260.
Portions of the seat may include buttons, switches, proximity sensors or other devices (not shown) that can receive input from the operator 210 not originating from the movement of the operator and/or the seat.
The device under control 260 may be a real apparatus such as an automobile, aircraft, construction equipment, wheelchair, or other vehicle. For example, the device under control may be a vehicle where the operator's pelvis angle, as measured by the pelvis angle sensor, and/or the operator's upper body rotation, as measured by the torso angle sensor, are used to control steering. Using the operator's body angles to control steering may leave the operator's hands and arms free for other tasks.
The device under control 260 may be a computing device running an application program which may be controlled, at least in part, by the sensed positional information. For example, the application program may be simulation of some physical system, and the positional information sensed by the pelvis angle sensor 242 and the torso angle sensor 244 may be used to control, at least in part, the simulation. For further example, the device under control 260 may be a video game (or training system) simulating motorcycle riding or racing. In this example, the device under control may be a computer running an application program that models the dynamics of a motorcycle and a rider and provides feedback to the operator via an image on a display device (not shown in
All of portions of the seat 220 may be wearable such that all of portions of the seat 220 may remain attached to the operator 210 when the operator 210 exits the vehicle or other device under control. A wearable seat may physically or virtually connect to the vehicle or other device being controlled. A wearable seat may physically connect to a structure for support of the operator such as a vehicle, a stand or other support device. For example, referring to
The seat 320 may allow independent rotation of the operator's thighs in elevation (raising and lowering the operator's thighs) and azimuth (opening and closing the operator's thighs). The seat 320 may also allow some amount of independent longitudinal roll of the operator's thighs and the left and right seats 322L/R about separate axes approximately parallel to the long dimension of each seat. The seat 320 may include left and right thigh angle sensors 346L/R to sense the angular position of each of the operator's thighs in one, two, or three dimensions. The thigh angle sensors 346L/R may use one or more potentiometers, accelerometers, gyros, linear differential transformers, optical sensors, acoustic sensors, or other sensors and combinations thereof to estimate the angles of the operator's thighs on one, two or three axis.
The seat 320 may include a left lower leg support 332L and a right lower leg support 332R. The left lower leg support 332L may be elastically coupled to the left seat 322L by a flexible element 333L located proximate to the inner side of the occupant's left knee. The right lower leg support 332R may be elastically coupled to the right seat 322R by a similar flexible element (not visible) located proximate to the inner side of the occupant's right knee. The right and left lower leg supports 332R/L may be elastically coupled to the right and left seats 322R/L, respectively by other mechanisms including flexible elements located on the outside of the occupant's knees, flexible elements on both the inside and outside of the occupants knees, hinges, hinges in combination with springs and/or dampers, and other mechanical structures. The left and right lower leg supports 332L/R may support and, to at least some degree, constrain an occupant's lower legs.
The seat may include left and right knee angle sensors 348L/R to sense the angles between the operator's lower legs and thighs. The knee angle sensors 348L/R may also sense the rotation of the operator's lower legs with respect to the operator's knees. The knee angle sensors 348L/R may use one or more potentiometers, accelerometers, gyros, linear differential transformers, optical sensors, acoustic sensors, or other sensors and combinations thereof to estimate the angles of the operator's knees on one or two axis.
The positional information measured by the thigh angle sensors 346L/R and the knee angle sensors 348L/R may be communicated to the device under control 360. The positional information may be communicated as analog signals, parallel digital signals, or serial digital signals. The communication path may be wired or wireless. The positional information may be communicated continuously or periodically (as is done for a conventional joystick or mouse). If the positional information is communicated periodically, the period must be sufficiently short to allow stable control of the device under control 360. The device under control 360 may also receive positional information from a pelvis angle sensor and a torso angle sensor (not shown in
The device under control 360 may be a real apparatus or may be a simulation of some system or activity. The device under control may be a simulation of a physical activity performed by a simulated person. For example, the device under control 360 may be a video game (or training system) simulating downhill skiing. In this example, the device under control may be a computer running an application program that models the dynamics of a skier and provides feedback to the operator via an image on a display device (not shown in
All of portions of the seat 320 may be wearable as previously described.
Referring now to
Although the computing device 460 may be implemented in a personal computer, the processes and apparatus may be implemented with any computing device. A computing device as used herein refers to any device with a processor, memory and a storage device that may execute instructions including, but not limited to, personal computers, server computers, computing tablets, set top boxes, video game systems, personal video recorders, telephones, personal digital assistants (PDAs), portable computers, and laptop computers. These computing devices may run an operating system, including, for example, variations of the Linux, Unix, MS-DOS, Microsoft Windows, Palm OS, Solaris, Symbian, and Apple Mac OS X operating systems.
The storage device 463 may be any storage device included with or otherwise coupled or attached to a computing device. Storage devices include hard disk drives, DVD drives, flash memory devices, and others. As used herein, a storage device is a device that allows for reading and/or writing to a storage medium. The storage device 463 may use a storage media to store program instructions which, when executed, cause the computing device to perform the processes and functions described herein. These storage media include, for example, magnetic media such as hard disks, floppy disks and tape; optical media such as compact disks (CD-ROM and CD-RW) and digital versatile disks (DVD and DVD±RW); flash memory cards; and other storage media.
The interface 468 may contain specialized circuits, firmware, and software to receive positional information 440 from sensors (not shown) such as the pelvis angle sensor, torso angle sensor, hip angle sensors, and knee angle sensors described in conjunction with
The sound module 466 and audio transducer 467 may be used to generate audible sounds which may serve as feedback to the operator. The audio transducer 467 may be one of more loudspeakers or headphones. The display controller 469 and display 470 may be used to provide a visual image which may serve as feedback to the operator.
Description of Processes
Referring now to
Controlling the device may involve using the sensed positional information to determine specific parameters used to control the device. For example, if the device is a vehicle, the positional information indicative of the angle of the operator's pelvis may be used to control, at least in part, the steering of the vehicle. For further example, if the device is a simulation of a physical activity by a simulated person, the sensed positional information may be used to define the corresponding joints angles of the simulated person. The sensed positional information may be used to control other functions in other types of devices.
The sensory feedback provided to the operator may include visible images, audible sounds, and tactile sensations such as pressure, force, and vibration applied to portions of the operator's limbs or body. The feedback may be intrinsic, such as the scene through the window of a moving vehicle, the sounds of the tires and other portions of the moving vehicle, the physical effects of acceleration and vibration. The feedback may be synthetic, such as a visible image formed on a display device, synthesized sounds, and force vibration, and other tactile sensations provided by transducers coupled to the operator. The feed back may include a combination of intrinsic and synthesized effects.
Closing Comments
Throughout this description, the embodiments and examples shown should be considered as exemplars, rather than limitations on the apparatus and procedures disclosed or claimed. Although many of the examples presented herein involve specific combinations of method acts or system elements, it should be understood that those acts and those elements may be combined in other ways to accomplish the same objectives. With regard to flowcharts, additional and fewer steps may be taken, and the steps as shown may be combined or further refined to achieve the methods described herein. Acts, elements and features discussed only in connection with one embodiment are not intended to be excluded from a similar role in other embodiments.
For means-plus-function limitations recited in the claims, the means are not intended to be limited to the means disclosed herein for performing the recited function, but are intended to cover in scope any means, known now or later developed, for performing the recited function.
As used herein, “plurality” means two or more.
As used herein, a “set” of items may include one or more of such items.
As used herein, whether in the written description or the claims, the terms “comprising”, “including”, “carrying”, “having”, “containing”, “involving”, and the like are to be understood to be open-ended, i.e., to mean including but not limited to. Only the transitional phrases “consisting of” and “consisting essentially of”, respectively, are closed or semi-closed transitional phrases with respect to claims.
Use of ordinal terms such as “first”, “second”, “third”, etc., in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.
As used herein, “and/or” means that the listed items are alternatives, but the alternatives also include any combination of the listed items.
Other related applications: copending U.S. patent application Ser. No. 11/860,497, filed Sep. 24, 2007, entitled “SEAT WITH 3D MOTION INTERFACE”.
