The invention relates to audiovisual systems, and more specifically to systems and methods to automatically track a position of a subject within the range of a magnetic field around the subject to facilitate capturing an image of the subject by an image capture device.
Through the evolution of robotics and automatic presenter tracking systems, visual presentations using automated systems have become commonplace. This capability has significantly increased remote attendance through the web, as well as event storage for a variety of situations, such as remote classrooms, professional conferences, legal and political debates, large area security and much more.
The traditional technologies used in automatic presenter tracking systems struggle to keep pace with the increasing range of audiovisual needs. This has become challenging when dealing with advanced technology audiovisual conference rooms, meeting rooms, churches, classrooms, auditoriums, virtual sets, etc. Automatic presenter tracking systems technologies can be categorized as follows:
Horizontal (no vertical) Motion Detection Follower
Horizontal (limited vertical) Infrared (with pendant) Follower
Horizontal (limited vertical) Face Recognition
Horizontal (no vertical) Sound Follower
Vertical plus Horizontal Telemetry Data Follower
Some manufacturers have used a combination of these techniques, such as sound combined with face recognition. Even in those cases, horizontal (pan) and vertical (tilt) is limited to a narrow visual range and rarely including the audience.
Only one presenter to move horizontally left-right on depth limited stage.
Only one camera and one presenter.
Limited room size, layout and lighting for tracking.
Sometimes effective indoors, inoperative or difficult outdoors.
Camera location must be centered in front of the presenter stage.
Presenter cannot move freely behind or around the tracking camera.
Automatic re-acquisition of a lost signal is difficult if not impossible.
Often requires presenters wear visible devices which can be distracting.
Two cameras are required one for sensing one for video.
Examples of conventional systems include the following.
The Motion Detection Camera Tracking Method. This system uses security camera motion detection technology, often combined with pre-programmed preset camera controls activated by electromechanical triggers such as step mat or by manually activated switches aided by IR detectors. It uses two cameras mounted close together; one for motion detection and one for video to ensure both cameras have the same field of view. The motion detection camera is adjusted to a wide-angle shot to see the whole presentation area and the video tracking camera looks at a smaller area and pans back and forth to follow a presenter. Only one person is allowed on stage. If more than one person is on stage, or something is moving in the visual range of the motion detection, tracking may stop and in many cases begin to track the wrong individual or freeze up locked on a bright object. Changes in lighting in the room from a projector or sunlight from a window can be interpreted as motion changes and cause tracking disruptions.
The limitation of this type of tracking system is that it is a technology that relies on motion and changes in brightness to follow the presenter in a limited area. Changes in lighting or distance from the sensing camera will influence the image and the system's ability to track the presenter. This method can be difficult to install and calibrate, requiring factory trained experts to operate. The motion tracking can be unreliable unless aided by triggers (mats, buttons, etc.) and requires a tandem of two cameras, one for motion and one for video and both cameras must be centered in front of the stage. In addition, a fixed zoom position and horizontal (pan) only are available in these systems and pan is limited to the wide angle capability of the motion sensing camera. Finally, motion tracking can be manually disabled providing for only preset shots if the auto tracking system stops working or image becomes too jerky.
The IR Pendant Detection Camera Tracking Method. Pendant camera tracking requires an infrared (IR) illuminator worn as a necklace, or a clip on device by the presenter that can be seen by a specialized IR camera. The IR camera follows the presenter IR source in the scene, commanding the video camera to follow the IR source in real time.
Experience in this environment has identified a few limitations of IR tracking. First, the necklace can be covered by clothing or become invisible all together when presenters turn their back to the IR camera. Despite thoughtful design and engineering, some presenters find the necklace cumbersome and prefer not to wear it over time, instead relying only on presets and static camera angles. In addition, the distance to the IR camera is limited to about 30 feet due to IR camera sensitivity or IR interference. Other IR sources, such as sunlight or florescent lights, make the target harder to find by the IR camera. Even bright lights with little IR content can lock-up an IR specialized camera. The IR camera has also trouble detecting depth of field, which means autofocus loss of the IR camera and the tracking video shot can become jerky or suddenly lockup.
The IR autotracking is limited by the vertical displacement of the IR source on the IR sensing camera image that occurs when the presenter moves back-forth instead of left-right because the camera(s) are aimed down from a high location down to the stage. Finally, illuminator failures and dead batteries tend to be an issue in systems that are frequently used. Though some systems have equipment cases that both secure and recharge the lanyards, the maintenance of the system reduces the overall practicality of use.
The Face Recognition Camera Tracking Method. Computer processing is becoming more powerful and cheaper to integrate into new devices and has led to the development of computer-based tracking. It is effective in a short distance and well-lit video conferencing arena and performs to expectations within a controlled conference style room. Using custom software, a computer-based processor analyses the video image in real time and identifies the subject's eyes within a frame. Much like your digital camera will identify red eye, the processor uses the subject's eyes in the image to get an idea where the presenter is in the frame. Once the camera knows where the subject is, the camera will look for the motion of the subject's lips to confirm who in the frame is talking. Some makers use the aid of sound detection with two microphones to find the presenter.
The face recognition systems can result in disadvantages due to variable lighting and distance from the camera that will influence the clarity of the captured image, affecting the system's ability to recognize and track the subject. These systems are limited to small spaces and to one robotic camera and require initialization before the tracking can begin. The system cannot be used in virtual studios or sets. In addition, movements like turning his or her head a little too far off axis can make for unpredictable results in how the processor interprets the image.
The Audio Based Tracking by Sound Source Localization (SSL). The key advantages of the SSL tracking method is the technology approach which separates the camera, control, and processor from each other. In this method the depth of the room, lighting, room finishes, or pendants are irrelevant to the performance of the system. The combination of sound systems, pressure mats, and IR sensors can create a technology for a multipurpose room. However, this proves a complex design challenge, requiring an integrated design and engineering approach.
The disadvantages of this approach can occur when the voices from two people get into a discussion and may be problematic for the system to decide who to follow. A microphone amplifier sound device may cause tracking to be lost. SSL systems are only effective in small rooms for videoconferencing and only a single presenter is allowed to be talking at a time. In addition, only one fixed zoom position horizontal panning camera system can be sued and the system is not scalable resulting in being limited to small spaces and one robotic camera. Finally, SSL requires initialization resulting in difficult acquisition of the presenter at start up. The system cannot be used in virtual studios or sets.
Thus, the present systems and methods disclosed herein have been developed to overcome the disadvantages described above. The present systems and methods describe various embodiments for scalable audiovisual systems that work in large multifunctional nontraditional spaces.
A system for automatically tracking and capturing an image of a subject is disclosed. The system includes at least one magnetic emitter device, at least one magnetic sensing device attached to at least one subject enabled to receive a magnetic field from the magnetic emitter device. The system further includes an image capturing device in communication with the to the magnetic sensing device in which the magnetic sensing device is enabled to communicate position data to the image capturing device and the position data enables the image capturing device to track a position of the subject.
Also disclosed is a method for automatically acquiring a position of a subject. The method includes receiving at at least one magnetic sensing device attached to at least one subject a magnetic field from at least one magnetic emitter device and processing the magnetic field to determine a position of the at least one magnetic sensing device. The method includes communicating the position data of the subject based on the position of the magnetic sensing device to an image capturing device.
In addition, a method for automatically directing an image capturing device to a subject position is disclosed. The method includes receiving at the image capturing device a position data from a magnetic sensing device attached to a subject and processing the position data to determine a position for directing the image capturing device. The method also directs the image capturing device to the position of the magnetic sensing device to capture an image of the subject based on the position of the magnetic sensing device.
The systems and methods discussed below are advantageous over the prior art because the presenter is totally free to move, stand, sit, and turn in any direction within the magnetic field cover zone, without risk of losing tracking. The Image capturing devices can be placed anywhere in the area and may be chosen based performance. A multiple number of Image capturing devices can be controlled, even if there is only one presenter allowing many different shots to be obtained. Several presenters can be tracked simultaneously and choreographed utilizing full automatic Pan, Tilt, Zoom and Focus control of all cameras in 360 degree horizontal pan camera control (presenter can even walk around the camera). Vertical Tilt camera control (within PTZ or pan/tilt limits) may be included and the system is fully scalable from one presenter one camera in a small room to multiple cameras. The Presenter hub does not have to be visible and can be covered by clothing or worn anywhere on the body. The systems and methods presented allow automatic re-acquisition is reliable once presenter is in the coverage area.
Systems and methods for establishing an automatic position tracking and image capturing system are described in connection with
Although the following descriptions use a limited number of image capturing devices, magnetic emitter devices and magnetic sensing devices, it is understood that any number of elements described below can be used in the automatic position tracking and image capturing system presented herein.
Referring to
Although magnetic emitter 202 is shown in
The transmitter 512 may be included as a component of magnetic sensing device 206 or separate from magnetic sensing device 206 but connected to magnetic sensing device 206 by a physical cable or wireless connection. The wireless connection may include any appropriate wireless standard, including, but not limited to Bluetooth©, IEEE 802.11 series standards, RF signal or other appropriate means.
It should be understood that the above descriptions are meant as examples only placed in simplest terms to aid in understanding the systems and methods described herein, and are not intended to be limiting embodiments of the present invention. For example, magnetic emitter 202 and magnetic sensing device 206 may include additional coils and a plurality of elements described above as well as additional elements not shown. Position processing unit 510 may include more than one processor unit 602 and additional memory units 604 and programing storage units 606, as well as other components not shown.
The control unit 704 and receiver 702 may be an integral part of image capturing device 210 or may be separate elements in communication with image capturing device 210 through a cable or wireless connection. The wireless connection may include any appropriate wireless standard, including, but not limited to Bluetooth©, IEEE 802.11 series standards, RF signal or other appropriate means.
Control unit 704, although discussed above as including one processor 802, may include many processors and other components including components not shown, and may include controls for other aspects of image capturing device 210 such as pan, tilt, zoom, focus and other control features.
Turning now to
In the event subject 1020 exits the magnetic field 230, the magnetic sensing device 206 will no longer be enabled to determine a position with respect to magnetic emitter 202. As a result image capturing device 210 will no longer track subject 1020. However, immediately upon reentry into magnetic field 230, the magnetic sensing devices 206, re-acquires the signal from magnetic emitter 202 because magnetic sensing devices 206 is continually sensing magnetic field 230. Thus, the magnetic sensing device 206 may continue to communicate position data to image capturing device 210 so that the subject 1020 may continue to be tracked by image capturing device 210 within their respective magnetic field 230.
Although as described above, image capturing devices 1110a and 1110b receive position data through RF signals 1125a and 1125b respectively, image capturing devices 1110a and 1110b may also receive position data from magnetic sensing devices 1106a and 1106b through, for example, a wireless signal such as Bluetooth©, IEEE 802.11 series standards or other appropriate means. Although the above discussed embodiments of the automatic position tracking and image capturing system included two of each of magnetic emitters 1102a, 1102b; magnetic sensing devices 1106a, 1106b; image capturing devices 1110a, 1110b; and subjects 1120, 1121, many combinations are possible and the above discussed examples are not intended to limit the possible embodiments and system configurations. For example, the above discussed systems may be configured for one magnetic emitter, one magnetic sensing device, one subject and several image capturing devices providing a variety of perspective video transmissions of the subject.
In an alternate embodiment, several of each of magnetic emitters, magnetic sensing devices and image capturing devices may track several subjects in order to automatically follow each presenter with one or several image capturing devices and provide a presentation of the captured image of each of the several subjects.
In another embodiment, the system may include several subjects and each subject may possess magnetic sensing devices that are in communication individually with image capturing devices, thus tracking each subject individually.
The automatic position tracking and image capturing system of the present invention provides multiple combinations of magnetic emitters, magnetic sensing devices and image capturing devices that can be used to follow a subject in a desired presentation environment.
The present invention may be implemented in many applications where it is desirable for a subject to be tracked automatically by an imaging device and may include live or virtual applications.
In some embodiments of the present invention, the presentation environment may be a lecture in academic, a group meeting, political debate, or other similar presentation environments.
In some other embodiments of the present invention, the presentation environment may be a video conferencing application.
In yet other embodiments of the present invention, the presentation environment may be theatrical, television, motion picture, virtual reality or other entertainment environments.
Furthermore, the present invention may flexibly allow multiple arrays of image capturing devices that may be applied to the systems and methods discussed herein.
In some embodiments of the present invention, the image capturing device may be a high definition video camera.
In some embodiments of the present invention, the image capturing device may be a camera capable of three dimensional image capture.
In some other embodiments of the present invention, the image capturing device may be a still image camera.
In yet other embodiments of the present invention, the image capturing device may a standard definition video camera.
In other embodiments of the present invention, the image capturing device may record images or transmit images for viewing or separate recording.
While the invention has been described in connection with what is presently considered to be the most practical and preferred embodiment, it is to be understood that the invention is not to be limited to the disclosed embodiment, but on the contrary, is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.