Patent Application
20060098087
The present invention relates to a recording device for image recording having a recording unit attachable to the head of the user, having a sensor device for recording eye motions of the user, and having an analysis unit connected downstream from the sensor device, which generates control signals that are applied to an actuator acting on the recording unit, through which an image section recorded by the recording unit is movable.
Furthermore, the present invention relates to a method for controlling the alignment of an image section recorded by the head-worn recording unit.
A recording device of this type and a method of this type are known from WO 96/36271. The known device comprises a head support, which may be attached to the head of a user. Cameras are attached to the head support, whose optical axes may be aligned by two-dimensional servo systems in accordance with the position of the eyes. For this purpose, the position of the eyes is detected with the aid of partially reflecting deflection mirrors, which are positioned in front of the eyes and image the eyes of the user on suitable sensors in the reflected wavelength range. The eye position detected with the aid of the sensors is then analyzed and converted into control signals for the servo systems. The optical axes of the camera are tracked on the instantaneous viewing direction of the user by the known device.
The known recording device is suitable above all for monitoring the viewing direction of the user.
A disadvantage of the known recording device and the known method is that no stationary images may be recorded by the camera. Rather, it is necessary to compensate for motions of the head and the external world with the aid of image-processing methods to produce a somewhat stationary image, which may be followed easily by an observer.
On the basis of this related art, the present invention is based on the object of providing a recording device and a method for controlling the recording device, using which stationary images may be recorded easily.
These objects are achieved by the recording device and the method having the features of the independent claims. Advantageous embodiments and refinements are specified in claims dependent thereon.
The recording device is distinguished in that the sensor device completely detects eye movements of at least one eye of the user. Therefore, not only pitching and yawing movements, but rather also the rolling movements of an eye are detected. In this case, the movement of the eye refers to the shifting and rotating of the pupil and the iris of the eye. The recording device thus detects rotational movements of the eyeball around three orthogonal spatial axes. These movements are analyzed by the analysis unit and converted into control signals for a movement of the recording unit which compensates for the head movement. The recording unit is particularly controlled in this case in such a way that the movement of the field of vision of the recording unit follows all components of the eye movement. The recording unit therefore performs not only pitching and yawing movements, but rather also rolling movements. As a result, essentially stationary images are imaged on the image sensor of the recording unit. This is because in the scope of the vestibulo-ocular reflex (VOR), the equilibrium organ in the inner ear of the user assumes the role of a motion sensor, which provides information to the brain on the rotational velocity of the head and on the alignment of the head in relation to gravity. In the brain, the velocity information is converted using an integration procedure into a position signal, which is relayed to the oculomotor nuclei with a reversed sign. This biological reflex results in the eyes always rotating against the head movement, so that the position of the surrounding image imaged on the retina of an eye is stabilized. Therefore, if the rotations of the eyeball around the three spatial axes are detected completely, and if the information obtained in this case is used for the purpose of producing control signals, through which the recording unit is caused to perform a movement that completely follows the eye movement, head movements of the user are compensated for and a largely stable viewing field of the recording unit results, which remains as stationary as the image of the surroundings imaged on the retina. Therefore, image processing methods for subsequent stabilization of the images recorded by the camera, as in the related art, do not have to be used in the recording device.
In a preferred embodiment of the present invention, a device for intrasaccadic suppression is provided in the analysis unit or in the recording unit, which freezes the image recorded by the recording unit in the event of rapid eye movements, the saccades. In this way, smearing of the images recorded by the recording unit in the event of rapid eye movements of the user is avoided.
In a further preferred embodiment, the sensor device comprises an infrared mirror positioned directly in front of the eye and an infrared camera, on which an infrared image of the eyes is imaged. This arrangement allows the user a free, unrestricted field of vision and, simultaneously, complete detection of the eye movements of the user by the sensor device.
In order that the recording units, which are particularly optical cameras, may follow all components of the eye movements, the recording units are preferably mounted so they are rotatable around three spatial axes. In particular, the cameras may be supported by a gimbal suspension.
If the movement of both eyes of the user is detected, important information for a possible autofocus function of the recording unit or the recording units may be obtained from the vergence position of both eyes. This is because the vergence angle of the eyes is a function of the distance to the observed object.
In the following, the present invention will be explained for exemplary purposes on the basis of the attached drawing.
The recording device 1 illustrated in
The images recorded by the infrared cameras 5 are fed to an analysis unit 9, which determines the components of the movement of the eyes 8. In this case, methods of video oculography known to those skilled in the art are used. These methods, which are known to those skilled in the art, are not subject matter of the present application per se. Control signals 10 are generated and relayed to a motor control 11 on the basis of the components of the movement of the eyes 8 determined by the analysis unit 9. In accordance with the control signals 10, the motor control 11 controls drive motors (not shown in
The different rotational movements of the eyes 8 are shown enlarged in
Since the movement of the cameras 12 follows the movements of the eyes 8, the images recorded by the cameras 12 are stationary. This is because the unconscious control of the eyes 8 is performed in such a way that the image on the retina is as stationary as possible. Thus, the vestibulo-ocular reflex already noted ensures that the eyes 8 always rotate against the movement of the head 3 so that the image of the surroundings on the retina is stationary. In the scope of the vestibulo-ocular reflex, the equilibrium organ in the inner ear of the user 4 assumes the role of the motion sensor, which provides information to the brain of the user 4 on the rotational velocity of the head 3 and on the alignment of the head 3 in relation to gravity. In the brain of the user 4, the velocity information is converted with the aid of an integration procedure into a position signal, which is in turn relayed with reversed sign to the oculomotor nuclei. As a result, the eyes therefore always rotate against the head movement. Recognizing faces or reading street signs while running is only possible in this way, for example. This vestibulo-ocular reflex is used by the recording device 1 for stabilizing the viewing field of the camera 12, in that the movement of the camera 12 is controlled by the pitching movements 22, yawing movements 24, and rolling movements 26 of the eyes 8 of the user 4.
In addition, still further reflexes contribute to stabilization of the images recorded by the cameras 12. A similar mechanism becomes active, for example, if large-area visual stimuli pass by the user 4. This is the case, for example, if the user 4 looks out the window of a moving train. The brain of the user 4 extracts velocity information from the optical stimuli and relays this to the same oculomotor structures which are also the basis of the vestibulo-ocular reflex. The eyes 8 are controlled against the movement, so that the user 4 obtains a clear image of his environment in spite of the moving train. Since the eyes 8 may not be deflected arbitrarily, the eyes 8 are reset at regular intervals using rapid eye movements, the saccades. This mechanism is referred to as optokinetic nystagmus (OKN), which comprises a sequence of slow compensatory eye movements and rapid restoring movements.
In addition to these involuntary eye movements, the user 4 may also perform voluntary eye movements. These are slow eye tracking movements and rapid viewing jumps, which are also referred to as saccades. The first are used by the user 4 if he follows a flying bird with his eyes, for example. The rapid viewing jumps are used, in contrast, if the eyes 8 are moved back and forth between two speech partners, for example.
In the recording device 1, these complex biological eye movements assume the control of the cameras 12. Movements of the eyes 8, which are used for the purpose of obtaining stable images of the environment and, in addition, for allowing active exploration of this environment, are detected by the infrared cameras 5 and converted into equivalent movements of the cameras 12. It is therefore not necessary to equip the recording device 1 with technical motion sensors and stabilize the image of the cameras 12 with the aid of these sensors, since the natural reflexes of the equilibrium organ described above are exploited to compensate for the movement.
A further biological effect which may be exploited for controlling the recording device 1 is the vergence position of the eyes 8. During binocular observation of objects, the eyes 8 assume an angle as a function of the object distance, as in triangulation, so that each eye 8 may observe the object in the region of highest resolution, the fovea. Important information for a possible autofocus function of the cameras 12 may be calculated from the vergence position, since the vergence angle is a function of the distance to the observed object.
Furthermore, it is possible to execute the image processing performed by the brain of the user 4 correspondingly on the images recorded by the cameras 12. In order to suppress erroneous apparent movements of the environment in the event of rapid movements of the eyes 8, for example, the intrasaccadic suppression system in the brain of the user 4 causes suppression of the movement perception during a saccade. This results in a transsaccadic constancy of the perceived spatial movement.
In order to avoid smearing of the image recorded by the cameras 12 in the event of rapid movements of the eyes 8, a saccadic suppression device may be provided in the analysis unit 9, which is always triggered when a rapid viewing change occurs. The artificial movement suppression may be performed, for example, in that the last image before the saccade is repeated or frozen for the duration of this saccade. The time span in which freezing of the image recorded by the camera 12 is necessary is in the range of 100 ms, as a function of the amplitude of the saccade.
The recording device 1 allows the user 4 to make video recordings even under conditions under which unblurred recordings were not possible until now. In this case, the user 4 may freely move his head 3 and eyes 8, as well as his arms and legs. During the film recording, the user 4 may, for example, run, move in difficult country, or concentrate on controlling a machine or operating a device, without having to worry about recording blurry images. This is because the vestibulo-ocular and optokinetic reflexes ensure a natural stabilization of the image field recorded by the camera 12 during any arbitrary movements of the head 3.
If, in an altered embodiment of the recording device 1, the images recorded by the cameras 12 are projected to the user 4 via a projection device on the infrared mirror 7, additional information about his environment may be overlaid to the user 4 as a function of the type of the camera 12 used and as a function of a possible upstream image processing device. Thus, for example, false color representations with emphasis of specific features of his environment or the representation of thermal images are conceivable. The projection of the images on the infrared meter 7 is preferably performed with the aid of liquid crystal displays.
In addition, the present invention allows an array of further applications.
In the field of film and television, films may be produced which reproduce the actual, subjective view of the user 4 or cameraman. Using the recording device 1, film products may thus be produced whose camera control corresponds to the natural viewing conditions, in contrast to the artificial, directed camera control of typical film products. A new quality of filmic representation in entertainment and documentation thus arises in relation to the film techniques currently typical. This also opens new artistic possibilities in the design of films.
The use of the recording device 1 is not solely restricted to human users 4. Modifying the head support 2 for animals is also conceivable. New possibilities would result in the field of animal films in this case.
In the field of sports reporting, the recording device 1 allows the transmission of unblurred images from the view of an athlete, even of skiers, ski jumpers, or dancers.
In the military field or in the field of border protection, a new type of night-vision devices is conceivable, which projects the thermal image in the direction of the viewing target to the user 4 on the semitransparent infrared mirror 7, with unrestricted field of vision and the highest possible mobility.
In neuroscientific research, the recording device 1 may be used for the purpose of analyzing the exploration behavior in freely mobile subjects or patients. For example, the development of viewing control from child to adult may be investigated with the aid of the recording device 1. Furthermore, disorders in psychiatric, neurological, or ophthalmological illnesses may be analyzed.
Furthermore, it is possible using the recording device 1 to investigate the stimulus-caused viewing control with advertising, in the workplace, or while operating complex devices in experiments in the framework of marketing, ergonomics, or work safety.
It is to be noted that in an altered embodiment of the recording device 1, the sensor device comprises at least one contact lens which may be applied to an eye 8 of the user 4 and is provided with an induction coil, which generates an induction signal displaying the orientation of the induction coil in a magnetic field extending on the eye region of the user 4. For this purpose, the different spatial components of the magnetic field are modulated differently, so that the components of the induction signal originating from the different spatial components of the magnetic field are separated in the induction signal and their relative strengths may be compared to one another. The orientation of the induction coil and therefore the position of the eye 8 may then be concluded from the relative strengths of the components of the induction signal.
Finally, it is to be noted that the recording device 1 may be operated both binocularly, as shown in
| Number | Date | Country | Kind |
|---|---|---|---|
| 102 51 933.1 | Nov 2002 | DE | national |
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/DE03/03671 | 11/6/2003 | WO | 12/6/2005 |
