The present invention relates generally to methods and devices for controlling a scanning mirror for a moving or rotating camera with the purpose of stabilizing the captured image by compensating for the rotation by the scanning mirror.
In imaging surveillance systems, usually high resolution images are generated from a scenery by rotating a camera with an image sensor or focal plane array, and by capturing images during the rotation from different view angles. These individual images can be merged together to form a high-resolution panoramic image of the scenery. However, as the camera is rotated to capture images from different angles of the scenery, the field of view of the pixels does not remain constant due to the rotation, and the integration time for light of the image sensor or focal plane array is often not fast enough to avoid substantial blurring of the image. Often, the distance moved by the camera is of the order of several pixels during an integration. Therefore, a system is needed that can efficiently compensate the rotational movement of the camera to capture images with no or substantially less blurring.
One aspect of the present invention provides for a scanning imaging apparatus. The scanning imaging apparatus preferably includes a rotatable support platform, and an imaging device that is attached to the support platform. Moreover, the scanning imaging apparatus further preferably includes a mirror that is rotatably attached to the support platform and is configured to deflect an optical path of the imaging device, a first motor configured to continuously rotate the rotatable support platform at a first angular velocity, and a second motor configured to change an angle of the mirror relative to an optical axis of the imaging device at a second relative angular velocity relative to the optical. Moreover, the scanning imaging apparatus also preferably includes a controller configured to control the angle of the mirror so that a waveform of the angle of the mirror as a function of time does not have high frequency components.
According to another aspect of the present invention a rotating camera system is provided. The rotating camera system preferable includes a first motor, a camera forming an optical axis, the camera being rotatable by the first motor, and a mirror arranged in a path formed by the optical axis configured to reflect the optical axis of the camera to form a reflected optical axis. Moreover, the rotating camera system further preferably includes a second positional motor that is connected to mirror for changing a relative angle between the optical axis of the camera and the reflected optical axis, and a controller configured to control the relative angle so that a waveform of the relative angle as a function of time does not have high frequency components.
The accompanying drawings, which are incorporated herein and constitute part of this specification, illustrate the presently preferred embodiments of the invention, and together with the general description given above and the detailed description given below, serve to explain features of the invention.
a-2g are schematic top views of the rotating optical assembly showing the control of the scanning mirror step-by-step in during an image acquisition and readout period;
Herein, identical reference numerals are used, where possible, to designate identical elements that are common to the figures. Also, the images in the drawings are simplified for illustration purposes and may not be depicted to scale.
Moreover,
With the rotation of disk 180 and the camera view that is redirected towards a scene 170 by scanning mirror 130 via optical paths O1 to O2, camera 110 can capture images 160, 162 at repeating moments during rotation to scan scene 170, so that a panoramic 360° degree view can be later generated from consecutive images 160, 162. Unlike the first motor 150 that usually only rotates in one direction, for example a continuous counter-clockwise rotation around rotational axis R2 with angular velocity Ω as shown in
The scanning mirror 130 is actuated by second motor 140 so as to compensate for the rotation of camera 110 and lens 120 by first motor 150 during a time an image is acquired by camera 110 by a counter-rotation. Therefore, the rotational axes R1 of first motor 150 and R2 of second motor 140 are substantially parallel, and during image capture of camera 110, second motor 140 rotates mirror 130 counter the rotation of first motor 150 at substantially the same rotational speed, so that ω corresponds to −Ω (negative Ω) within a certain tolerance. For example, while camera 110 is capturing an image 160 of scene 170 and at the same time camera 110 is rotated by first motor 150 by angular velocity Ω, mirror 130 is counter-rotated by second motor 140 with a angular velocity ω that is the same or substantially similar to angular velocity Ω of first motor 150. This counter-rotation during image capture allows to stabilize the reflected optical axis O2 to be oriented towards the same direction during the capturing of image 160 regardless of rotation Ω of camera 110. Next, when an adjacent image 162 is captured from scene 170, the scanning mirror 130 is repositioned by second motor 140 to direct second optical axis O2 towards a new position on the scene 170 to capture image 162, and the second optical axis O2 is again stabilized to the same direction O2 by the counter-rotation. This movement of scanning mirror 130 is repeated for each capturing of a subsequent image along the scene 170 to minimize motion blur on the image that would result from rotation of camera 110 during image capture with angular velocity Ω. Consecutively captured images 160, 162 may be entirely separate from each other, may be bordering each other closely, or may also overlap, depending on angular velocity Ω, image capturing frequency f of camera 110, and the width of the field of view generated by camera 110 and optics 120.
In particular, the second motor 140 that positions scanning mirror 130 is controlled such that scanning mirror 130 is moved to stabilize optical axis O2 to a direction that is present at the start of an image integration by image sensor 114 of camera 110, and this direction is maintained until the image integration is completed, and no more image data is captured for the present frame. Ideally, and as explained above, the relative angular position α is linearly decreased by angular velocity ω to counter the linear increase of absolute angular position γ. Next, instead of abruptly and rapidly moving back scanning mirror 130 to the initial angular position for the next image capture, scanning mirror 130 is moved back in a sine-like waveform, and in the variant shown, without any angular accelerations dω/dt above a certain threshold, and without exceeding a maximal angular velocity Ωmax for the relative angular position α after the image integration in camera 110 has ended. The time period for moving back the scanning mirror to a new image capturing position includes at least the time all the pixel values from the matrix of the image sensor 114 is read out. This is different from background scanning systems, in which a scanning mirror snaps back immediately, as shown in the waveforms represented in
However, as shown in
The above described control method of the scanning mirror presents many advantages. For example, rotating optical assembly 100 for low-light surveillance systems often uses cameras 110 having image sensors 114 with a very high sensitivity to be able to capture valuable images at low light. Such image sensors 114 usually operate without a pixel-based electronic shutter mechanism, and also have a high pixel fill factor, so that high pixel sensitivity is guaranteed. In light of the architecture of these image sensors, it may not possible to integrate a new image while the previous image has not yet been read out, and therefore a dead time between two successive image integrations tends to be longer. Therefore, the increased duration of the dead time as compared to some less sensitive image sensors, such as interline image transfer sensors, can be used to move back scanning mirror 130 to its initial position for the next image capture without the need of a fast and powerful motor that allows very fast angular speeds and accelerations, and at the same time, the image acquisition process from camera 110 is not delayed.
Also, such abrupt movement of the scanning mirror 130 has several disadvantages. First, when a scanning mirror is snapped back rapidly to an image capturing position, the motor positioning the scanning mirror is subject to very high forces due to the inertia of the mirror, and therefore would require a more powerful motor that may be heavier, more expensive, more voluminous, and require more power. For example, an exemplary scanning mirror 130 may have a size of 10 cm to 10 cm, a thickness of 5 mm with a weight of 100 grams, thereby having a moment of inertia that would require an motor with substantial torque for high angular accelerations to move a scanning mirror. In addition, the rapid acceleration on scanning mirror can also cause the mirror to be subject to bending forces and mechanical oscillations that could impact the image quality of images 160, 162 captured by camera 110, even if scanning mirror 130 stabilizes optical axis O2. These mechanical oscillations and forces can also be the cause of rapid aging of the materials shortening the lifetime of the system.
In addition, because second motor 140 and scanning mirror 130 are usually not located in the rotational axis R1, but offset by a radius D, it is important to keep mirror 130 and motor 140 as light weight as possible, to avoid additional weight to compensate for the unequal weight distribution around rotational axis R1 on disk 180. Depending on the angular velocity of rotation Ω, additional weights would have to be added to create an axi-symmetrical weight distribution. Overall this leads to a smaller and lighter design of the rotating optical assembly 100. Also, in combination with the smaller motor 140, to reduce the size of scanning mirror 130, mirror 130 is located in close proximity to the lens of the camera, to keep the size of mirror 130 as small as possible.
As shown with respect to
As explained above, instead of instantaneously or very rapidly snapping back the scanning mirror 130 to a start position with start angle α1 for the scanning for every image integration cycle, relative angular position α of scanning mirror 130 is moved with an approximation of a sawtooth or triangular signal that is composed of a fundamental sine wave and additional higher order harmonics. An example of an definition of such waveform is given below with respect to
With respect to
b and 2c, show the positions of camera 110 and mirror 130 while the image sensor 114 is acquiring a single image, while
e and 2f shows positions of the camera 110 and mirror 130 after the first image 160 has been captured by the pixels of image sensor 114 of camera 110, and preferably, the data of image sensor 114 is being read out, and no new image is acquired yet, because mirror 130 has not yet been brought back to a scanning position. The relative angular position α of mirror is increased again from α4 to α5 and α6, to bring the mirror continuously back to the maximal relative angular position α7.
With respect to
Also, after relative angular position α has reached its maximal value with α7, the time period T1 for image integration and capture does not just start yet and relative angular position α is decreased first to avoid any abrupt changes in the relative angular position. With increasing time, absolute angular position γ of disk 180 and camera 110 linearly increases, showing the constant rotation of camera 110. As can be seen from the waveform representing relative angular position α, mirror 130 is not instantaneously snapped back to initial angle position α1, but the relative angular velocity α follows a special waveform that allows to reduce the torque the second motor 140 has to provide for positioning mirror 130. From these waveforms it can also be seen that angular velocity ω of mirror 130 in time period T1 is approximately negative angular velocity −Ω of disk 180 within a certaing tolerance value of +ΔΩ/2 and −ΔΩ/2 actuated by second motor 140, and angular acceleration dΩ/dt of mirror never exceeds dω/dtmax.
Accordingly, for implementing the above described waveforms for relative angular position α, instead of using a sawtooth or pure triangular waveform as a set value for the relative angular position α that first linearly decreases from a maximal value to a minimal value and then jumps back instantaneously to its maximal value, it is possible to use a periodic waveform that is based on sine-waveforms that approximate an ideal triangular waveform to a certain degree. By using such a waveform, the frequency content of the waveform can be limited to lower-order harmonics. The waveform can be described by the following mathematical equation:
in which t is the time, m is the waveform mode selector that can take any positive Integer value, f is the frequency of the waveform that will correspond to frame rate f of camera 110, and C[i] represents a set of coefficients that are determined below. To determine the values of C[i] a set of m equations is generated that is represented by the first m odd derivatives of α(t). The first derivative is set to be equal to 1, and all higher derivatives are set to zero.
The resulting equations are all linear in C[i] with constant coefficients that are straightforward to solve. With a value 1 for m we receive a sine-waveform with the frequency f. In the limit that m becomes infinite the waveform takes the shape of a perfect triangular waveform. For values of m between 1 and infinite a family of waveforms are received that can be used as a set value for second motor 140 that do not produce any overshoot over the triangle waveform as an envelope. The set of C[i] values, also called Murray numbers, for m=1 and m=7 and frequency f=1 are as follows, where the sets are represented in rows, starting with i=1:
A graphical representation of one period of the first i=20 modes is represented in
While the invention has been described with respect to specific embodiments for complete and clear disclosures, the appended claims are not to be thus limited but are to be construed as embodying all modifications and alternative constructions that may occur to one of ordinary skill in the art which fairly fall within the basic teachings here set forth.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/US2013/021222 | 1/11/2013 | WO | 00 | 6/17/2014 |
Number | Date | Country | |
---|---|---|---|
61586432 | Jan 2012 | US |