This application is a National Stage Application, filed under 35 U.S.C. § 371, of International Application No. PCT/EP2012/069800, filed Oct. 5, 2012, which claims priority to Great Britain Application No. 1117143.6, filed Oct. 5, 2011, the contents of both of which are hereby incorporated by reference in their entirety.
The present invention relates to a camera. In particular, the present invention relates to a fixed lens panoramic camera that has a substantially three hundred and sixty degree field of view.
Panoramic cameras are designed to provide a wide angle view of a scene. There are two basic types of camera: a rotating head camera, which is physically rotated to allow a panoramic image to be captured, and a fixed lens camera. Rotating head cameras provide a three hundred and sixty degree field of view, but they require a mechanical drive to cause physical rotation of the camera. Consequently, rotating head cameras tend to be large and relatively complex. Fixed lens cameras rely on wide angle lenses to capture images of a scene, which are then stitched together. Fixed lens cameras generally require complex optical arrangements to ensure that a three hundred and sixty degree field of view can be captured.
WO 9750252 describes a fixed lens panoramic camera. The camera has a convex mirror for capturing light of the surrounding panorama, an astigmatic lens for correcting astigmatism, a camera objective lens for converging light to a single point, a field flattening lens for converting polar format images into two-dimensional planar images, and an image capture mechanism for capturing optimized two-dimensional annular images of the surrounding panorama. A problem with this camera is that it is bulky and the optics are relatively complex. This makes it unsuitable for many applications.
According to the present invention, there is provided a camera that has four imaging means, each arranged to capture a different field of view, each field of view being associated with a separate sensor or detector, so that the camera is operable to capture four separate images, wherein each field of view is tilted to capture light from a zenith above the camera. The four lenses are equally positioned around a central axis, so that each contributes around at least a quarter of the scene being imaged. Preferably, the camera has an external volume of around 10 cm3 or less.
By providing four fields of view and associated sensors/detectors, a 360 degree image of a scene can be captured. This can be achieved using simple, robust components. Each sensor or detector may comprise an array of sensors of detectors. Each field of view may be defined by one or more lenses.
A single cradle may be provided for locating and accurately aligning all four of the detectors and their associated lenses. The cradle may be shaped to ensure that each sensor is mounted with the same tilt, for example a 20 degree tilt. Using a single piece mount allows for minimum parts and easy assembly and is extremely rugged and durable.
As an alternative to a single cradle, each imaging means may be physically paired with a sensor, and located within a single element, so that their physical positions are fixed prior to insertion into the camera. This allows multiple imaging means, typically lenses, and sensors to be tested and calibrated as a single assembly calibration step. This allows calibration to be done externally from the camera unit.
According to another aspect of the invention, there is provided a camera that has multiple imaging means each arranged to define a different field of view, a sensor associated with each imaging means, and alignment means for aligning the imaging means and the sensor, wherein the alignment means is adapted to engage with an active surface of the sensor.
The alignment means may have a recess for receiving the sensor, and the recess is shaped to engage with the active surface of the sensor, and provide an opening to allow light from the imaging means to fall on the sensor. The alignment means may comprise an alignment cylinder that is adapted to receive the imaging means at one end and the sensor at the other.
The camera may comprise a case or housing. Each alignment means, lens and sensor may form an integral unit that is fitted in the case or housing.
The alignment means may have a fixing means at one end for fixing the imaging means in place; optionally the fixing means may comprise a thread for mating with a corresponding thread on the imaging means.
According to yet another aspect of the invention, there is provided an optical device for use in a camera, the optical device comprising an imaging means, a sensor and an alignment means, wherein the alignment means are adapted to optically align the imaging means and the sensor and the device forms an integral unit for insertion in the camera.
The alignment means may have a recess for receiving the sensor, and the recess is shaped to engage with an active surface of the sensor, and provide an opening to allow light from the imaging means to fall on the sensor. The alignment means may comprise an alignment cylinder that is adapted to receive the imaging means at one end and the sensor at the other. The alignment means may have a fixing means at one end for fixing the imaging means in place; optionally the fixing means may comprise a thread for mating with a corresponding thread on the imaging means.
According to yet another aspect of the invention there is provided a camera that has multiple imaging means for simultaneously capturing images of multiple fields of view, wherein between the multiple fields of view is at least one blind spot, and an external part of the camera extends into the blind spot. The camera may have at least three fields of view and at least three blind spots and the parts of the camera that extend into the blind spots protect the imaging means and/or act as feet or supports for the camera.
According to still another aspect of the invention, there is provided a laser scanner comprising a laser for scanning a scene and a camera that has multiple fields of view for simultaneously capturing a panoramic image of the scene. The camera may be mounted on the laser scanner. The camera preferably has a 360 degree field of view. The camera may be configured to transfer image data captured to a memory in a scanner unit, optionally wherein the transfer is done via cable, wifi or bluetooth. Means may be provided for automatically stitching together images from the multiple fields of view of the camera to provide a single panoramic image.
According to a still further aspect of the invention, there is provided a camera comprising imaging means for capturing an image; an orientation sensor for determining the orientation of the camera at the time the image is captured, and means for correcting the image captured in the event that the orientation sensor detects that the camera is not horizontally aligned.
The camera may have four imaging means, each arranged to capture a different field of view, each field of view being associated with a separate sensor or detector, so that the camera is operable to capture four separate images. The four imaging means may be equally spaced around a central axis, so that each contributes around at least a quarter of the scene being imaged. Each imaging means may be tilted to capture light from a zenith above the camera.
A single cradle or housing may be provided for locating and aligning all four of the imaging means and their associated detectors. The cradle may be shaped to ensure that each imaging means and each sensor are mounted with the same tilt.
According to yet another aspect of the invention there is provided an imaging system that includes an imager for capturing an image of a scene and a camera as claimed in any of claims 1 to 30 for capturing another image of a scene. Means may be provided for causing simultaneous capture of an image using the imager and the camera. Means may be provided for overlaying the image from the imager and the image from the camera. A display may be provided for displaying the overlaid images. The imager may be a thermal imager or a laser scanner.
Various aspects of the invention will now be described by way of example only and with reference to the following drawings, of which:
The camera 10 has a one-piece housing 14 that has four external recesses 16, each shaped to receive and accurately locate field of view optics, typically one or more lenses. The housing also provides a support 18 for sensors 20 for sensing light from each of the field of view optics. The sensor support 18 is shown in more detail in
Each sensor 20 is paired with one of the lenses 12. Each sensor 20 is titled at the same angle and is centred on the same optical axis as its paired lens 12. In the example shown in
The housing 14 holds the lenses 12 and all internal workings in place. The body 12 may be made from any suitable material, for example solid aluminium. The camera 10 is 10 cm3. The lens portion is 5 cm×10 cm. The single piece housing 14 allows for heat dissipation and provides extremely robust and durable protection against bumps, drops, dust, water and many other harsh environmental in use factors.
Each of the wide angle lenses 12 has a maximum, unimpeded field of view of around 180 degrees, but when in situ in the camera 10, each lens 12 has a more limited usable field of view. In particular, in the camera 10 of
Limiting the usable field of view, in this case to around ninety degrees, means that the best quality parts of the image are used. This in turn means that the image quality is optimised. This is shown in
Each sensor 20 is of a size that is smaller than the image projected from its associated lens 12, so that the sensor 20 is over-scanned. This is in contrast to conventional arrangements where typically only part of the active area of the sensor 20 is used. This is illustrated in
Included in the housing 14 is a power supply 24, for example a re-chargeable battery, control electronics, an on-board processor, a memory and software for receiving signals from the sensors and processing them to provide an image. Processing involves stitching images from each of the four lenses together to provide a single panoramic image. An ethernet connection 28 is also provided to allow internet or remote access. Wifi and/or bluetooth transceivers may also be included to allow wireless communications with the camera. On a top surface of the camera 10 is a screen 30 on which processed images can be viewed. Real time or stored images can be viewed.
The camera case design makes use of the blind spots shown in
The lens/sensor barrel 34 is used to align the sensor plane on all three axes with the lens, and ensure the image is centred on the sensor array. In some embodiments, the lens/barrel 34 is small enough to be inserted into the camera housing 14 through the lens holes. The trailing sensor cables can then be connected from inside the camera 10. The backplane of the lens 12 is mated with the camera body, thus ensuring a flush and accurate fit without the adjustment of the focus extension in the lens/sensor barrel affecting the protrusion of the lens from the body. This flush fit ensures the blind spot camera body lens protectors function within their designed tolerances. The lens/barrel unit 34 is secured in place by attaching the lens to the camera housing 14. The end of the barrel to which the sensor is attached is unsecured and floats relative to the housing 14.
Using the lens/sensor alignment barrel 34 each lens 12 can be paired with a sensor 20 prior to insertion into the camera body. This allows multiple lenses 12 and sensors 20 to be tested and calibrated as a single assembly calibration step to provide accurate live positioning verification prior to final bonding the assembly. This allows the calibration to be done externally of the camera, and fine tuning per lens and sensor.
The camera 10 includes a configuration file for calibrating the camera 10 as a whole. This is determined using results from all four lens/sensor sets. No camera 10 has the same configuration. The configuration file is stored in the camera and used to allow the camera 10 to manipulate image data very quickly. This enables extremely fast output of image data and live video feed.
The camera 10 has at least one tilt sensor (not shown) for sensing its tilt angle. For example, the camera 10 may include a tri-axis tilt sensor, such as an accelerometer, for example the LSM303DLM sensor from ST Microelectronics. This allows the tilt or orientation of the camera 10 relative to the horizontal to be determined. Also included in the camera 10 is image processing software for using the sensed tilt angle to correct image alignment, so that the image presented to the user on screen or saved in the memory is aligned to the horizontal regardless of the orientation of the camera 10.
The tilt information is used by the image processing software in the camera 10 to determine how to stitch the images from each of the lenses together, and how to present the image to the user keeping the horizon of the image relative to the user's view. If tilt sensors are not used and the camera 10 orientation is altered, the user's view will follow the orientation of the camera 10 at that point. Viewing a 360 degree image from the camera 10 of the invention allows a user to look around, above, below, back and forward. This is very confusing if the camera 10 is also moving, causing additional unexpected movement when a user is trying to control the view. Using tilt sensors allows the camera 10 to know which way up it is and its orientation. This information can be used by the image processing software to cancel out changes in the real orientation of the camera 10 from what the user wants to see. This allows a user to experience a stable and predictable view even when the camera 10 is moving in multiple axes.
The camera 10 may be designed to allow it to be mounted upside down. This allows it to be mounted on a pole allowing a large area below to be viewed. Inverting the camera 10 means that Zenith above the camera looks down, and the actual Zenith (previously the Nadir) is missing around 60 degrees. To correct for the inverted position, the tilt sensor can detect its orientation. In the event that the camera 10 detects it is up-side down, the on-board image processing software automatically inverts the processed output images to allow the image to be viewed the correct way round. As far as the user is concerned, the only indication that this has happened would be the position of the blind spot that is normally below the camera. Inversion of the camera causes the blind spot to move from the Nadir in the normal up right position to the Zenith in the inverted camera position.
The camera 10 may be arranged to allow a user to switch the orientation correction functionality off or on, as and when desired. This can be done using any form of user input either using an input on the camera 10 or a control input from a wireless device that is able to communicate with the camera 10 via the wi-fi or ethernet connection. When the orientation correction functionality is on, the image processing software uses the sensed tilt angle to correct image alignment, so that the image stored or presented is aligned to the horizontal regardless of the orientation of the camera. Alternatively, the camera 10 may be operable to allow the user to set the desired orientation (i.e. not necessarily horizontal) in which case all corrections are made relative to the set orientation. When the orientation correction functionality is off, no correction is made for image alignment, so if the camera 10 is up-side down and facing a floor, then the image stored or presented is an image of the floor.
The camera 10 of the present invention can be used in numerous different ways and applications. As one example, the camera can be used with a laser scanner to provide an image of an area being scanned.
Laser scanners are widely used to capture detailed and accurate position information on scenes of interest. Laser scanners have a laser that sends highly directional beams of light to a scene and a detector for receiving light reflected back. The laser is rotated in use so that a wide field of view of a scene can be captured (in most cases the desired field of view is 360 degrees). Laser scanners are commonly used by architects and structural engineers for imaging complex buildings, where highly accurate position information is needed. Typically, the images captured are presented as three dimensional line drawings. However, this makes visualising the real scene difficult. In recent years, attempts have been made to overlay real photographic images with those captured by laser scanners. To do this, a known solution is to mount a single fixed lens camera on the laser scanner mount and rotate this whilst simultaneously capturing images of the scene. The images are then later stitched together to form a complete image. However, in practice, aligning the highly accurate laser scanner images with relatively inaccurate conventional images from a rotating camera 10 can be difficult.
To avoid the camera imaging the scanner, the scanner should ideally be 45 degrees or lower in the FOV of the camera 10. To achieve this for a camera that is 10 cm3, the offset distance is 225 mm. If the camera is 100 mm at its base, this blocks about 25 degrees (full angle) of the Zenith FOV of the laser scanner. Such a large offset distance is not desirable from a parallax point of view, but it allows the camera to operate with all four imagers at the same time.
In the arrangement of
In use, the laser scanner is used to capture the image data. It is then removed from its mount and the camera on its bracket is put in place. A panoramic image of the scene is then captured. An advantage of this arrangement is that the camera 10 can capture an image of the scene at the same level as the laser scanner and there is no need to correct for parallax errors. To accommodate different scanner heights, a range of brackets or mounts may be provided, as shown in
The image captured by the laser scanner and the image captured by the fixed lens camera are overlaid to provide a combined image that has detailed position information and a realistic, lifelike image of the scene. To ensure that the overlay is correct, one or more known targets are used to co-orientate the two horizontally. For example, if the image includes a building with a spire, the spire may be used as the alignment marked. Software for overlaying images in this way is known, for example the image processing applications provided by Point Tools.
The camera of the invention could also be used in conjunction with a thermal imaging camera. In this case, the camera is mounted on top of a thermal imaging unit and the camera processor is connected to a processor in the thermal imager (or alternatively each is controlled by the same external processor), typically via a USB port, so that images captured by the thermal imager and the camera can be automatically transferred and stored. The combined camera and thermal imager arrangement is configured so that when thermal imaging starts, a signal is sent to cause the fixed lens camera to capture a panoramic image. This image is transferred to the thermal imager where it is stored for later use to allow the thermal and visible light images to be compared and if desired overlaid. The image capture rate for the thermal camera and the fixed lens camera is preferably the same so that images are captured simultaneously. As before, ideally the thermal camera should be 45 degrees or lower in the FOV of the camera.
The camera of the invention can be used to capture still or moving, i.e. video, images. The frame rate is defined by the capture rate of the sensors. Ideally, the frame rate would be more than 10 frames per second for good resolution video.
The use of a four lens arrangement in the camera of the present invention provides numerous advantages over other lens arrangements. For example, using three lenses or fewer, as is most commonly done in fixed lens cameras, means that the same image is captured, but at lower resolution and clarity because all of the circular image has to fall on to the square/rectangular sensor. In practice, this means only two thirds of each sensor is in use. This equates to a loss of output resolution of 10 megapixel when 3×10 megapixel sensors used, and a loss of 6.7 megapixel when 2×10 megapixel sensors used. Also, resolution is compromised. Using five lenses or more allows for better resolution, but the downside is power consumption increases; battery life is reduced; the camera has to be relatively large physically; and the data rates increase, which means more software correction is required resulting in slower performance. The advantageous selection of four lenses allows relatively high resolution images to be captured, over a very wide field of view using a camera that is simple, robust and very small.
A skilled person will appreciate that variations of the enclosed arrangement are possible without departing from the invention. For example, although the cradle shown in
Number | Date | Country | Kind |
---|---|---|---|
1117143 | Oct 2011 | GB | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2012/069800 | 10/5/2012 | WO |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2013/050576 | 4/11/2013 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
4994670 | Noble et al. | Feb 1991 | A |
5657073 | Henley | Aug 1997 | A |
5900909 | Parulski et al. | May 1999 | A |
20040027451 | Baker | Feb 2004 | A1 |
20040042782 | Yoshikawa et al. | Mar 2004 | A1 |
20040246333 | Steuart | Dec 2004 | A1 |
20060197867 | Johnson et al. | Sep 2006 | A1 |
20100045774 | Len et al. | Feb 2010 | A1 |
20100097443 | Lablans | Apr 2010 | A1 |
20100110259 | Hsu | May 2010 | A1 |
20100182396 | Kroepfl | Jul 2010 | A1 |
20100245539 | Lin | Sep 2010 | A1 |
20100245601 | Kato | Sep 2010 | A1 |
20110149094 | Chen et al. | Jun 2011 | A1 |
20120162360 | Ohtomo et al. | Jun 2012 | A1 |
Number | Date | Country |
---|---|---|
2518995 | Oct 2012 | EP |
2006-033513 | Feb 2006 | JP |
2009-294523 | Dec 2009 | JP |
2010-276977 | Dec 2010 | JP |
WO 1997050252 | Dec 1997 | WO |
WO 2009022178 | Feb 2009 | WO |
WO 2010121506 | Oct 2010 | WO |
WO 2011040239 | Apr 2011 | WO |
WO 2011078244 | Jun 2011 | WO |
Entry |
---|
International Searching Authority, International Search Report and Written Opinion for International Application No. PCT/EP2012/069800, dated Jun. 12, 2013, 14 pages, European Patent Office, The Netherlands. |
Stmicroelectronics, LSM303DLM Sensor Module: 3-Axis Accelerometer and 3-Axis Magnetometer, Apr. 2011, Doc ID 018725 Rev 1, 38 pages, STMicroelectronics NV. |
Intellectual Property Office, Search Report under Section 17(5) for Application No. GB1117143.6, dated Nov. 25, 2011, 10 pages, UK. |
European Patent Office, Communication pursuant to Article 94(3) EPC for Application No. 12778984.0, dated Jun. 6, 2017, 5 pages, Germany. |
Number | Date | Country | |
---|---|---|---|
20140253678 A1 | Sep 2014 | US |