Patent Application
20220301225
It has become a common place for devices to embed a camera with lens. For example, laptop computers, smartphones, and tablet PCs, include cameras. A physical alignment between the lens and an on-board image sensor is important for capturing images with minimal distortion. An offset of an optical center (i.e., pixel position where the lens optical axis intersects with image sensor) from an image sensor pointing reference may cause captured image distortion and a lack of focus. There are varieties of factors that cause the offset: a lack of uniformity of materials of one or more lenses in a lens barrel, misalignment among lenses inside the lens barrel, misalignment between the lens barrel and the on-board image sensor, and the like.
Calibrating the camera for attaining the physical alignment between the lens barrel and the on-board image sensor takes place during an assembly of the camera and a device. Image processing software on the device reduces distortions when taking pictures. The calibration at the time of camera assembly allows for a higher level of control and a larger magnitude of changes in a physical alignment, as compared to a calibration after assembling the device. On-board processing may reduce distortion in a captured image to an extent that is more limited than adjusting a physical alignment of the parts. Adjusting alignment during assembly with accuracy at a uniform quality level on individual cameras is more costly than software-based adjustment. The adjusting alignment during assembly incurs operational costs in use of fixtures for physically aligning parts. Repeatability becomes an issue in calibrating cameras in bulk. Thus, developing a technology that meets these needs while minimizing trade-offs would be desirable.
It is with respect to these and other general considerations that the aspects disclosed herein have been made. Also, although relatively specific problems may be discussed, it should be understood that the examples should not be limited to solving the specific problems identified in the background or elsewhere in this disclosure.
According to the present disclosure, the above and other issues are resolved by use of a kinematic mount to calibrate a physical alignment between lens and an on-board image sensor for assembling a camera. Additionally or alternatively, the present disclosure measures an offset between a lens optical axis, or an optical center (i.e., a pixel position where the lens optical axis intersects with image sensor) and an image pointing reference (i.e., desired position where the lens optical axis intersects with image sensor) of the assembled camera part for de-warping captured images. In aspects, a lens barrel may enclose a set of lenses as a package. The lens barrel attaches to the on-board image sensor. A physical alignment may take place when affixing the lens barrel and lens holder with a board with the on-board image sensor.
The disclosed technology includes a kinematic mount. The term “a kinematic mount” may also be referred to as a lens barrel mount, an alignment station, and a kinematic alignment fixture. Use of the kinematic mount allows determining with accuracy a position and an orientation of the lens barrel relative to a fixed and/or a calibrated target scene, such that the on-board image sensor may be aligned relative to a lens barrel to reference a target camera pointing accuracy.
The kinematic mount receives the on-board image sensor and the lens barrel for calibrating a relative position and orientation of the on-board image sensor and the lens barrel. The calibration includes determining the position of the on-board image sensor, the position of the lens barrel by measuring an offset of the center of a test image, of calibrated target scene, with a symmetrically flipped image of the test image. The camera is assembled by affixing the on-board image sensor and the lens barrel at the determined position and orientation.
The kinematic mount further receives the camera for determining the optical center of the camera, which represents a total pixel offset value for the camera between the desired pixel pointing position on and image sensor (e.g., pixel position of image sensor center, or other targeted pixel position) and the pixel position where the optical axis intersects the image sensor plane. The kinematic mount determines offset values, or optical center, from the desired pixel pointing position on an image sensor to the lens optical axis, by determining the average offset between desired pixel pointing position and the target features in image at two opposing pair of orientations (e.g., 0 degrees and 180 degrees) of the camera. The disclosed technology generates the total offset value based on the pair of offset values (e.g., an average of the opposing pair of offset values). Errors in the calibrated position of scene target features are substantially canceled out (i.e. “nulled out), providing the optical center data at an accuracy higher than each measurement alone. The generated optical center data may be stored (e.g., in an onboard EEPROM) so as to be accessed by the firmware of the device. An image processor of the device uses the total offset values for reducing distortions of images while the device is in use. The image processor de-warps captured images.
This Summary is provided to introduce a selection of concepts in a simplified form, which is further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Additional aspects, features, and/or advantages of examples will be set forth in part in the following description and, in part, will be apparent from the description, or may be learned by practice of the disclosure.
Non-limiting and non-exhaustive examples are described with reference to the following figures.
Various aspects of the disclosure are described more fully below with reference to the accompanying drawings, which from a part hereof, and which show specific example aspects. However, different aspects of the disclosure may be implemented in many different ways and should not be construed as limited to the aspects set forth herein; rather, these aspects are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the aspects to those skilled in the art. Aspects may be practiced as methods, systems or devices. Accordingly, aspects may take the form of a hardware implementation, an entirely software implementation or an implementation combining software and hardware aspects. The following detailed description is, therefore, not to be taken in a limiting sense.
A quality of images taken by a device with a camera depends on alignment of an optical axis and an image sensor pointing reference, and/or optical center, associated with various parts of the camera. The parts of the camera embedded on the device may include an on-board image sensor and a lens barrel that encloses a prepackaged set of lenses. In aspects, capturing an image using the device with the embedded camera takes place as follows. First, a light enters the lens barrel. The light passes through a series of lenses, and forms an image on the on-board image sensor of the device. An image processing program then generates a frame of image data based on data from the on-board image sensor. There are varieties of factors that may cause misalignment between the lens optical axis and the optical center. The misalignment results in distortions in a captured image. For example, a physical positioning and an orientation of the on-board image sensor and the lens barrel affect the alignment. A misalignment among lenses within the prepackaged lens barrel may exist. A distortion and non-uniformity of lens materials may misalign the lens optical axis and the image sensor pointing reference. Yet another cause for misalignment may include an offset of one or more lenses inside the prepackaged lens barrel.
The present application solves these issues by providing a system and a method that efficiently calibrate the position and the orientation of the on-board image sensor and the lens barrel when affixing the two parts during manufacturing. Additionally, or alternatively, the disclosed technology measures overall offset values of the lens optical axis from the image sensor pointing references with a higher degree of accuracy than the calibration. An image processor on-board the device uses the overall offset values to correct distortion on captured images by de-warping the captured images. The present application enables balancing competing demands of accuracy and efficiency.
As discussed in more detail below, the present disclosure relates to calibrating a camera and measuring an offset for de-warping. In particular, the calibration and the measurement use a kinematic mount to attain a level of accuracy in measurement by maintaining the on-board image sensor and the lens barrel at a predetermined position and orientation. The kinematic mount further provides repeatable operations in positioning and orienting the on-board image sensor and the lens barrel at a predetermined setting.
The device 102 includes a camera 104, firmware 106, and an image processor 108. The camera 104 includes a lens barrel 110 and an on-board image sensor 112. The lens barrel 110 may include one or more lenses prepackaged and calibrated. The on-board image sensor 112 senses incoming light for capturing an image. Firmware 106 manages processing that occurs on the device 102. Additionally, or alternatively, the firmware 106 receives offset values of the camera 104 and sets the values for the image processor 108 to correct distortion of captured images by de-warping. The image processor 108 processes a captured image in various manners including de-warping the captured image to reduce distortion found in the captured image.
The kinematic mount 130 (a camera assembler) mounts the lens barrel 110 and the on-board image sensor 112 of the camera 104 to serve the following two purposes. The first purpose is to adjust a position and an orientation for assembling the camera 104 by affixing the lens barrel 110 and the on-board image sensor 112. The second purpose is to mount the affixed pieces as the camera 104 to measure and determine an offset between the lens optical axis and the image sensor pointing reference of the camera 104. The kinematic mount 130 may also be referred to as a lens barrel mount or a kinematic alignment fixture. The kinematic mount 130 provides for fixing specific degrees of freedom of kinematics while allowing changes in other degrees of freedom.
In aspects, the kinematic mount 130 provides two of six degrees of freedom as are shown in
Turning back to
In aspects, a misalignment of less than 1 degree may be unnoticeable by human eyes. A cumulative stack error caused by a combination of various causes of the offsets may be greater than 1 degree. In some aspects, use of the kinematic mount 130 for mounting the lens barrel on the on-board image sensor attains the offset error of less than 0.5 degree. For some imaging applications (e.g., generating a panorama photo by image stitching multiple images), the offset of less than 0.05 degree (i.e., 50 milli-degrees) may be desirable for the stitching of the multiple images to be unnoticeable in the panorama photo.
The lens aligner 140 includes an on-board image sensor aligner 142, a lens barrel aligner 144, and a camera assembler 146. The on-board image sensor aligner 142 aligns the on-board image sensor 112 with the lens image sensor pointing reference positions (e.g., the center or offset pixel position) of the on-board image sensor 112. The lens barrel aligner aligns the lens barrel 110 with the on-board image sensor 112. The lens barrel aligner 144 aligns the lens barrel 110 by adjusting the position and the orientation of the on-board image sensor 112. In aspects, the lens aligner 140 uses image feedback for detecting a rotation error. The lens aligner 140 further corrects a rotational misalignment. In some aspects, the lens aligner 140 captures an image of a horizontally-aligned target, and extracts (e.g., crop) a region of the image containing the lens optical axis at the center of the image. The lens aligner 140 generates a symmetrical (i.e., “flipped) image along the X-axis. The lens aligner 140 determines an absolute value of difference of the flipped image with the cropped image data. Then, the lens aligner 140 aligns the position of the lens barrel by moving the on-board image sensor located underneath the lens barrel at a position and an orientation where the cropped image and the flipped image match.
The camera assembler 146 affixes the on-board image sensor 112 and the lens barrel 110 at the aligned position and the orientation as determined by the combination of the on-board image sensor aligner 142 and the lens barrel aligner 144.
The offset measurer 150 measures an offset between the lens optical axis and the image sensor pointing reference. For example, the offset measurer 150 measures a total offset of the assembled camera and stores the total offset in a memory of the device 102 through the firmware 106 of the device. The device uses an image processor 108 based on the total offset to correct distortions in captured images by de-warping the captured images. In aspects, the offset measurer 150 measures an offset value in the accuracy range of 20 milli-degrees for a subpixel alignment.
The offset measurer 150 includes a per-orientation offset measurer 152, a total offset determiner 154, and a total offset uploader 156. The per-orientation offset measurer 152 measures an offset from the lens optical axis to the image sensor pointing reference at a predetermined position and orientation (e.g., the yaw rotation at 0-degree or 180 degree). In aspects, the per-orientation offset measurer 152 measures the offset for the camera 104 installed in the kinematic mount 130. In aspects, the kinematic mount 130 fixes the five out of the six degrees of freedom of kinematics. The kinematic mount 130 enables rotating the camera 104 on the z-axis (i.e., yaw) at a predetermined degree (e.g., 0-degree or 180 degrees).
The total offset determiner 154 determines a total offset value between the lens optical axis and the image sensor pointing reference based on one or more measured offset values at varying positions and orientations. For example, the total offset determiner 154 may determine the total offset value based on the offset value measured at 0-degrees and another offset value measured at 180-degrees in yaw. In aspects, the total offset determiner 154 may determine the total offset value by determining an average value of the two offset values. For example, when an offset value at 0-degrees is (x, z)=(+6, +3) and the value at 180-degrees is (+2, +1), the total offset value is (+4, +2). In aspects, the total offset value represents an offset of the lens optical axis from the image sensor pointing reference in the camera 104 in the device. For example, the image processor de-warps the image data and corrects distortions of the captured image. As a result, the image processor provides a sub-pixel alignment of captured image.
The total offset uploader 156 uploads the determined total offset value to the firmware 106 of the device 102. When the device 102 is in use, the image processor 108 of the device 102 reads and uses the total offset value to de-warp captured images to remove distortion in the captured images.
As will be appreciated, the various methods, devices, applications, features, etc., described with respect to
In
In
In aspects, the total offset represents an offset from the image sensor pointing reference and the lens optical axis. The total offset may be stored in the firmware of the device (e.g., onboard EEPROM) for the on-board image processor to correct distortions in captured images by de-warping.
Following start operation 402, the method 400 begins with align operation 404, which aligns an on-board image sensor. In aspects, the pre-calibrate operation 404 takes place before affixing the on-board image sensor with the lens barrel in assembling the camera. In some aspects, the align operation 404 receives the on-board image sensor in an adjustable mount and positions and orients the on-board image sensor where the lens, or other desired image sensor pointing reference position, passes through the center of the on-board image sensor at a perpendicular angle. In some aspects, the align operation 404 pre-calibrates the kinematic mount as a target setup. The pre-calibration includes aligning the image sensor pointing reference position at a z position from a target and at an x, y position of an optical path to the target (e.g., a laser reference beam aligned to optical axis of target setup).
In aspects, the pre-calibrate operation 404 includes determining a position such that the lens is held in position relative to an object scene containing an imaging target feature at a prescribed z distance. The optical axis as referenced by the lens barrel is placed such that the 0-degrees field angle of lens points to and is aligned with the target feature in a scene. In addition, the adjustable mount holding the image sensor board is placed in a nominal starting position so that the sensor may be aligned with respect to the lens for optimum z focus and x, y position once the pre-calibrate operation 404 completes and aligns the parts. Accordingly, for the module alignment/integration (e.g., sensor alignment to kinematically-mounted lens), a lens is placed in kinematic mount referencing the lens barrel, the sensor is aligned for a desirable focus position z, tip, tilt and minimum x,y deviation of optical center from image sensor pointing reference position. In this case, lens is mounted in the kinematic mount with respect to target features in scene, and image sensor is then aligned to the lens to achieve the desirable focus and minimum optical center error.
In some other aspects, the pre-calibrate operation 404 repeats the pre-calibrate operation until the 0-degrees field angle points to scene target features in the scene.
Calibrate operation 406 calibrates a camera. In aspects, the calibrate operation includes aligning the on-board image sensor and the lens barrel. In aspects, the align operation 406 aligns the lens barrel with the lens optical axis by moving the on-board image sensor in the kinematic mount. The align operation 406 may move the on-board image sensor for an accurate alignment because the on-board image sensor is smaller and lighter than the lens barrel. In aspects, the calibrate operation 406 forms a calibration processing of the camera, as marked by “A.”
Affix operation 408 affixes the on-board image sensor and the lens barrel at the aligned position and orientation. In aspects, the affix operation 408 affixes the on-board image sensor with respect to the lens barrel. The lens barrel includes a lens holder and a gap between the lens holder and the sensor board. The affix operation 408 assembles a camera part of the device.
Calibrate operation 410 calibrates the measurement setup. The measurement setup may include the kinematic mount (e.g., the kinematic mount 206 in
Measure operation 412 measures an offset value from the lens optical axis to the optical center at the orientation of 0-degrees. In aspects, the measure operation 412 measures a location of the image sensor pointing reference by capturing a pattern image data and measuring a distance from the lens optical axis to a predetermined location within the pattern image data.
Measure operation 414 measures an offset value the lens optical axis to the optical center at the orientation of 180 degrees. In aspects, the measure operation 412 measures a location of the image sensor pointing reference by capturing a pattern image data and measuring a distance from the lens optical axis to the predetermined location within the pattern image data. Accordingly, the measured offset values from the measure operation 412 and the measure operation 414 form an opposing pair.
Determine operation 416 determines a total offset value. The total offset value may be an average of the measured offset data of the opposing pair. By taking an average, a measurement error of the set up using the kinematic mount may cancel out because the two data are the opposing pair.
Upload operation 418 uploads the determined total offset value to the firmware of the device that embeds the camera. The firmware may store the total offset value for de-warping captured images when the device is in use. In aspects, the device processes captured images, including de-warping the captured images based on the offset value stored in the firmware of the device. By de-warping the captured images based on the offset value, the device eliminates distortions in the captured image. In some aspects, a combination of the measure operation 412, the measure operation 414, the determine operation 416, and the upload operation 418 forms measuring and providing an offset value for correcting distortions of captured images. The combination is marked as “B.”
De-warp operation 420 de-warps captured images and reduces distortions in the captured images. The de-warp operation 420 may include transforming the captured images. In aspects, method 400 may end with end operation 422.
As should be appreciated, operations 402-422 are described for purposes of illustrating the present methods and systems and are not intended to limit the disclosure to a particular sequence of steps, e.g., steps may be performed in different order, additional steps may be performed, and disclosed steps may be excluded without departing from the present disclosure.
Following start operation 502, the method 500 begins with receive operation 504, which receives the on-board image sensor on the kinematic mount. In aspects, the kinematic mount is a part of a fixture that limits predetermined degrees of freedom on the installed on-board image sensor. The kinematic mount (i.e., the adjustable mount) holds the on-board image sensor in proximity to the lens at nominal focus position.
Receive operation 506 receives the lens barrel on the pre-calibrated kinematic mount. In aspects, the receive operation receives the lens barrel at a predetermined position inside the kinematic mount. The kinematic mount provides repeatability in placing the received lens barrel in predetermined position and orientation by controlling degrees of freedom upon the received lens barrel.
Align operation 508 aligns the on-board image sensor to the lens. The alignment positions the lens at z, tip, tilt and minimum x, y deviation of optical center from image sensor pointing reference position.
Receive operation 510 receives a test image through the lens barrel on the on-board image sensor. The test image provides a set of points on the test image as targets for determining offsets between the lens optical axis and the optical center.
Extract operation 510 crops the captured test image for determining offset values. In aspects, the cropped test image contains an optic axis at the center of the test image.
Generate operation 514 generates an X-Y flip copy of the cropped region In aspects, the X-Y flip copy is an mirror image where the image is flipped along the X axis toward the Y axis for determining the offset values.
Determine operation 516 determines the offset values based on difference between cropped region and the X-Y flip copy. In aspects, the determine operation 516 includes superimposing the cropped region and the X-Y flip copy for identifying an offset of pixels between the center of the cropped region and the center of the X-Y flip copy. In some aspects, the type of features and size of area of the chart may be used to indicate how well nulled is the absolute value of the difference between the cropped image (i.e., a region of interest, ROI) and X, Y-flipped image. For course offset errors, the features in the combined output image appear expanded in width, whereas fine offset errors result in tighter width of the feature region. Further, for case where chart features include radial spatial frequencies (e.g., a star chart), the feature area of the combined output image may contain higher or lower spatial frequency content dependent on offset error. For minimal offset approaching zero offset, the output may appear “nulled” (e.g., all dark, all white, or all gray), depending on the number of spokes of star chart and rotational position of the star chart. In some aspects, the aliased pattern and shape indicates a direct correlation with offset and direction, thus offset position. By using a feature imaged to have a large area of pixels, sub-pixel accuracy may be achieved by leveraging the full area of pixels representing the chart area.
Align operation 518 aligns the position of the lens barrel to “null” (e.g., canceling out) the target between the cropped region and the X-Y flip copy. In aspects, method 500 may end with end operation 520.
As should be appreciated, operations 502-520 are described for purposes of illustrating the present methods and systems and are not intended to limit the disclosure to a particular sequence of steps, e.g., steps may be performed in different order, additional steps may be performed, and disclosed steps may be excluded without departing from the present disclosure.
Following start operation 602, the method 600 begins with receive operation 606, which receives the camera on the kinematic mount at the 0-degrees orientation. In aspects, the kinematic mount is a part of a fixture that limits predetermined degrees of freedom on the installed on-board image sensor.
Determine operation 608 determines an offset value from the lens optical axis to the image sensor pointing reference when the camera is at the 0-degrees orientation. The determine operation 608 determines the offset value by taking an image of a target pattern with a reference point. In aspects, the offset value is a distance from the lens optical axis, which intersects image sensor at the optical center, to the image sensor pointing reference, which depending on application may be defined (e.g., center or offset position of the image) as a reference point on the image.
Receive operation 610 receives the camera on the kinematic mount at the 180-degrees orientation. In aspects, the kinematic mount controls the degree of freedom of the camera installed in the kinematic mount. For example, the kinematic mount includes a camera align place with a set of pins that fixes an orientation of the camera at either 0 degrees or at 180 degrees. The kinematic mount restricts all other movement of the camera.
Determine operation 612 determines an offset value from the lens optical axis to the image sensor pointing reference when the camera is at the 180-degrees orientation. As detailed above for determining the offset value for the 0-degrees orientation, the determine operation 612 determines the offset value by taking an image of a target pattern with a reference point. The offset value is a distance from the lens optical axis, which intersects image sensor at the optical center, to the image sensor pointing reference, which depending on application may be defined (e.g., center or offset position of the image) as a reference point on the image.
Determine operation 614 determines a total offset value from the lens optical axis to the image sensor pointing reference based on the opposing pair of offset values with orientations at 0-degrees and 180 degrees. In aspects, the two orientations may be at other azimuth, or clocking, angles that are 180 degrees apart. Further, more than one opposing pair may be used in determining optical center offset values (e.g., 0-degrees and 180-degrees, 90-degrees and 270-degrees, 45-degrees and 235-degrees, etc.). The average of these actual offset values determined from opposing pairs comprises the final optical center offset values to be used or stored. When the opposing pair is 180 degrees apart, the determine operation 614 determines the total offset by generating an average of the opposing pair. The determine operation 614 cancels out (i.e., “nullify”) errors that are rotationally symmetrical. The rotationally symmetrical errors may include an error in setting up the camera in the kinetic mount.
Upload operation 616 uploads the total offset value to the firmware of the device. The firmware may store the total offset value in a memory of the client. The device may use the total offset value among parameters for an image processor to de-warp captured images. The de-warping of the captured images based on the total offset value reduces distortions of the captured images. In aspects, method 600 may end with end operation 618.
As should be appreciated, operations 606-616 are described for purposes of illustrating the present methods and systems and are not intended to limit the disclosure to a particular sequence of steps, e.g., steps may be performed in different order, additional steps may be performed, and disclosed steps may be excluded without departing from the present disclosure.
As stated above, a number of program tools and data files may be stored in the system memory 704. While executing on the at least one processing unit 702, the program tools 706 (e.g., an application 720) may perform processes including, but not limited to, the aspects, as described herein. The application 720 includes an on-board image sensor aligner 722, a lens aligner 724, a lens affixer 726, a per-orientation offset measurer 728, a total offset generator 730, and a total offset uploader 732, as described in more detail with regard to
Furthermore, aspects of the disclosure may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, aspects of the disclosure may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in
The computing device 700 may also have one or more input device(s) 712, such as a keyboard, a mouse, a pen, a sound or voice input device, a touch or swipe input device, etc. The output device(s) 714 such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. The computing device 700 may include one or more communication connections 716 allowing communications with other computing devices 750. Examples of suitable communication connections 716 include, but are not limited to, radio frequency (RF) transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.
The term computer readable media as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program tools. The system memory 704, the removable storage device 709, and the non-removable storage device 710 are all computer storage media examples (e.g., memory storage). Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by the computing device 700. Any such computer storage media may be part of the computing device 700. Computer storage media does not include a carrier wave or other propagated or modulated data signal.
Communication media may be embodied by computer readable instructions, data structures, program tools, or other data in a modulated data signal, such as a carrier wave or other transport mechanism, and includes any information delivery media. The term “modulated data signal” may describe a signal that has one or more characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media may include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, radio frequency (RF), infrared, and other wireless media.
One or more application programs 866 may be loaded into the memory 862 and run on or in association with the operating system 864. Examples of the application programs include phone dialer programs, e-mail programs, information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. The system 802 also includes a non-volatile storage area 868 within the memory 862. The non-volatile storage area 868 may be used to store persistent information that should not be lost if the system 802 is powered down. The application programs 866 may use and store information in the non-volatile storage area 868, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on the system 802 and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area 868 synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into the memory 862 and run on the mobile computing device 800 described herein.
The system 802 has a power supply 870, which may be implemented as one or more batteries. The power supply 870 might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.
The system 802 may also include a radio interface layer 872 that performs the function of transmitting and receiving radio frequency communications. The radio interface layer 872 facilitates wireless connectivity between the system 802 and the “outside world,” via a communications carrier or service provider. Transmissions to and from the radio interface layer 872 are conducted under control of the operating system 864. In other words, communications received by the radio interface layer 872 may be disseminated to the application programs 866 via the operating system 864, and vice versa.
The visual indicator 820 (e.g., LED) may be used to provide visual notifications, and/or an audio interface 874 may be used for producing audible notifications via the audio transducer 825. In the illustrated configuration, the visual indicator 820 is a light emitting diode (LED) and the audio transducer 825 is a speaker. These devices may be directly coupled to the power supply 870 so that when activated, they remain on for a duration dictated by the notification mechanism even though the processor 860 and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. The audio interface 874 is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to the audio transducer 825, the audio interface 874 may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with aspects of the present disclosure, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below. The system 802 may further include a video interface 876 that enables an operation of an on-board camera 830 to record still images, video stream, and the like.
A mobile computing device 800 implementing the system 802 may have additional features or functionality. For example, the mobile computing device 800 may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated in
Data/information generated or captured by the mobile computing device 800 and stored via the system 802 may be stored locally on the mobile computing device 800, as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio interface layer 872 or via a wired connection between the mobile computing device 800 and a separate computing device associated with the mobile computing device 800, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information may be accessed via the mobile computing device 800 via the radio interface layer 872 or via a distributed computing network. Similarly, such data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.
The description and illustration of one or more aspects provided in this application are not intended to limit or restrict the scope of the disclosure as claimed in any way. The claimed disclosure should not be construed as being limited to any aspect, for example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Having been provided with the description and illustration of the present application, one skilled in the art may envision variations, modifications, and alternate aspects falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed disclosure.
As will be understood from the foregoing disclosure the present disclosure relates to a computer-implemented method of reducing distortions in mage data. The method comprises determining, using a kinematic mount for an alignment, a physical offset value associated with a position and an orientation of an on-board image sensor and a lens, assembling a camera, wherein the camera includes the on-board image sensor and the lens affixed based on the physical offset value, determining a pair of offset values, wherein each of the pair of offset values corresponds to an offset as an optical center from an optical axis and an image sensor pointing reference of the camera, generating, based on the determined pair of offset values, a total offset value, and causing a device to store the total offset value for reducing the distortions of image data. Additionally, the method includes receiving the on-board image sensor in the kinematic mount, aligning the on-board image sensor using a reference light beam, receiving a lens barrel in the kinematic mount, wherein the lens barrel includes the lens. generating, based on incoming light through the lens barrel, an image on the on-board image sensor, extracting a region of the image, wherein the region includes an lens optical axis, generating a flip copy of the extracted region, and determining, based on the extracted region and the flip copy, the physical offset value. The kinematic mount controls one or more degrees of freedom, wherein the one or more degrees of freedom include at least one of: a surge position along the x-axis, a sway position along the y-axis, a heave position along the z-axis, a roll orientation around the x-axis, a pitch orientation around the y-axis, or a yaw orientation around the z-axis. The method further includes receiving, by the kinematic mount, the camera at a first orientation, determining, based on the camera received in the kinematic mount at the first orientation, a first offset value, receiving, by the kinematic mount, the camera at a second orientation, determining, based on the camera received in the kinematic mount at the second orientation, a second offset value, wherein the second orientation is 180 degrees apart from the first orientation in yaw, and generating, based on the first offset value and the second offset value, the pair of offset values. The first orientation corresponds to zero degrees in the yaw, and wherein the second orientation corresponds to 180 degrees in the yaw, and wherein the total offset value includes an average value of the pair of offset values. The kinematic mount includes at least one degree of freedom including the yaw. The kinematic mount includes a lens retainer, a camera alignment plate including a plurality of pins for securing the camera in a pair of predetermined opposing orientations, and a cavity configured to receive the camera secured on the camera alignment plate.
Another aspect of the technology relates to a system for determining an offset for reducing distortions of an image. The system includes a processor; and a memory storing computer-executable instructions that when executed by the processor cause the system to: determine, using a kinematic mount for an alignment, a physical offset value associated with a position and an orientation of an on-board image sensor and a lens, assemble a camera, wherein the camera includes the on-board image sensor and the lens affixed based on the physical offset value, determine a pair of offset values, wherein each of the pair of offset values corresponds to an offset as an optical center from an optical axis and an image sensor pointing reference optical center of the camera, generate, based on the determined pair of offset values, a total offset value, and cause a device to store the total offset value for reducing distortions of image data. The computer-executable instructions when executed further causing the system to: receive the on-board image sensor in the kinematic mount, align the on-board image sensor using a reference light beam, receive a lens barrel in the kinematic mount, wherein the lens barrel includes the lens, generate, based on incoming light through the lens barrel, an image on the on-board image sensor, extract a region of the image, wherein the region includes an lens optical axis, generate a flip copy of the extracted region, and determine, based on the extracted region and the flip copy, the physical offset value. The kinematic mount controls one or more degrees of freedom, wherein the one or more degrees of freedom include at least one of: a surge position along the x-axis, a sway position along the y-axis, a heave position along the z-axis, a roll orientation around the x-axis, a pitch orientation around the y-axis, or a yaw orientation around the z-axis. The computer-executable instructions when executed further causing the system to: receive, by the kinematic mount, the camera at a first orientation, determine, based on the camera received in the kinematic mount at the first orientation, a first offset value, receive, by the kinematic mount, the camera at a second orientation, determine, based on the camera received in the kinematic mount at the second orientation, a second offset value, wherein the second orientation is 180 degrees apart from the first orientation in yaw, and generate, based on the first offset value and the second offset value, the pair of offset values. The first orientation corresponds to zero degrees in the yaw, and wherein the second orientation corresponds to 180 degrees in the yaw, and wherein the total offset value includes an average value of the pair of offset values. The kinematic mount includes at least one degree of freedom including the yaw. The kinematic mount includes a lens retainer, a camera alignment plate including a plurality of pins for securing the camera in a pair of predetermined opposing orientations, and a cavity configured to receive the camera secured on the camera alignment plate.
Yet another aspect of the technology relates to a kinematic mount device. The device includes a physical offset determiner configured to determine a physical offset value associated with a position and an orientation of an on-board image sensor and a lens, a camera assembler configured to assemble a camera, wherein the camera includes the on-board image sensor and the lens affixed based on the physical offset value, an offset determiner configured to determine a pair of offset values, wherein each of the pair of offset values corresponds to an offset as an optical center from an optical axis and an image sensor pointing referenceoptical center of the camera, a total offset determiner configured to generate, based on the determined pair of offset values, a total offset value, wherein the total offset determiner is further configured to cause to store the total offset value for reducing distortions of image data. The device further includes an on-board image sensor receiver configured to receive the on-board image sensor on a kinematic mount, an on-board image sensor aligner configured to align the on-board image sensor using a reference light beam, a lens barrel receiver configured to receive a lens barrel in the kinematic mount, wherein the lens barrel includes the lens, an image generator configured to generate, based on incoming light through the lens barrel, an image on the on-board image sensor, a region extractor configured to extract a region of the image, wherein the region includes a lens optical axis, a flip copy generator configured to generate a flip copy of the extracted region, and a physical offset determiner configured to determine, based on the extracted region and the flip copy, the physical offset value. The kinematic mount device controls one or more degrees of freedom, wherein the one or more degrees of freedom include at least one of: a surge position along the x-axis, a sway position along the y-axis, a heave position along the z-axis, a roll orientation around the x-axis, a pitch orientation around the y-axis, or a yaw orientation around the z-axis. The device further includes a first camera receiver configured to receive the camera at a first orientation, a first offset determiner configured to determine, based on the camera received in a kinematic mount at the first orientation, a first offset value, a second camera receiver configured to receive, by the kinematic mount, the camera at a second orientation, a second offset determiner configured to determine, based on the camera received in the kinematic mount at the second orientation, a second offset value, wherein the second orientation is 180 degrees apart from the first orientation in the yaw, and a total offset generator configured to generate, based on the first offset value and the second offset value, the pair of offset values. The first orientation corresponds to zero degrees in the yaw, and wherein the second orientation corresponds to 180 degrees in the yaw orientation, and wherein the total offset value includes an average value of the pair of offset values.
Any of the one or more above aspects in combination with any other of the one or more aspect. Any of the one or more aspects as described herein.
