Patent Application
20140313325
Hyperspectral cameras are capable of capturing hyperspectral image frames, or datacubes at video frame rates. These cameras acquire high spatial and spectral resolution imagery. In combination with techniques relating to computer vision and spectral analysis, operators of hyperspectral cameras have engaged in surveillance applications relating to detection, tracking and identification of imaged objects.
One aspect of the invention relates to a method of improving a spectral reflectance profile of an object using a hyperspectral imaging device. The method comprises obtaining a series of hyperspectral images of the object wherein there is relative motion between the object and the hyperspectral imaging device; determining at least one parameter of the relative motion; mapping the at least one parameter to determine an orientation of the object in each hyperspectral image in the series; identifying at least two spatial portions of the object in each hyperspectral image in the series; assigning a spectral signature to each spatial portion; and generating a multi-dimensional spectral reflectance profile from the orientation, the at least two spatial portions, and the spectral signatures.
In the drawings:
In the background and the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the technology described herein. It will be evident to one skilled in the art, however, that the exemplary embodiments may be practiced without these specific details. In other instances, structures and devices are shown in diagram form in order to facilitate description of the exemplary embodiments.
The exemplary embodiments are described with reference to the drawings. These drawings illustrate certain details of specific embodiments that implement a module, method, or computer program product described herein. However, the drawings should not be construed as imposing any limitations that may be present in the drawings. The method and computer program product may be provided on any machine-readable media for accomplishing their operations. The embodiments may be implemented using an existing computer processor, or by a special purpose computer processor incorporated for this or another purpose, or by a hardwired system.
As noted above, embodiments described herein may include a computer program product comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media, which can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of machine-executable instructions or data structures and that can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or another communication connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such a connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions comprise, for example, instructions and data, which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.
Embodiments will be described in the general context of method steps that may be implemented in one embodiment by a program product including machine-executable instructions, such as program codes, for example, in the form of program modules executed by machines in networked environments. Generally, program modules include routines, programs, objects, components, data structures, etc. that have the technical effect of performing particular tasks or implement particular abstract data types. Machine-executable instructions, associated data structures, and program modules represent examples of program codes for executing steps of the method disclosed herein. The particular sequence of such executable instructions or associated data structures represent examples of corresponding acts for implementing the functions described in such steps.
Embodiments may be practiced in a networked environment using logical connections to one or more remote computers having processors. Logical connections may include a local area network (LAN) and a wide area network (WAN) that are presented here by way of example and not limitation. Such networking environments are commonplace in office-wide or enterprise-wide computer networks, intranets and the internet and may use a wide variety of different communication protocols. Those skilled in the art will appreciate that such network computing environments will typically encompass many types of computer system configurations, including personal computers, hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, and the like.
Embodiments may also be practiced in distributed computing environments where tasks are performed by local and remote processing devices that are linked (either by hardwired links, wireless links, or by a combination of hardwired or wireless links) through a communication network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.
An exemplary system for implementing the overall or portions of the exemplary embodiments might include a general purpose computing device in the form of a computer, including a processing unit, a system memory, and a system bus, that couples various system components including the system memory to the processing unit. The system memory may include read only memory (ROM) and random access memory (RAM). The computer may also include a magnetic hard disk drive for reading from and writing to a magnetic hard disk, a magnetic disk drive for reading from or writing to a removable magnetic disk, and an optical disk drive for reading from or writing to a removable optical disk such as a CD-ROM or other optical media. The drives and their associated machine-readable media provide nonvolatile storage of machine-executable instructions, data structures, program modules and other data for the computer.
Technical effects of the method disclosed in the embodiments include increasing the utility and performance of remote imaging systems for object detection, tracking and identification. A hyperspectral tracking system implementing a multi-dimensional spectral reflectance profile generated by the method of the current invention may robustly mitigate false positives and false negatives that typically occur during reacquisition or object signature prediction. For example, fusion of the multi-dimensional spectral reflectance profile with spatial tracking techniques will reduce errors in traditional spatial tracking due to occlusions. The multi-dimensional spectral reflectance profile provides a profile of the life characteristics of an imaged object of interest. From this profile, operators of hyperspectral tracking systems may characterize and infer properties of the object including: how the object moves, what the object looks like when it moves, how big the object is, how opposing sides of the object differ, etc.
The hyperspectral imaging device 101 may, for example, be stationary on a mobile platform 200 or movable on a stationary platform 300, or any combination thereof. On a movable platform 200, as shown for example in
Referring again to
At step 104, the processor may determine at least one parameter 105 of relative motion between the object of interest 212, 310 and the hyperspectral imaging device 101. For example, the processor may use data from an onboard sensor positioning system that measures relative and absolute positioning. Example onboard systems may include relative positioning systems like inertial navigation systems in combination with absolute positioning systems like GPS. Along with the onboard positioning data, the processor may ascertain differences in the series of hyperspectral images 103 to infer motion of the object of interest 212, 310 and estimate a range from the hyperspectral imaging device 101 to the object of interest 212, 310. The processor may determine relative motion as rotational (i.e. roll, pitch, yaw) and translational (i.e. x, y, z) changes between the hyperspectral imaging device 101 and the object of interest 212, 310. The processor may parameterize the relative motion with Euler angles and direction vectors. Other parameterizations of the relative motion between the hyperspectral imaging device 101 and the object of interest 212, 310 may apply depending upon the implementation. The processor may map the parameter 105 of the relative motion between the object 212, 310 and the hyperspectral imaging device 101 to determine an orientation 107 of the object 212, 310 in each hyperspectral image in the series at step 106.
Upon determination of an orientation 107 of the object of interest 212, 310 in each of the series of hyperspectral images 103, the processor at step 108 may identify spatial portions 109 of the object of interest 212, 310 in each of the series of hyperspectral images 103. Then, at step 110, the processor may assign a spectral signature 111 to each spatial portion 109 of the object of interest 212, 310 in each of the series of hyperspectral images 103. Based on the assignment of a spectral signature 111 to a spatial portion 109 of the object of interest 212, 310, the processor may generate, at step 112, a multi-dimensional spectral reflectance profile 113. The dimensionality of the spectral reflectance profile 113 is determined by the orientation 107 of the object 212, 310, the spatial portions 109, and the spectral signatures 111 associated with the spatial portions 109. Therefore, the multi-dimensional spectral reflectance profile 113 may describe both the spectral reflectance signatures 111 of an object of interest 212, 310 and the spatial relationships among the spectral reflectance signatures 111 along with a spatial, or geometrical, description of the object.
To illustrate,
Returning to
Upon completion of the method at step 114, the hyperspectral imaging device 101 may reacquire the object of interest 212, 310 in successive series of hyperspectral images 103. The processor may improve the multi-dimensional spectral reflectance profile 113 of the object based upon the successive collections of hyperspectral imagery. While initial passes may result in unobserved orientations of the object, successive passes may begin to fill in the model of the multi-dimensional spectral reflectance profile 113 for the previously unobserved orientations.
Conversely, the processor may improve the multi-dimensional spectral reflectance profile 113 for previously observed orientations. When the processor reacquires an object at a previously observed orientation, the processor may update a previously generated multi-dimensional spectral reflectance profile 113 by weighting the spectral signature 111 based upon the integration time of the hyperspectral image. For example, if a given spatial portion 109 for a given orientation 107 has been previously observed for 0.1 seconds to determine a spectral signature 111 and then an additional measurement is made for 0.2 seconds, the spectral signature 111 for the spatial portion 109 for the orientation 107 in the multi-dimensional spectral reflectance profile 113 may be adjusted to weight the new measurement twice as heavily as the old measurement.
This written description uses examples to disclose the invention, including the best mode, and also to enable any person skilled in the art to practice the invention, including making and using any devices or systems and performing any incorporated methods. The patentable scope of the invention is defined by the claims, and may include other examples that occur to those skilled in the art. Such other examples are intended to be within the scope of the claims if they have structural elements that do not differ from the literal language of the claims, or if they include equivalent structural elements with insubstantial differences from the literal languages of the claims.
