The present invention relates generally to processing data from a synthetic aperture radar (SAR) imaging system and, more particularly, to a system and method for generating compressed, multi-resolution SAR images.
Synthetic aperture radar (SAR) is a well known technique for developing radar imagery with excellent two-dimensional (2D) resolution. This is typically done by flying an airplane over the ground to be mapped, and successively transmitting a sequence of radar pulses. With the forward motion of the airplane, each successive radar pulse is transmitted from a position a little further along on the flight path, thus simulating a very long radar array.
Synthetic aperture radar is now a mature technique used to generate radar images in which fine detail can be resolved, and provides unique capabilities as an imaging tool. Because it provides its own illumination (the radar pulses), it can image at any time of day or night, regardless of sunlight illumination. And because the radar wavelengths are much longer than those of visible or infrared light, SAR can also “see” through cloudy and dusty conditions that visible and infrared instruments cannot.
As the radar moves, a pulse is transmitted at each position; the return echoes pass through the receiver and are recorded in an “echo store.” Because the radar is moving relative to the ground, the returned echoes are Doppler-shifted (negatively as the radar approaches a target; positively as it moves away). Comparing the Doppler-shifted frequencies to a reference frequency allows many returned signals to be “focused” on a single point, effectively increasing the length of the antenna that is imaging that particular point. This focusing operation, commonly known as SAR processing, is now done digitally on powerful computer systems. SAR processing must correctly match the variation in Doppler frequency for each point in the image, however, and this requires very precise knowledge of the relative motion between the platform and the imaged objects (which is the cause of the Doppler variation in the first place).
Radar energy is transmitted in the form of sequential pulses, at different time instances in the flight path of the vehicle. The pulses interact with the terrain (and any object on the terrain) and a portion of the pulse energy is reflected back towards the platform and recorded by a detector. Returns from different objects arrive at different times at the detector. These time differences provide information on range, which is then used to create a final radar image.
Typically, the transmitted pulses in a SAR system are sinusoidal in nature. The detector records the reflected sinusoids by recording a complex number, whose phase component is directly related to the time-of-flight, or range, and its magnitude component is proportional to the reflected energy. Thus, data captured in a SAR system is complex in nature, with properties that are unique with respect to data captured in systems from other imaging modalities.
Generally, return signals from the transmitted pulses are sampled in the airplane and either processed on board for immediate exploitation or stored or transmitted for processing at another site. The processing is computationally expensive, employing such techniques as FFT, inverse FFT, or correlation on vast amounts of data. These operations require vast processing power and storage.
Technology trends in the field of SAR indicate that SAR system designs are continuously pushing the envelope for increases in area coverage and resolution. These trends imply massive amounts of collected data, which in turn, stress the ability to store collected SAR data and rapidly disseminate SAR data. Furthermore, SAR data transmission is undesirably time-consuming for customers who are only interested in a particular target in the scene, as opposed to the entire scene, or customers who aren't interested in a high resolution image of the scene.
The present invention is embodied in a method of generating multi-resolution synthetic aperture radar (SAR) images. The method includes the steps of: (a) obtaining SAR phase history data, denoted as PH; (b) calculating an inverse Fourier transform of a first data set, denoted as ph0, of the SAR phase history data, PH, to form a highest resolution image, denoted as G0, where G0=FT−1(ph0); (c) calculating the inverse Fourier transform of a plurality of other data sets, denoted as ph1 . . . phN, to form a plurality of correlated lower resolution images, denoted as G1 . . . GN, where Gi=FT−1(phi), for i=1 to N; (d) subtracting successive ones of the highest resolution image, G0, and the lower resolution images, G1 . . . GN, to obtain a plurality of edge enhanced images, denoted as L0 . . . LN−1, where Li=Gi−Gi+1, for i=0 to N−1; and (e) obtaining a lowest resolution image, denoted as LN.
In a further embodiment, the present invention includes a method of generating multi-resolution synthetic aperture radar (SAR) images. The method includes the steps of: (a) obtaining SAR phase history data, denoted as PH; (b) computing an inverse Fourier transform of a first data set, denoted as ph0, of the SAR phase history data, PH, to form a highest resolution image, denoted as G0, where G0=FT−1(ph0); (c) successively low-pass filtering the highest resolution image, G0, to form a plurality of correlated lower resolution images, denoted as G1 . . . GN, where Gi+1=LPF(Gi), for i=0 to N−1; (d) subtracting successive ones of the highest resolution image, G0, and the lower resolution images, G1 . . . GN, to obtain a plurality of edge enhanced images, denoted as L0 . . . LN−1, where Li=Gi−Gi+1, for i=0 to N−1; and (e) obtaining a lowest resolution image, denoted as LN.
In yet another embodiment, the present invention includes a method of iterative target detection in a synthetic aperture radar (SAR) scene. The method includes (a) obtaining SAR phase history data, denoted as PH, corresponding to the scene; (b) integrating over a plurality of data sets, denoted as ph0 . . . phN, of the phase history data, PH, to obtain a plurality of features, denoted as F0 . . . FN; (c) storing the plurality of features, F0 . . . FN; (d) performing target detection based on the stored plurality of features to obtain a receiver operating characteristic (ROC) of the scene; (e) determining if the ROC is acceptable; and (f) repeating steps (b)–(e) for a new plurality of other data sets if the ROC is unacceptable. In still another embodiment, the present invention includes a method of progressively transmitting multi-resolution synthetic aperture radar (SAR) images. The method includes the steps of: (a) obtaining SAR phase history data corresponding to a scene; (b) generating a correlated image pyramid including a plurality of multi-resolution images of the scene; (c) generating a Laplacian image pyramid including a plurality of Laplacian images corresponding to a difference of the multi-resolution images in successive levels of the correlated image pyramid; (d) encoding each one of the plurality of Laplacian images; (e) receiving, from a client, a request for a desired image of the scene; and (f) transmitting each one of the plurality of Laplacian images, starting from a bottom level of the Laplacian image pyramid, until the desired image of the scene is transmitted.
It is to be understood that both the foregoing general description and the following detailed description are exemplary, but are not restrictive, of the invention.
The invention is best understood from the following detailed description when read in connection with the accompanying drawing. It is emphasized that, according to common practice, the various features of the drawing are not to scale. On the contrary, the dimensions of the various features are arbitrarily expanded or reduced for clarity. Included in the drawing are the following figures:
Airborne and spaceborne based SAR imaging systems are active coherent image acquisition systems and work like flashlight photography. A SAR imaging system emits a microwave frequency-modulated chirp and receives its echo. The collected raw data are then digitally processed to generate 2D SAR phase history data and a SAR image.
More specifically, a SAR system consists of the following subsystems: a pulse generator for generating particular microwave wavelength pulses, an antenna for transmitting the generated pulse to illuminate a region of interest, an antenna for receiving the backscattered returned pulses (i.e., echoes) from target points in the region of interest, a digitizer for converting the received complex analog signal (amplitude and phase) to a digital format, and a signal processor for constructing the SAR 2D phase history data and the SAR image.
SAR signal preprocessing and postprocessing are based on convolution of the SAR emitted beam with the scene and de-convolution of the collected data into a high quality 2D image. To describe the process of constructing the phase history data and forming the SAR image, a few relevant terms are defined, below.
Discrete Fourier Transform (DFT):
Those skilled in the art will recognize that the DFT may be efficiently implemented with the Fast Fourier Transform (FFT).
The convolution of two signals is represented by f{circle around (x)}g, where:
DFT[f{circle around (x)}g]=DFT[f]*DFT[g]=F(ω)G(ω).
The cross-correlation of two signals is represented by f*g, where:
DFT[f*g]=DFT[
The multiplication of two signals is represented by fg, where:
DFT[fg]=DFT[f]{circle around (x)}DFT[g]=F(ω){circle around (x)}G(ω).
Projection theorem states:
In one embodiment of the present invention, the system is a patch of terrain, the input to this system is a chirp modulated on a microwave carrier frequency and the output of the system is the received backscattered 2D phase history data. Let h(m,n) be the impulse response, x(m,n) be the input (i.e., the emitted chirp), and y(m,n) (i.e., the received data) be the output of the system. Accordingly, y(m,n)=x(m,n){circle around (x)}h(m,n), or, in the frequency domain, Y(u,v)=X(u,v)H(u,v), where Y(u,v)=FFT[y(m,n)], X(u,v)=FFT[x(m,n)], and H(u,v)=FFT[h(m,n)].
The impulse response may be computed by
and then h(m,n)=FFT−1[H(u,v)], noting that x(m,n), h(m,n), and y(m,n) are complex functions.
Also, the SAR system may be characterized by the following parameters: PRF (Pulse Repetition Frequency); RSF (Range Sampling Frequency); k (Chirp slope); τp (Pulse duration); λ (Radar wavelength); β (Real beam size); D (Real aperture size); βs (Synthetic beam size); Ls (Synthetic aperture size); and v (Platform velocity).
To construct x(m,n) and form y(m,n) from the range time domain and azimuth spatial domain, first the pulse/chirp generated by the pulse generator is mathematically modeled. The generated pulse/chirp is a sinusoidal pulse with a Pulse Repetition Frequency equal to PRF, and may be represented as:
where
is the two-dimensional RECT function:
Then the modulated chirp may be modeled as:
This modulated chirp is emitted/transmitted by the antenna and subsequently illuminates a target in a region of interest or a terrain patch.
The number of pulses per second, or Pulse Repetition Frequency, is equal to PRF. The number of the samples in range direction (at a given θ), NR, is equal to the Listening Range Echo Time (LRET) multiplied by the Range Sampling Frequency:
NR=LRET*RSF.
The number of the samples in the azimuth direction, NA, is equal to Listening Azimuth Echo Time (LAET) multiplied by the Pulse Repetition Frequency:
where v is the sensor velocity and Ls the systematic aperture length.
All the points in the field of the view are illuminated simultaneously while the chirp beam footprint (i.e., the beam spot size) interacts with the target. The terrain surface cover modifies the chirp (e.g., the amplitude, phase, and frequency), whereby the distorted and delayed chirp echoes back toward the antenna (i.e., as a backscattering signal).
The backscattering signal of a target point has an amplitude (i.e., signal strength) and a phase history. Since the SAR antenna is traveling, the echo of each point is received many times, or, in other words, the phase history of each target point is the phase delay of the point, which affects the phase delay of every other point in the field of view. Accordingly, the phase history of a point (k,l) in an M by N array of patch points for a given scene may be given by:
y(k,l)=Ae−i(nk+ml),0≦n≦N,0≦n≦N,0≦m≦M.
Each point on the ground is imaged many times as an airborne or spaceborne SAR platform moves along its path while measuring many points on the ground simultaneously.
To clarify, the received signal for an incidence angle/pulse may be modeled by:
where A(m,n) is the surface reflectance of the (m,n)th target point and φ(m,n)(t) is the phase history delay of the (m,n)th point. The phase history delay is related to the time of an echo return
The received data is the convolution of the transmitted pulsed beams and target topographical points, where for one look, for example, a terrain patch is illuminated from many different incidence angles and is, thus, unfocused and must be de-convolved improve its resolution in both cross-track and along-track directions.
The data processing is first done on the range (i.e., cross-track) and then on the azimuth (i.e., along-track) directions.
In general, the received signal is given by:
y(t)=x(t)exp(iωct){circle around (x)}pθ(t)
y(t)=FT−1[(X(ω−ωc)(Pθ(ω))]
For demodulation, a low pass filter may be used:
such that
where
Y(ω)=X(ω)Pθ(ω)=X(ω)H(ω cos θ, ω sin θ).
Tables 1–4, below, summarize the mathematical modeling, described above, of the aircraft position and beam incident angle, the transmitted signal, the preprocessed received signal in the time domain, and the Fourier transform of the preprocessed received signal for times t0, t1, t2, and tNA−1, respectively.
Next, the NA polar impulse frequency responses of the target may be computed as:
where for each impulse frequency response, (H(ω cos(θ), ω sin(θ)), there are NR polar sample points (frequency domain). Therefore, there are NA×NR sample points of the target polar impulse response in the frequency domain.
Now, 2D Cartesian impulse frequency response of the target impulse response may represented by H(u,v)=H(ωk,ωk), or, simply, H(k,l), where 0≦k≦NA−1 and 0≦l≦NA−1, and may be interpolated from the 2D polar impulse frequency response points:
Finally the SAR complex image may be reconstructed by:
Accordingly, the SAR phase history data, H(u,v), may be computed from the reconstructed SAR image, h(m, n), by taking the Fourier transform: H(u,v)=FFT [h(m,n)]. In one embodiment of the invention, only the magnitude of h(m,n) may be retained for display as the SAR image.
Referring now to the drawing, in which like reference numbers refer to like elements throughout the various figures that comprise the drawing,
Mathematically, taking a sub-aperture of the phase history data corresponds to multiplying the entire phase history data set, H(u,v), by a two-dimensional filter. For a rectangular sub-aperture, for example, this corresponds to multiplying by a 2D RECT function; for a circular sub-aperture, this corresponds to multiplying by a 2D Gaussian distribution.
A 1D box filter, for example, may be represented as:
where taking the Fourier transform gives:
FT(h(x))=F(u)=α sinc(αu).
A 2D box filter (i.e., a rectangle) may be represented as:
where taking the Fourier transform gives:
FT(g(x,y))=F(u,v)=α sinc(αu)β sinc(βv).
A 1D Gaussian filter, on the other hand, may be represented as:
where taking the Fourier transform gives:
A 2D circularly symmetric Gaussian distribution may be represented as:
where the Fourier transform gives:
A 2D circularly asymmetric Gaussian distribution may be represented as:
where the Fourier transform gives:
According to other embodiments of the invention, various other sub-aperture geometries may be used in addition to the rectangular, circularly symmetric, and circularly asymmetric embodiments described above.
Those skilled in the art will recognize the following properties of the Fourier Transform with respect to a 1D convolution:
g(x)=f(x){circle around (x)}h(x)
FT(g(x))=FT(f(x))FT(h(x))
and with respect to a 2D convolution:
g(x,y)=f(x,y){circle around (x)}h(x,y)
FT(g(x,y))=FT(f(x,y))FT(h(x,y)
BY representing a high-quality reconstructed zero-frequency center SAR phase history data as F(u,v), a SAR image may be formed by taking the inverse Fourier transform (f(x,y)=FT−1[F(u,v)]) or the inverse Fast Fourier transform (IFFT) (f(x,y)=FFT−1[F(u,v)]).
In one embodiment, phase history data F(u,v) is filtered by multiplication with a 2D RECT function representing a rectangular sub-aperture, which corresponds to a convolution of the SAR image with the Fourier transform of the 2D RECT function, as described above. This is shown by:
where α and β determine the size of a rectangle in the Fourier domain (i.e., the phase history domain).
In another embodiment, phase history data F(u,v) is filtered by multiplication with a 2D circularly symmetric Gaussian function representing a circular sub-aperture, which corresponds to a convolution of the SAR image with the Fourier transform of the 2D circularly symmetric Gaussian function, as described above. This is shown by:
where σ determines the size of the circle in the Fourier domain (i.e., the phase history domain).
In yet another embodiment, phase history data F(u,v) is filtered by multiplication with a 2D circularly asymmetric Gaussian function representing an elliptical sub-aperture, which corresponds to a convolution of the SAR image with the Fourier transform of the 2D circularly asymmetric Gaussian function, as described above. This is shown by:
where σx and σy determine the size of the ellipse in the Fourier Domain (i.e., the phase history domain).
Accordingly, an embodiment of the present invention may iteratively filter the phase history data by iteratively multiplying the phase history data, in the Fourier domain, with 2D filter functions corresponding to different sub-apertures. Furthermore, the inverse Fourier transform of the filtered phase history data may be taken after each iteration in order to obtain a multi-resolution SAR image.
For example, a first sub-aperture filtered phase history data set may correspond to
where the first SAR image may be obtained by taking the inverse Fourier transform:
In some instances, the first SAR image may not be satisfactory for a number of reasons, which may include, for example, undesirable aliasing effects in the first SAR image that arise as a result of the first sub-aperture selected. Accordingly, a new sub-aperture may be chosen, where the new sub-aperture filtered phase history data set may correspond to
The new SAR image, therefore, may be obtained by taking the inverse Fourier transform:
This iterative process may be repeated by changing σ until a desirable SAR image is formed from the sub-aperture filtered phase history data set, and/or until a correlated image pyramid is formed for image compression, to be described below. In general:
where n is the iteration number.
In an alternate embodiment, the size of the filter is fixed, but the filter may be applied successively. For example, in the first iteration, the phase history data is filtered once:
giving SAR image:
In the next iteration, the same filter is applied to the already once-filtered phase history data:
giving SAR image:
In general:
where n is the iteration number.
Those skilled in the art will recognize that the iteratively generated SAR images, obtained by exemplary embodiments of the invention such as the ones described above, may be spatially correlated in the image domain. In one embodiment of the invention, therefore, obtaining a plurality of correlated SAR images from different sub-apertures may include generating a correlated image pyramid. Accordingly, a Laplacian pyramid technique may be used to compress the plurality of correlated SAR images in the correlated image pyramid. In a further embodiment, target detection and surface or subsurface type classification may be performed by taking the integral of each level in the correlated image pyramid, where each level may correspond to a distinct sub-aperture filtered phase history data set.
In another embodiment, the phase history data may also be iteratively smoothed by: Fn+1(u,v)={Fn(u,v){circle around (x)}H(u,v)}, where H(u,v) is the Fourier transform of an optimal filter and n is the iteration number. As described above, convolution in one domain corresponds to multiplication in the transform domain, thus fn+1(x,y)=fn(x,y)h(x, y).
In one embodiment of the invention, a plurality of correlated SAR images may be obtained by iteratively changing a sub-aperture on the phase history data, as described above. In another embodiment of the invention, a plurality of correlated SAR images may be obtained by successively applying a sub-aperture on previously filtered phase history data, as described above. In yet another embodiment, a plurality of correlated SAR images may be obtained by choosing one particular SAR image and then successively downsampling it (e.g., successively convolving the image with a weighting function). Accordingly, the plurality of SAR images that are obtained may be compressed using a Laplacian pyramid, for example, by taking advantage of the correlation between each image in the plurality of images.
In an embodiment of the invention, therefore, successively downsampling a SAR image to form a plurality of SAR images may begin by setting the frame as the entire phase history data set, ph0=F(u,v), or, alternately, by choosing a sub-aperture on the phase history data set, ph0=Sub_Aperture[F(u,v)], where F(u,v) represents the phase history data and choosing a sub-aperture on the phase history data set corresponds to multiplying the phase history data set by a 2D function representing the sub-aperture, as described above. Then, to obtain a lower resolution SAR image for each ith iteration, the higher resolution SAR image from the previous (i−1)th iteration is low-pass filtered to produce the lower resolution SAR image (i.e., FT−1(phi)=LPF[FT−1(phi)]). This corresponds to convolving the image with a weighting function (such as a Gaussian weighting function, for example) in the spatial domain, or, equivalently, multiplying the image with the weighting function in the frequency domain. Accordingly, a sequence of multi-resolution images {FT−1(ph0), FT−1(ph1), . . . , FT−1(phn) } are generated, where the image generated by each ith iteration may be a fourth of the size of the image of the previous (i−1)th iteration
depending on the weighting function chosen. Those skilled in the art will recognize that a plurality of different weighting functions such as a Gaussian, triangular, rectangular, trimodal, broad Gaussian, or other weighting function may be chosen without departing from the scope of the present invention.
In general, by downsampling (i.e., low-pass filtering) the SAR images successively with a low-pass filter, high frequency content (i.e., edges) is removed from the image. As such, a Laplacian pyramid of edge enhanced images may be generated by subtracting the SAR image of each (i+1)th iteration from the SAR image of the previous ith iteration (i.e., Li=FT−1(phi)−FT−1(phi+1)). Accordingly, a sequence of Laplacian edge-enhanced images, L0, L1, . . . , Ln, may be generated and subsequently encoded to achieve greater data compression, where each Li is smaller than its predecessor, Li−1, by a scale of one half due to reduced image sample density in the horizontal and vertical axes.
Reconstruction of original SAR images g0–g3 may begin by obtaining quantized value Q3, which is then decoded to obtain a reconstructed fourth Laplacian image L3* (which is also equivalent to reconstructed lowest resolution SAR image g3*). Quantized value Q2 is obtained next and subsequently decoded to obtain reconstructed third Laplacian image L2*. Reconstructed third Laplacian image L2* may then be added to reconstructed lowest resolution SAR image g3 by adder 804 to obtain reconstructed second lower resolution SAR image g2*. Quantized value Q1 is obtained next and subsequently decoded to obtain reconstructed second Laplacian image L1. Reconstructed second Laplacian image L1 may then be added to reconstructed second lower resolution SAR image g2* by adder 805 to obtain reconstructed first lower resolution SAR image g1. Finally, quantized value Q0 is obtained and subsequently decoded to obtain reconstructed first Laplacian image L0*. Reconstructed first Laplacian image L0* may then be added to reconstructed first lower resolution SAR image 91 by adder 806 to obtain reconstructed high resolution SAR image g0*. Those skilled in the art will recognize the reconstruction steps described above may constitute a progressive transmission of multi-resolution SAR images to a client. Accordingly, such a progressive transmission may stop whenever the client has received a SAR image having an adequate resolution, thereby possibly precluding the need to transmit all of quantized values Q0–Q3.
Those skilled in the art will recognize that many types of encoding schemes may be used to obtain quantized values without departing from the scope of the present invention, including, for example, entropy encoding techniques such as Huffman coding, range encoding, and arithmetic encoding, or static coding techniques such as Unary coding, Elias coding, Fibonacci coding, Golomb coding, or Rice coding.
In an embodiment of the invention, the method of forming the correlated image pyramid, described above, may be iteratively repeated if any images in the pyramid have undesirable characteristics. Such undesirable characteristics may include, but are not limited to, image distortion, aliasing, blur, noise, and the like.
In one embodiment, for example, the path labeled “Option A” in
In another embodiment, the path labeled “Option B” in
FT−1{[PH·f(SA0)]·f(SA0)}=FT−1{FT(G1)·f(SA0)}.
Step 923 then determines whether the current iteration number, n, is less than or equal to a total number of desired iterations, N, and then increments the current iteration number, n. If n is less than or equal to N, the process moves back to step 920 to form the next Laplacian image, and then to steps 921–922 to form the next SAR image. Accordingly, steps 920–922 are repeated until the correlated image pyramid and corresponding Laplacian pyramid are formed for a desired number of iterations (or levels), N. Once the desired number of iterations are completed, the process ends in step 900.
The corresponding Laplacian pyramid may then be formed by subtracting g1 from g0 in subtractor 1105 to form first Laplacian image L0, g2 from g1 in subtractor 1106 to form second Laplacian image L1, and g3 from g2 in subtractor 1107 to form third Laplacian image L2. Lowest resolution image g3 may be used as fourth Laplacian image L3. Those skilled in the art will recognize that a sub-aperture of any desirable size and/or shape may be chosen, that the highest resolution image g0 may be taken from any sub-aperture filtered phase history data set, and that any number of iterations may be performed to obtain a desired number of levels of the correlated image pyramid.
In yet another exemplary embodiment, the path labeled “Option C” in
The corresponding Laplacian pyramid may then be formed by subtracting g1 from g0 in subtractor 1200 to form first Laplacian image L0, g2 from g1 in subtractor 1201 to form second Laplacian image L1, and g3 from g2 in subtractor 1202 to form third Laplacian image L2. Lowest resolution image g3 may be used as fourth Laplacian image L3. Those skilled in the art will recognize that new sub-apertures of any desirable size and/or shape may be chosen, that the highest resolution image g0 may be taken from any sub-aperture filtered phase history data set, and that any number of iterations may be performed to obtain a desired number of levels of the correlated image pyramid.
In a further embodiment of the present invention, the path labeled “Target Detection” in
Exemplary features may also include annuli and sector masks, where each annuli and sector corresponds to a sub-aperture. Accordingly, in a target detection method according to the present invention, only a few concentric annuli in an upper Fourier half-plane (SAR phase history data) and equal-angle sectors in a lower Fourier half-plane, for selected hyper spectral bands or in an orthogonal subspace, features may be extracted.
In yet another embodiment of the invention, multi-resolution SAR images may be progressively transmitted to a client that requests a desired image of a scene or one or more targets within the scene. The client may require that the desired image have a desired resolution and be substantially free from undesirable characteristics such as aliasing, noise, blur, and the like, for example. In general, the method may begin by obtaining SAR phase history data corresponding to the scene. A correlated image pyramid may then be generated, as described above, comprising a plurality of multi-resolution images of the scene. A Laplacian pyramid may then be generated, as described above, comprising a plurality of edge-enhanced images corresponding to a difference in the multi-resolution images in successive levels of the correlated image pyramid. Each one of the Laplacian pyramid images may then be encoded for further compression.
A client request for the desired image may be received either before, after, or simultaneous to the steps described above. Once the encoded Laplacian pyramid images are ready, they are successively transmitted to the client, starting from the bottom level of the pyramid (i.e., lowest resolution and smallest size image) and working up the pyramid until the desired image has been sent.
In one embodiment of the invention, a request for a desired image is received from a client, but the characteristics of the desired image are unknown at the time of transmission. Accordingly, the method may continue to transmit the encoded images starting from a bottom level until an indication is received from the client that the desired image has been transmitted. The method may stop sending images once it's received such an indication. In a further embodiment, the method may transmit all the encoded images without receiving an indication that the desired image has been transmitted. This corresponds to a situation where the client is not happy with the characteristics of the images in the correlated image pyramid. In such a case, therefore, a new correlated image pyramid and corresponding encoded Laplacian pyramid may be formed, and the method repeated, until the desired image is transmitted.
In another embodiment, the method may also detect one or more targets in the scene, as described above. Accordingly, correlated image pyramids may be generated as comprising a plurality of multi-resolution images of one or more desired targets, rather than the entire scene. The method may then receive a request from a client for a desired image of a particular target in the scene. The appropriate image pyramid is then selected and the progressive transmissions commenced until the desired image is transmitted.
Although illustrated and described above with reference to certain specific embodiments, the present invention is nevertheless not intended to be limited to the details shown. Rather, various modifications may be made in the details within the scope and range of equivalents of the claims and without departing from the invention.
Number | Name | Date | Kind |
---|---|---|---|
5309522 | Dye | May 1994 | A |
5659318 | Madsen et al. | Aug 1997 | A |
5898399 | Carrara et al. | Apr 1999 | A |
5923278 | Poehler et al. | Jul 1999 | A |
6011505 | Poehler et al. | Jan 2000 | A |
6046695 | Poehler et al. | Apr 2000 | A |
6861978 | Lam | Mar 2005 | B2 |
20030142000 | Cho | Jul 2003 | A1 |
20030222807 | Cho | Dec 2003 | A1 |
20040004569 | Lam | Jan 2004 | A1 |
20040017307 | Cirillo et al. | Jan 2004 | A1 |
20050007269 | Carrara et al. | Jan 2005 | A1 |
20060109165 | Cho | May 2006 | A1 |
20070040121 | Kalayeh | Feb 2007 | A1 |
Number | Date | Country |
---|---|---|
4117849 | Dec 1992 | DE |
4240225 | Jun 1994 | DE |
05126943 | May 1993 | JP |
06118166 | Apr 1994 | JP |