Method for forming an image transformation matrix for an arbitrarily shaped image segment of a digital image with a computer

Abstract

In a method for forming an image transformation matrix for an arbitrarily shaped image segment of a digital image with a computer, a prescribable scan sequence is determined for the picture elements of an image segment of the digital image, a covariance matrix for the picture elements is determined on the basis of this scan sequence. An image transformation matrix is derived from the covariance matrix on the basis of the eigenvectors of the covariance matrix.

Description

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention is directed to a method for encoding video signals, and in particular to a method for forming an information transformation matrix for an arbitrarily shaped image segment of a digital image using a computer.

2. Description of the Prior Art

The encoding of video signals according, for example, to the image encoding standards H.261, H.263, MPEG1 as well as MPEG2 is often based on a block-oriented discrete cosine transformation (DCT). These block-oriented encoding methods, however, are not suitable for image encoding which is not based on rectangular blocks but wherein, for example, subjects from an image are segmented and the segments of the image are encoded. These latter methods are known as region-based (region-oriented) or subject-based (subject-oriented) image encoding methods. A segmenting of subjects in digital images thereby ensues according to the subjects occurring in the scene. A separate encoding of these segmented subjects is implemented instead of the encoding of image blocks as in block-based image encoding methods. The encoding thereby usually ensues by modeling the segmented subjects and subsequent transmission of the modeling parameters of these segmented subjects.

After the transmission of the image information from a transmitter to a receiver, the individual subjects of the image are in turn reconstructed in the receiver on the basis of the transmitted modeling parameters.

One possibility for modeling the subjects is a series development of the image function according to a set of suitably selected basic functions. The modeling parameters then correspond to the development coefficients of these image functions. Such a modeling of the image is the basis of the transformation encoding. When individual, arbitrarily bounded subjects of the image are to be encoded, a transformation for segments with arbitrary, usually not convex, bounds is required.

Two basic approaches have heretofore existed for such a transformation.

In the method that is described in M. Gilge, T. Engelhardt and R. Mehlan, Coding of arbitrarily shaped image segments based on a generalized orthogonal transform, Signal Processing: Image Communication 1,00. 153-180, October 1989, the given image segment is first embedded into a circumscribing rectangle with the smallest possible dimensions. A discrete cosine transformation (DCT) that is completely specified by the basic functions of the transformation can be recited for this rectangle. In order to match this transformation to the segment shape, the basic functions defined on the rectangle are successively orthogonalized with respect to the shape of the segment. The resulting orthogonal, shape-dependent basic functions then form the segment-matched transformation that is sought.

One disadvantage of this approach is that there is a large capacity and a large memory space needed for the implementation of this method. Further, this known method exhibits the disadvantage that no reliable statements can be made about the resultant transformation for data compression, since the transformation is essentially dependent on the orthogonalization sequence, and thus on the specific implementation.

T. Sikora and Bela Makai, Shape-adaptive DCT for generic coding of video, IEEE Trans. Circuits and Systems for Video Technology 5, pp. 59-62, February 1995 describes a method wherein the given image segment is transformed separated according to rows and columns. To that end, all rows of the image segment are first left-justified and are successively subjected to a one-dimensional horizontal transformation whose transformation length respectively corresponds to the number of picture elements in the corresponding row. The resultant coefficients are subsequently transformed a second time in the vertical direction.

This method has the disadvantage that the correlations of the brightness values of the picture elements (similarities of the picture elements) cannot be fully exploited because of the resorting of the picture elements.

For improving this method known from Sikora et al., T. Sikora, S. Bauer and Bela Makai, Efficiency of shape-adaptive 2-D transforms for coding of arbitrary shaped image segments, IEEE Trans. Circuits and Systems for Video Technology 5, pp. 254-258, June 1995 describe a method wherein a transformation for convex image segment shapes adapted to a simple image model is implemented. Only image segment shapes that exhibit no interruptions (holes) upon traversal of rows or columns, however, are allowed in this method.

A considerable disadvantage that underlies both known approaches is that the energy concentration in the coefficients turns out lower than in the case of an optimum exploitation of all linear correlations. This is caused by the unfavorably selected basic functions given the method known from Gilge et al., the resorting of the picture elements given the first-discussed Sikora et al. article and the limitation to convex images regions given the second-discussed Sikora et al. article.

As a consequence thereof, the described, known methods do not achieve the best possible image quality at a given data rate as measured by the signal-to-noise ratio.

Further, various possibilities are known for determining the eigenvectors of a covariance matrix, for example from W. H. Press, S. Teukolsky and W. Vetterling, Numerical Recipes in Pascal, Cambridge University Press, pp. 375-389, 1992.

Various, known image transformation methods are described in J. -R. Ohm, Digitale Bildcodierung, Berlin Springer Verlag, ISBN 3-540-58579-6, pp. 46-51 and pp. 72-77, 1995.

The use of the Karhunen-Loeve transformation is known from M. Dekker, BOW "Pattern Recognition" Inc. 1984, pp. 213-217. The use of principal axis transformation is known from Ernst, Einfuhring in die digitale Bildverarbeitung, Franzis-Verlag, 1991, pp.250-252.

SUMMARY OF THE INVENTION

An object of the present invention is to provide a method for forming an image transformation matrix with which an improved signal-to-noise ratio is achieved compared to the known methods.

The above object is achieved in accordance with the principles of the present invention in a method for forming an image transformation matrix for an arbitrarily shaped image segment containing an arbitrary number of picture elements in a digital image including the steps, practiced in a computer, of defining a scan sequence of the picture elements in an image, forming a covariance matrix for the picture elements according to the scan sequence, and forming an image transformation matrix from the covariance matrix by determining the eigenvectors of the covariance matrix.

A scan sequence for scanning the individual picture elements that comprise an image segment is thereby defined for each image segment. A covariance matrix for the picture elements is formed corresponding to the scan sequence and, after a determination of the eigenvectors for the covariance matrix, the inventive image transformation matrix is derived from the eigenvectors of the covariance matrix.

The image transformation matrix formed in the inventive way has the property that the individual picture elements of the image segment are optimally completely linearly decorrelated by the transformation implemented with the image transformation matrix. A further advantage of the inventive method is that, for a given data rate, the signal-to-noise ratio is considerably improved compared to known transformations.

It is advantageous to form the elements of the covariance matrix according to the rule R(x.sub.ij, x.sub.kl)=r.sup..vertline.i-k.vertline.+.vertline.j-l.vertline., since a decoupling of the individual directions of the calculation of the elements of the covariance matrix is possible in this way. A parallelization of the implementation of the formation of the covariance matrix, and thus a considerable acceleration of the implementation of the method with a computer is also achieved in this way.

In a further version of the method, a shape-matched two-dimensional cosine transformation is defined from the image transformation matrix of the inventive method by fixing the value of the pixel correlations to the value 1.

It is also advantageous to sort the coefficients according to descending eigenvalues for encoding the individual coefficients of an image segment transformed using the image transformation matrix. This saves substantially in terms of the required transmission rate in the transmission of the individual coefficients since the outlay for encoding interspaces wherein only "zero coefficients" occur is considerably reduced because there are substantially fewer zero coefficients between the individual coefficients.

DESCRIPTION OF THE DRAWINGS

FIG. 1 is a flowchart in which the individual methods steps of the inventive method are shown.

FIG. 2 schematically illustrates an image with individual picture elements and an image segment is shown as an example for explaining the inventive method.

FIG. 3 shows an arrangement with a camera, two computers and two picture screens for practicing the inventive method.

FIG. 4 is a flowchart in which additional method steps of another embodiment of the inventive method are shown.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The assumption of a separable, two-dimensional Markoff set of the first order as image model for picture elements BP of the respective digital image B is made for the inventive method. The picture elements BP are elements of an arbitrary image segment S with an arbitrary shape that is also non-convex or multiply connected.

In the inventive method, a prescribable scan sequence for the picture elements BP of the respective image segment S that is formed by an arbitrary number of picture elements is defined in a first step 101. This, for example, can be accomplished by scanning the individual picture elements BP of the image segment S line-by-line. This line-by-line scanning is only an example, however, which serves for simplification of the inventive method and does not limit the universal validity of the scan sequences in any way. The scan sequence can be arbitrarily fixed.

A "segment-specific" covariance matrix R is now determined in step 102 for the image segment S. This occurs by taking the previously determined scan sequence is taken into consideration in the formation of the covariance matrix R that corresponds to the scan sequence. A first row of the covariance matrix R is obtained by evaluating covariance function R(x.sub.ij, x.sub.kl) for a first image pixel x.sub.ij of the image segment S for all picture elements BP of the image segment S. The coordinate position of a first picture element x.sub.ij is thereby described with a first row index i and a first column index j. The respective coordinate positions of all other picture elements BP of the image segment S are described with a second row index i and a second column index j.

The sequence of the selection of the first picture element x.sub.ij and of a second picture element x.sub.kl in the formation of the covariance matrix R ensues according to the previously defined scan sequence. The number of elements in the first row of the covariance matrix R thus just corresponds to the number of picture elements within the image segment. The determination of a second row of the covariance matrix R ensues analogously, as does the determination of all further rows of the covariance matrix R. The second picture element x.sub.kl is "held fast" in the second row of the covariance matrix R and the covariance function R(x.sub.ij, k.sub.kl) is in turn determined for all other picture elements BP.

This procedure is implemented for all n rows of the covariance matrix R, whereby the number n simultaneously represents the number of picture elements BP in the image segment S of the image B.

An image transformation matrix A.sup.T is then determined row-by-row from the eigenvectors of the covariance matrix R in step 103. This means that the basic functions of the transformation correspond to the eigenvectors of the covariance matrix R.

Those skilled in the art know of various possibilities for determining the eigenvectors of the covariance matrix R, for example from the aforementioned Press et al. publication.

FIG. 2 shows a very simple example of an image B that contains an arbitrary plurality of picture elements BP and an image segment S with n picture elements within the image B.

A brightness value and/or a color value with which the brightness and/or the color of the respective picture element BP is indicated is respectively allocated to the picture elements BP in the form of a numerical value that usually lies between 1 and 255.

In FIG. 2, each picture element BP that is located in the image segment S is identified with shading.

For illustration of the inventive method, this method shall be described with the numerical values that are shown by way of example in FIG. 2 for the image segment S in the image B. This trivial example is merely intended to illustrate the procedure and does not limit the universal validity of the inventive method in any way whatsoever.

The brightness values are first combined in an arbitrary but fixed scan sequence to form a picture element vector x, line-by-line in this example. The picture element vector x accordingly is derived as

x=(x.sub.11, x.sub.12, x.sub.14, x.sub.22, x.sub.23, x.sub.24, x.sub.31, x.sub.32).sup.T ==(100, 128, 111, 92, 85, 130, 57, 79).sup.T

The individual picture elements BP within the image B are respectively unambiguously identified with the two subscript indices at every component x.sub.ij of the picture element vector x, namely by the first row index i as well as by the first column index j. In the example of FIG. 2, the first row index i is a number between 0 and 5, generally between 0 and m-1, whereby m references the number of image lines of the image B. In the image B shown in FIG. 2, the first column index j is a number between 0 and 6 but generally a number between 0 and t-1, whereby t references the number of image columns of the image B.

The individual elements of the covariance matrix B are determined with a covariance function R(x.sub.ij, x.sub.kl). Those skilled in the art are aware of a variety of covariance functions. For simple presentation of the inventive method, a formation of the elements of the covariance matrix R on the basis of the following covariance function R is shown:

r(x.sub.ij,x.sub.kl)=r.sup..vertline.i-k.vertline.+.vertline.j-l.vertline.( 1) r thereby references an arbitrary number with .vertline.r.vertline..ltoreq.1.

Using of this covariance function R, the following derives for the covariance matrix R given the selected numerical example from FIG. 2: ##EQU1## The value of the covariance function R(x.sub.11, x.sub.11)=R.sup..vertline.1-1.vertline.+.vertline.1-1.vertline. =1, for example, derives for the first element in the first row of the covariance matrix R.

Those skilled in the art know of a variety of methods for determining the eigenvectors from the covariance matrix R which can be used without restrictions in the inventive method.

When the eigenvectors are sorted in ascending sequence according to the size of the eigenvalues, which is not compulsory for the inventive method, then the following derives in the simple example shown here for the image transformation matrix A.sup.T for a given value r=0.95: ##EQU2##

The respective picture element vector x of the image segment can now be transformed with the image transformation matrix A.sup.T, whereby the individual picture elements BP are completely linearly decorrelated.

For a decorrelated picture element vector y, which is determined by transformation of the picture element vector x with the image transformation matrix A.sup.T, the following derives for the numerical example shown here:

y=A.sup.T x=(19.026 19.2285 20.3278 -11.2088 11.0457 28.1784 -46.0299 27.5481)

Another of many possibilities for determining the individual elements of the covariance matrix R, i.e. a possibility of the covariance function R(x.sub.ij, x.sub.kl), derives from the following rule: ##EQU3##

The covariance function R(x.sub.ij, x.sub.kl) according to Equation (1), however, exhibits the advantages that, first, the directions of the individual picture elements are decoupled, and thus determination of the elements of the covariance matrix R in parallel is possible, which leads to an accelerated implementation of the inventive method.

Among other things, FIG. 3 shows a computer R1 with which the inventive method is necessarily implemented.

FIG. 3 also shows a camera KA with which a sequence of images is registered that is converted into a sequence of digital images B in the computer R1. These digital images B are stored in a memory SP1 of the computer R1. A picture screen BS1 is also provided for the computer R1 in this arrangement.

Given a transmission of the digitized image B, an image transformation using the inventively formed image transformation matrix A.sup.T is implemented before the transmission. A second computer R2 with a second memory SP2 and a second picture screen BS2 is also provided, this being coupled to the computer R1 via a channel K.

FIG. 4 shows a few additional method steps of modifications of the inventive method. For example, one or more images can be registered in step 401 with the camera KA. The image or the images are digitized in step 402 in the computer R1, and the individual picture elements BP of the image B have brightness values allocated to them in step 403. The digitized image B is now divided in step 404 into individual image segments S.

A segment-specific image transformation matrix A.sup.T is now respectively inventively formed for each individual image segment S in the way shown in FIG. 1 with the steps 101, 102 as well as 103.

The picture element vector x is transformed in step 405 to the decorrelated picture element vector y with the image transformation matrix A.sup.T. The coefficients resulting therefrom, i.e. the components of the decorrelated picture element vector y, are transmitted in step 406 from the computer R1 to the second computer R2 via the channel K. The coefficients are received in step 407 in the second computer R2 and inversely transformed in step 408 according to the inverse image transformation matrix (A.sup.T).sup.-1. The digitized image B is reconstructed in step 409 on the basis of the reconstructed picture element vectors x that have now been determined again. This image is displayed to a user on the second picture screen BS2 or on the first picture screen BS1.

In a further version of the method, it is also advantageous to sort the coefficients of the image transformation matrix A.sup.T according to descending eigenvalues. A considerable amount of transmission capacity is saved by this procedure. Usually, the coefficients of the image segment S transformed with an arbitrary image transformation matrix are subjected to a quantization and a subsequent scan process. Zero coefficients that arose due to the quantization are thereby encoded such that the number of zero coefficients are encoded as a natural number between the coefficients whose values are unequal to zero. When a large number of zero coefficients lies between two "non-zero" coefficients, a considerable number of bits are required for encoding this natural number and, of course, these must be transmitted. The number of zero coefficients between non-zero coefficients is considerably reduced by the sorting according to the descending eigenvalues. The need for transmission rate that is present for the encoding of the large natural numbers is thus also considerably reduced.

Although modifications and changes may be suggested by those skilled in the art, it is the intention of the inventor to embody within the patent warranted hereon all changes and modifications as reasonably and properly come within the scope of his contribution to the art.

Claims

1. A method for forming an image transformation matrix for an arbitrarily shaped image segment that comprises an arbitrary plurality of picture elements in a digital image, using a computer, comprising the steps of:

conducting a scan of individual picture elements in an arbitrarily shaped image segment of an image in a picture element-by-picture element scan sequence of the picture elements at least in the arbitrary image segment;

forming a covariance matrix for the picture elements according to the scan sequence; and

forming an image transformation matrix from the covariance matrix by determining the eigenvectors of the covariance matrix.

2. A method as claimed in claim 1 comprising the additional steps of:

allocating brightness values and/or color values to the picture elements;

forming a picture element vector from the brightness values and/or color values according to the scan sequence; and

transforming the picture element vector with the image transformation matrix to form a decorrelated picture element vector.

3. A method as claimed in claim 2 comprising the additional steps of:

registering said image with a camera;

dividing said image into image segments;

transmitting said decorrelated picture element vector from the computer to a further computer;

inversely transforming the decorrelated picture element vector at said further computer;

reconstructing a digital image at said further computer using the inversely transformed decorrelated picture element vector; and

displaying the reconstructed digital image on a picture screen.

4. A method as claimed in claim 3, comprising defining said scan sequences by scanning along an image line using said camera.

5. A method as claimed in claim 1 comprising determining elements of the covariance matrix according to the following rule:

r(x.sub.ij,x.sub.kl)=r.sup..vertline.i-k.vertline.+.vertline.j-l.vertline.

wherein R(x.sub.ij, x.sub.kl) indicates a value of the covariance matrix,

i,k, indicate row indices, and

j,l indicate column indices.

6. A method as claimed in claim 5 comprising the additional step of normalizing elements of the covariance matrix.

7. A method as claimed in claim 1 whereby the elements of the covariance matrix are determined according to the following rule: ##EQU4## wherein R(x.sub.ij, x.sub.kl) indicates a value of the covariance matrix, i,k, indicate row indices, and

j,l indicate column indices.

8. A method as claimed in claim 7 comprising the additional step of normalizing elements of the covariance matrix.

9. A method as claimed in claim 1 comprising the additional step of sorting coefficients of the image transformation matrix A.sup.T according to descending eigenvalues.