1. Field of the Invention
The present invention relates to an image transmitter, and more specifically to an image transmitter for compressing incoming image data and transmitting the compressed image data to a receiver via a transmission path.
2. Description of the Background Art
Examples of conventional image compression schemes are the MPEG (Motion Picture Experts Group) scheme and the DVC (Digital Video Cassette) scheme. According to these image compression schemes, incoming image data is subjected to DCT (Discrete Cosine Transform) and variable-length coding on a macro block-by-macro block basis, whereby a high compression rate for the image data may be realized. An implementation example of such an image compression scheme is a moving picture encoder which is disclosed in Japanese Patent Laid-Open Publication No. 7-280911.
However, after the aforementioned moving picture encoder receives one line of pixels arranged along a horizontal direction within an image to be processed, the moving picture encoder may also receive a next line of pixels. As a result, before all of the pixels which compose one macro block are received, the moving picture encoder may receive a number of pixels which are unrelated to that macro block. The receipt of such unnecessary pixels causes a delay time associated with compression processing in conventional moving picture encoders.
Therefore, an object of the present invention is to provide a transmitter which is capable of compressing an incoming image for transmission to a receiver with a relatively small delay time.
The present invention has the following features to attain the object above. According to one aspect of the invention, there is provided an image transmitter for compressing image data and transmitting the image data to a receiver via a transmission path, wherein the image data at least contains i pixel values of pixels arranged in line along a single direction, each pixel value being expressed in n bits, the image transmitter comprising: a blocking section for taking every p pixel values among the i pixel values in the image data to form a data block, and sequentially outputting a plurality of the data blocks each including the p pixel values; a data compression section for reducing an amount of data from each data block outputted from the blocking section to output a compressed block; and a data sending section for sending the compressed block outputted from the data compression section onto the transmission path, wherein i, n, and p are predetermined natural numbers.
These and other objects, features, aspects and advantages of the present invention will become more apparent from the following detailed description of the present invention when taken in conjunction with the accompanying drawings.
The blocking section 1 receives image data TD to be processed. As shown in
As shown in
As shown in
In addition to the pixel values XV1 to XVi×j, the image data TD may contain any other additional information to be used for certain purposes. However, such additional information is not essential for the present embodiment of the invention, and therefore is omitted from the illustration and descriptions.
In accordance with the aforementioned format of the image data TD, the blocking section 1 receives the pixel values XV1 to XVi×j in this order. Among the incoming pixel values XV1 to XVi×j, the blocking section 1 takes every p (where p is a predetermined number) pixel values to generate q data blocks DB, each of which is composed of p pixel values. Herein, “p” is a divisor of i, and in the present embodiment is assumed to be 8; and “q” is equal to {(i×j)/p}. Under the above assumptions where i=640, j=480, and p=8, q is 38400. For conciseness, a data block DB which is the rth generated data block will be denoted as a “data block DBr” with a suffix r. Herein, “r” is a natural number such that 1≦r≦q. As shown in
As described in connection with the background art, according to the typical conventional image compression schemes such as the MPEG scheme or the DVC scheme, a number of unnecessary pixels are likely to have been received before all of the pixels composing a macro block are received, giving rise to an unwanted delay time. In contrast, according to the present embodiment of the invention, the blocking section 1 generates a data block DBr which is composed of p pixel values XV1 to XVi×j in the order they are received. In other words, the blocking section 1 becomes ready to construct a data block DBr as soon as it receives p contiguous pixel values XV along the width direction HD, which can then be quickly passed to the next stage, i.e., the data compression section 2. As a result, the delay time can be reduced as compared to the conventional image compression schemes.
The data compression section 2 performs a first or second compression process (described later) for each incoming data block DBr. Thus, the data compression section 2 generates a compressed block CBr (r=1,2, . . . q) having a fixed length composed of predetermined s bits. Each compressed block CBr thus generated is outputted from the data compression section 2 to the data sending section 3 (see
Next, with reference to
Next, the first compression process which is performed by the data compression section 2a having the above-described structure will be specifically described. Data blocks DBr which are sent from the blocking section 1 (described above) are sequentially received by the splitter section 21 in the data compression section 2a. Each data block DBr includes an array of p pixel values XVp×(r−1)+1 to XVp×r (see
As shown in
More specifically, the delay section 221 sequentially receives pixel values XVp×(r−1)+1 to XVp×r−1. The delay section 221 applies a delay amount DL1 to the respective received pixel values XVp×(r−1)+1 to XVp×r−1, and outputs the resultant pixel values as delayed pixel values LXVp×(r−1)+1 to LXVp×r−1 to the subtraction section 222. Now, the delay amount DL1 will be specifically described. The subtraction section 222 receives (as described later) pixel values XVp×(r−1)+2 to XVp×r from the splitter section 21. The delay amount DL1 is prescribed at a value which ensures that the delayed pixel values LXVp×(r−1)+1 to LXVp×r−1 will be received by the subtraction section 222 substantially concurrently with the pixel values XVp×(r−1)+2 to XVp×r. In the present embodiment of the invention, the delay amount DL1 is prescribed to be equal to one clock which defines the operation timing of the DPCM encoding section 22.
The subtraction section 222 receives the pixel values XVp×(r−1)+2 to XVp×r from the splitter section 21. The subtraction section 222 also receives the delayed pixel values LXVp×(r−1)+1 to LXVp×r−1 from the delay section 221. Note that the aforementioned delay amount DL1 ensures that the delayed pixel value LXVp×(r−1)+1 from the delay section 221 and the pixel value XVp×(r−1)+2 from the splitter section 21 are received by the subtraction section 222 substantially simultaneously. The subtraction section 222 subtracts the currently-received delayed pixel value LXVp×(r−1)+1 from the currently-received pixel value XVp×(r−1)+2 to generate a differential data DDp×(r−1)+2 representing a difference value therebetween. In other words, the subtraction section 222 calculates a difference value between the received pixel value XVp×(r−1)+2 and a preceding pixel value XVp×(r−1)−1 in the image data TD (see
Moreover, the subtraction section 222 subtracts the delayed pixel value LXVp×(r−1)+2 from the concurrently-received pixel value XVp×(r−1)+3 to generate a differential data DDp×(r−1)+3. Thereafter, the subtraction section 222 repeats similar processes until it generates a differential data DDp×r from the pixel value XVp×r and the delayed pixel value LXVp×r−1. As can be seen from
As described above, q data blocks DBr are generated for one frame of image MG, and the DPCM encoding section 22 generates differential data DDp×(r−1)+2 to DDp×r for each data block DBr. Therefore, the near-instantaneous compression section 23 sequentially receives q sets of differential data DDp×(r−1)+2 to DDp×r. The near-instantaneous compression section 23 generates compressed data CDp×(r−1)+2 to CDp×r from each set of received differential data DDp×(r−1)+2 to DDp×r, respectively, in accordance with a near-instantaneous compression scheme.
More specifically, the buffer section 231 shown in
The level determination section 232 generates one level value LVr for every set of differential data DDp×(r−1)+2 to DDp×r received from the buffer section 231. The data reduction section 233 deletes predetermined t bits from each of the received differential data DDp×(r−1)+2 to DDp×r, as will be described in more detail later. Herein, “t” is a natural number such that 1≦t<(n+1), and in the present embodiment is assumed to be 5. Stated differently, the data reduction section 233 leaves u bits intact among the (n+1) bits which compose each of the differential data DDp×(r−1)+2 to DDp×r. Herein, “u” is equal to (n+1−t), which under the above assumptions is 4. The level value LVr specifies the positions of the u bits to be left intact among the (n+1) bits which compose each of the differential data DDp×(r−1)+2 to DDp×r.
In order to derive the aforementioned level value LVr, the level determination section 232 shown in
The level selection section 2322 receives the maximum differential data MDDv from the differential data selection section 2321. The level selection section 2322 determines the sign (i.e., positive or negative) of the received maximum differential data MDDv based on the value of its sign bit SBv.
If the current maximum differential data MDDv has a positive value (i.e., the sign bit SBv is “0”), then the level selection section 2322 operates in the following manner. Herein, in the case where the sign bit SBv is “0”, the maximum differential data MDDv has one of n bit patterns BP1 to BPn as shown in
In the present embodiment of the invention, as shown in
In the following description, as shown in
The level selection section 2322 detects the bit position at which the first instance of “1” appears, the check being begun at the MSB of the current maximum differential data MDDv. In other words, the level selection section 2322 determines one of the reference bit positions RBL1 to RBLn which corresponds to the current maximum differential data MDDv. Next, the level selection section 2322 selects one of the level values LV1 to LVt+1 which is assigned to the currently-determined one of the reference bit positions RBL1 to RBLn, and outputs this level value to both the data reduction section 233 and the packet assembling section 24. Among the level values LV1 to LVt+1, the currently-outputted level value is referred to as the “level value LVy”, as defined earlier.
As described above, the data reduction section 233 receives the differential data DDp×(r−1)+2 to DDp×r from the buffer section 231. The data reduction section 223 also receives the level value LVy from the level determination section 232. Based on the current level value LVy, the data reduction section 233 deletes t bits from each of the current differential data DDp×(r−1)+2 to DDp×r. As a result, as shown in
According to the present embodiment of the invention, as shown in
As mentioned earlier, it is assumed that the sign bits SBp×(r−1)+2 to SBp×r are “0”. Under this assumption, the data reduction section 233 may alternatively operate in the manner shown in
Thus, the data reduction section 233 deletes t bits from each of the currently-received differential data DDp×(r−1)+2 to DDp×r to generate compressed differential data CDDp×(r−1)+2 to CDDp×r, each of which is composed of u bits. The respective generated compressed differential data CDDp×(r−1)+2 to CDDp×r are outputted to the packet assembling section 24.
The above description illustrates the case where the sign bits SBp×(r−1)+2 to SBp×r are “0”. On the other hand, in the case where the sign bits SBp×(r−1)+2 to SBp×r are “1”, the level selection section 2322 detects the reference bit position at which the first instance of “0” appears, as counted from the MSB of the currently-received maximum differential data MDDv. Furthermore, the level selection section 2322 selects one of the level values LV1 to LVt+1 which is assigned to the currently-determined reference bit position, and outputs this level value to both the data reduction section 233 and the packet assembling section 24.
As mentioned earlier, the packet assembling section 24 receives the pixel value XVp×(r−1)+1 from the splitter section 21 as well as the level value LVy from the level determination section 232. The packet assembling section 24 also receives the compressed differential data CDDp×(r−1)+2 to CDDp×r from the data reduction section 233. Based on these received data, the packet assembling section 24 assembles a data packet DPr as shown in
As shown in
After the data packet DPr is stored in the buffer section 31, the sending control section 32 receives the data packet DPr from the buffer section 31 and sends it onto the transmission path N. The data packet DPr, as an example of compressed data CDr, is transmitted through the transmission path N and then received by the receiver Rx, as shown in
The receiver Rx subjects the received data packet DPr to predetermined processing. Hereinafter, with reference to
Next, the reproduction processing for the image data TD which is performed by the receiver Rxa will be described in detail. The data packets DPr from the transmission path N are sequentially received by the receiving section 5. In the receiving section 5, the buffer section 51 stores the data packet DPr. The buffer section 51, which is only required to store the fixed-length data packets DPr, as is the case with the buffer section 31, contributes to the minimization of the delay time. After the buffering, the reception control section 52 receives the data packet DPr from the buffer section 51, and outputs the data packet DPr to the packet deassembling section 6.
As described above, the data packet DPr contains the pixel value XVp×(r−1)+1, the level value LVy, and the compressed differential data CDDp×(r−1)+2 to CDDp×r (see
As described earlier, the decompression/decoding section 7 receives the pixel value XVp×(r−1)+1, the level value LVy, and the compressed differential data CDDp×(r−1)+2 to CDDp×r, as shown in
More specifically, the near-instantaneous decompression section 71 in the decompression/decoding section 7 receives the level value LVy and the compressed differential data CDDp×(r−1)+2 to CDDp×r. The near-instantaneous decompression section 71 performs a near-instantaneous decompression to decompress the compressed differential data CDDp×(r−1)+2 to CDDp×r based on the received level value LVy. Thus, the near-instantaneous decompression section 71 generates decompressed differential data DDDp×(r−1)+2 to DDDp×r, which are sequentially outputted to the adder section 722.
To describe the above process more specifically, the near-instantaneous decompression section 71 recognizes the reference bit position RBLw in the received level value LVy. Specifically, as can be seen from
Once the reference bit position RBLw is determined, the near-instantaneous decompression section 71 knows the bit positions which were deleted from the differential data DDp×(r−1)+2 to DDp×r in the transmitter Tx. More specifically, as can be seen from
Having thus determined the deleted bit positions, the near-instantaneous decompression section 71 adds a bit(s) of predetermined values to each of the compressed differential data CDDp×(r−1)+2 to CDDp×r, thereby generating the decompressed differential data DDDp×(r−1)+2 to DDDp×r, which can be regarded as substantially the same as the aforementioned differential data DDp×(r−1)+2 to DDp×r.
More specifically, when receiving the level value LV1, as shown in
When receiving the level value LV2, as shown in
Thereafter, when the level values LV3 to LVt are received, similarly to when the level value LV2 is received, the near-instantaneous decompression section 71 places the MSBs of the respective compressed differential data CDDp×(r−1)+2, CDDp×(r−1)+3, . . . CDDp×r at the first bit, while placing the other (u−1) bits at the reference bit positions RBL3, RBL4, . . . RBLt through the (u+1)th bit. Furthermore, for those compressed differential data CDDp×(r−1)+2, CDDp×(r−1)+3, . . . CDDp×r which have positive values, the near-instantaneous decompression section 71 sets “0” at the second bit through the bit immediately before the reference bit positions RBL3, RBL4, . . . RBLt, “1” at the (u+2)th bit, and “0” at the (u+3)th through nth bits. For those compressed differential data CDDp×(r−1)+2, CDDp×(r−1)+3, . . . CDDp×r which have negative values, bits which are reverses of those set for the compressed differential data having positive values are set. As shown in
As shown in
More specifically, the delay section 721 in the DPCM decoding section 72 receives the pixel value XVp×(r−1)+1 from the packet deassembling section 6. The delay section 721 applies a delay amount DL2 to the received pixel value XVp×(r−1)+1, and outputs the resultant pixel value to the adder section 722 as a delayed pixel value LXVp×(r−1)+1. Herein, the delay amount DL2 is typically an amount of time corresponding to predetermined clocks. More specifically, the delay amount DL2 is prescribed to a value which ensures that the decompressed differential data DDDp×(r−1)+2 from the near-instantaneous decompression section 71 and the delayed pixel value LXVp×(r−1)+1 are received by the adder section 722 in the DPCM decoding section 72 substantially simultaneously.
The adder section 722 also sequentially receives sets of decompressed differential data DDDp×(r−1)+2 to DDDp×r. The adder section 722 adds the decompressed differential data DDDp×(r−1)+2 (which is received before any other decompressed differential data) and the concurrently-received delayed pixel value LXVp×(r−1)+1 to generate a decoded pixel value DXVp×(r−1)+2. The decoded pixel value DXVp×(r−1)+2 which has been thus generated is outputted to the image data reproduction section 8 as mentioned above, and is also fed back to the adder section 722. Next, the adder section 722 adds the decompressed differential data DDDp×(r−1)+3 from the near-instantaneous decompression section 71 and the concurrently-received decoded pixel value DXVp×(r−1)+2 to generate a decoded pixel value DXVp×(r−1)+3. The decoded pixel value DXVp×(r−1)+3 which has been thus generated is outputted to the image data reproduction section 8, and also fed back to the adder section 722. Thereafter, in a repetition of similar processing, the adder section 722 adds the decompressed differential data DDDp×(r−1)+4, DDDp×(r−1)+5, . . . DDDp×r−1 from the near-instantaneous decompression section 71 and the previously-generated decoded pixel values DXVp×(r−1)+3, DXVp×(r−1)+4, . . . DXVp×r−2 to generate decoded pixel values DXVp×(r−1)+4, DXVp×(r−1)+5, . . . DXVp×r−1, respectively, which are outputted to the image data reproduction section 8 and itself. Furthermore, the adder section 722 adds the decompressed differential data DDDp×r from the near-instantaneous decompression section 71 and the previously-generated decoded pixel value DXVp×r−1 to generate a decoded pixel value DXVp×r, which is outputted only to the image data reproduction section 8. Thus, the DPCM decoding section 72 generates the decoded pixel values DXVp×(r−1)+2 to DXVp×r, and outputs these decoded pixel values to the image data reproduction section 8.
As a result of the above-described processing, the image data reproduction section 8 sequentially receives q sets of decoded pixel values DXVp×(r−1)+2 to DXVp×r. Furthermore, prior to the arrival of each set of decoded pixel values DXVp×(r−1)+2 to DXVp×r, the pixel value XVp×(r−1)+1 is received from the packet deassembling section 6. In summary, the image data reproduction section 8 first receives the pixel value XV1 and the decoded pixel values DXV2 to DXVp. The pixel value XV1 and the decoded pixel values DXV2 to DXVp which are thus generated are substantially identical to the pixel value XV1 and the pixel values XV2 to XVp in the first line (along the width direction HD) of the image MG (see
As described above, according to the present embodiment of the invention, encoding and compression are performed for a fixed-length data block DBr which is composed of p pixel values XVp×(r−1)+1 to XVp×r arranged in line along the width direction HD. In other words, unlike in the conventional image compression schemes (MPEG or DVC) where image correlation on a macro block-by-macro block basis is utilized, the correlation between pixels arranged in line along the width direction HD is utilized to compress an image MG. As a result, the delay time which is incurred before the reproduced image data RTD is generated by the receiver Rx can be reduced.
In the above embodiment of the invention, the blocking section 1 is illustrated as generating data blocks DBr each composed of p received pixel values XV1 to XVi×j, where p is a divisor of i. However, i does not need to be an exact integer multiple of p, but any number a of pixel values XV may be left as a remainder. In such cases, as shown in
Based on the above consideration, the data compression section 2 can subject the padded data blocks to the same processing as that for the data blocks to generate compressed blocks. Furthermore, since the blocking section 1 only needs to add a padding bit sequence PBS for the pixel values XV arranged in line along the width direction HD, and there is no need to add any bit sequence along the longitudinal direction VD, the total number of extra bits to be added to the compressed data CDr is much smaller than that required for a typical conventional image compression scheme such as the MPEG scheme.
Next, a variant of the above-described data compression section 2a will be described with reference to
As shown in
In the DPCM encoding section 25, the near-instantaneous decompression section 251 receives the level value LVy from the level determination section 232 and the compressed differential data CDDp×(r−1)+2 to CDDp×r from the data reduction section 233. By performing a near-instantaneous decompression similar to that performed by the near-instantaneous decompression section 71 shown in
The delay section 253 sequentially receives the pixel values XVp×(r−1)+1 to XVp×r−1, as does the aforementioned delay section 221. The delay section 253 applies a delay amount DL3 to each of the received pixel values XVp×(r−1)+1 to XVp×r−1 to generate delayed pixel values LXVp×(r−1)+1 to LXVp×r−1, which are outputted to the adder section 252. The delay amount DL3 is prescribed to a value which ensures that the delayed pixel value LXVp×(r−1)×2 from the delay section 253 and the decompressed differential data DDDp×(r−1)+2 from the near-instantaneous decompression section 251 are received by the adder section 252 substantially simultaneously. Generally speaking, the delay amount DL3 is prescribed to a value which ensures that the delayed pixel values LXVp×(r−1)+2, LXVp×(r−1)+3, . . . LXVp×r−1 and the compressed differential data CDDp×(r−1)+2, CDDp×(r−1)+3, . . . CDDp×r−1 (which are generated on the basis of the same pixel values XVp×(r−1)+2, XVp×(r−1)+3, . . . XVp×r−1, respectively) are received by the adder section 252 substantially simultaneously.
The adder section 252 adds the delayed pixel values LXVp×(r−1)+2, LXVp×(r−1)+3, . . . LXVp×r−1 and the concurrently-received compressed differential data CDDp×(r−1)+2, CDDp×(r−1)+3, . . . CDDp×r−1 to generate added pixel values AXVp×(r−1)+2, AXVp×(r−1)+3, . . . AXVp×r−1, which is outputted to the subtraction section 222. Note that, when the delayed pixel value LXVp×(r−1)+1 from the delay section 253 is received by the adder section 252, the adder section 252 is receiving no input from the near-instantaneous decompression section 251, so that the adder section 252 passes the received delayed pixel value LXVp×(r−1)+1 through to the subtraction section 222 as the added pixel value AXVp×(r−1)+1.
As described earlier, the subtraction section 222 receives the pixel values XVp×(r−1)+2 to XVp×r from the splitter section 21, and the added pixel values AXVp×(r−1)+1 to AXVp×r−1 from the adder section 252. First, the subtraction section 222 subtracts the added pixel value AXVp×(r−1)+1 from the currently-received pixel value XVp×(r−1)+2, and generates a differential data DDp×(r−1)+2 representing a difference value therebetween, such that the generated differential data DDp×(r−1)+2 is in the same format as that shown in
For various reasons such as transmission errors, it might be possible for the receiver Rxa to miss, i.e., fail to correctly receive, all of the data packet DPr which have been sent from the transmitter Tx. Next, with reference to
The missing block recovery section 9 receives q sets of decoded pixel values DXVp×(r−1)+2 to DXVp×r from the decompression/decoding section 7. Prior to the arrival of each set of decoded pixel values DXVp×(r−1)+2 to DXVp×r, the missing block recovery section 9 receives the pixel value XVp×(r−1)+1 from the packet deassembling section 6. Thus, granted that there is no transmission error or the like, the missing block recovery section 9 will first receive the pixel value XV1 and the decoded pixel values DXV2 to DXVp. Subsequently, as the missing block recovery section 9 receives the qth set of decoded pixel values, all of the pixel value XVp×(r−1)+1 and the decoded pixel values DXVp×(r−1)+2 to DXVp×r which are necessary for the reproduction of the image MG are on hand. Thus, when the pixel value XVp×(r−1)+1 and the set of decoded pixel values DXVp×(r−1)+2 to DXVp×r, which together compose one frame, are all correctly received, the missing block recovery section 9 sequentially outputs these values to the image data reproduction section 8.
However, due to the aforementioned transmission error or the like, the receiver Rxb may miss or fail to receive the pixel value XVp×(r−1)+1 and the decoded pixel values DXVp×(r−1)+2 to DXVp×r to be generated from one or more data packets DPr. In the following description, any data packets DPr which the receiver Rxb fails to receive will be referred to as “missing data packets DDPr”. In such cases, the missing block recovery section 9 is able to virtually reproduce the pixel values XVp×(r−1)+1 to DXVp×r which have been contained in the missing data packet DDPr from the correctly-generated set of the pixel value XVp×(r−1)+1 and the decoded pixel values DXVp×(r−1)+2 to DXVp×r.
For example, as shown in
Next, the missing block recovery section 9 assigns an average value AVp×(r1−1)+1 of the pixel value XVp×((r1−i/p)−1)+1 and the XVp×((r1+i/p)−1)+1 as the pixel value XVp×(r1−1)+1. Moreover, the missing block recovery section 9 assigns an average value AVp×(r1−1)+2 of the decoded pixel value DXVp×((r1−i/p)−1)+2 and DXVp×((r1+i/p)−1)+2 as the decoded pixel value XVp×(r1−1)+2. Thereafter, similar average values AVp×(r1−1)+3 to AVp×r1 are assigned as the decoded pixel values DXVp×(r1−1)+2 to DXVp×r1. Instead of the pixel value XVp×(r−1)+1 and the decoded pixel values DXVp×(r−1)+2 to DXVp×r which were never obtained, the missing block recovery section 9 outputs the average values AVp×(r1−1)+1 to AVp×r1 which have virtually been reproduced in the above manner to the image data reproduction section 9. Based on the above-described average values AVp×(r1−1)+1 to AVp×r1 as well as the pixel value XVp×(r−1)+1 and the decoded pixel values DXVp×(r−1)+2 to DXVp×r which have been generated from all the data packets DPr except for the missing data packet DDPr1, the image data reproduction section 9 generates reproduced image data RTD2 which is similar to the aforementioned reproduced image data RTD1.
The reproduced image data RTD2 and the aforementioned reproduced image data RTD1 are substantially identical (i.e., indistinguishable to the human eye) for the following reasons. The transmitter Tx performs DPCM encoding and near-instantaneous compression on the basis of the data blocks DBr, so that the missing data packet DDPr does not exert any influence on the near-instantaneous decompression and DPCM decoding performed for the other data packets DPr. Furthermore, each data block DBr only includes p pixel values XVp×(r−1)+1 to DXVp×r. Therefore, the reproduced image data RTD2 containing the aforementioned average value AVp×(r1−1)+1 to AVp×r1 would hardly present any difference to the human eye from the reproduced image data RTD1.
As can be seen from the above, in accordance with the receiver Rxb, any missing data packets DDPr can be approximately recovered based on the other data packets DPr, thereby solving the aforementioned problem and generating reproduced image data RTD2 which is visually acceptable.
Alternatively, the missing block recovery section 9 may utilize, in order to deal with the missing data packet DDPr, the pixel value XVp×(r−1)+1 and a set of decoded pixel values DXVp×(r−1)+2 to DXVp×r from one line above or below (along the longitudinal direction VD) the missing data packet DDPr. Further alternatively, the missing block recovery section 9 may utilize the pixel value XVp×(r−1)+1 and a set of decoded pixel values DXVp×(r−1)+2 to DXVp×r from one pixel value right or left (along the width direction HD) of the missing data packet DDPr. In particular, in the case where moving pictures are represented by the image data TD, the pixel value XVp×(r−1)+1 and a set of decoded pixel values DXVp×(r−1)+2 to DXVp×r which are generated from a data packet DPr pertaining to a preceding and/or succeeding frame may be employed in the receiver Rxb.
Next, with reference to
Next, the second compression process which is performed by the data compression section 2c will be specifically described. Data blocks DBr from the aforementioned blocking section 1 (see
Hereinafter, an Hadamard transform will be described as an example of the aforementioned orthogonal transform, and the process performed by the orthogonal transform section 26 will be more specifically described. As described earlier, each data block DBr includes p pixel values XVp×(r−1)+1 to XVp×r. In the following description, p is conveniently assumed to be 16. Under this assumption, the orthogonal transform section 26 retains a (16×16) Hadamard transform matrix H as expressed by eq. 1 below:
For conciseness, the pixel values XV16×r−15 to XV16×r contained in the currently-received data block DBr are represented as a matrix expressed by eq. 2 below. The coefficients CF16×r−15 to CF16×r obtained through the Hadamard transform are represented as a matrix expressed by eq. 3 below.
x=[pixel value XV16×r−15, pixel value XV16×r−14, . . . , pixel value XV16×r]t eq. 2
y=[coefficient CF16×r−15, coefficient CF16×r−14, . . . , coefficient CF16×r]t eq. 3
In eq. 2 and eq. 3, “t” means transpose.
Under the definitions expressed by eq. 1 to eq. 3 above, the orthogonal transform section 26 multiplies the Hadamard transform matrix H by the matrix “x” beginning from the right side thereof, as expressed by eq. 4 below.
y=H×x/4 eq. 4
The resultant coefficient CF16×r−15 is an integer in the range from 0 to 255, and the resultant coefficients CF16×r−14 to CF16×r are integers in the range from −127 to 127. Therefore, these coefficients can all be expressed in n bits as mentioned above. According to the Hadamard transform matrix H expressed by eq. 1 above, the coefficient CF16×r−15 represents a component associated with the lowest frequency region. Likewise, the coefficients CF16×r−15 to CF16×r having greater suffix values represent components which are associated with respectively higher frequency regions.
As described earlier, the data reduction section 27 sequentially receives sets of coefficients CFp×(r−1)+1 to CFp×r. From each received set of coefficients CFp×(r−1)+1 to CFp×r, the data reduction section 27 deletes b coefficients CFp×r−b to CFp×r which represent components associated with predetermined high-frequency regions, thereby generating compressed coefficients CFp×(r−1)+1 to CCFp×r−(b−1). Herein, “b” is a natural number in the range from 1 to n.
According to the present embodiment of the invention, as shown in
According to the present embodiment of the invention, the next bit to the MSB and the least significant bit (hereinafter referred to as “LSB”) are deleted from both coefficients CF16×r−11 and CF16×r−10. Furthermore, the next bit to the MSB and the lower two bits are deleted from both coefficients CF16×r−9 and CF16×r−8. More specifically, as shown in
The present embodiment is not limited to the exemplary cases shown in
In the above-described manner, the data reduction section 27 generates a set of compressed coefficients CCFp×(r−1)+1 to CCFp×r−(b−1), which are outputted to the data sending section 3 as the aforementioned compressed block CBr.
As shown in
The receiver Rx subjects the received data packet DPr to predetermined processing to reproduce the image data TD. Hereinafter, a second implementation of the receiver Rx of
Next, the reproduction processing for the image data TD which is performed by the receiver Rxc having the aforementioned structure will be described in detail. In the data receiving section 5, the buffer section 51 stores a set of compressed coefficients CCFp×(r−1)+1 to CCFp×r−(b−1) from the transmission path N. The buffer section 51, which is only required to store the set of fixed-length compressed coefficients CCFp×(r−1)+1 to CCFp×r−(b−1), as is the case with the buffer section 31, contributes to the minimization of the delay time. After the buffering, the compressed coefficients CCFp×(r−1)+1 to CCFp×r−(b−1) are outputted to the bit decoding section 10 via the reception control section 52.
The bit decoding section 10 performs an inverse process of the process which is performed by the data reduction section 27 so as to generate the decompressed coefficients DCFp×(r×1)+1 to DCFp×r from the compressed coefficients CCFp×(r−1)+1 to CCFp×r−(b−1), which are then outputted to the inverse orthogonal transform section 11. Herein, the decompressed coefficients DCFp×(r−1)+1 to DCFp×r have such small differences from the coefficients CFp×(r−1)+1 to CFp×r that the decoded pixel values DXVp×(r−1)+1 to DXVp×r (described later) and the pixel values XVp×(r−1)+1 to XVp×r would hardly present any difference to the human eye. To describe the bit restoration process more specifically, an average value of a bit sequence of 1 and/or 0, i.e., values which can be expressed in two bits, is added after the LSB of each of the compressed coefficients CCF16×r−9 and CCF16×r−8. Furthermore, in the case where the MSB in the compressed coefficient CCF16×r−9 and/or CCF16×r−8 has a positive value, “0” is inserted after the MSB of that compressed coefficient; otherwise, “1” is inserted after the MSB of that compressed coefficient. Thus, the decompressed coefficients DCF16×r−9 and DCF16×r−8 are generated. Moreover, a bit “1” or “0” is added after the LSB of each of the compressed coefficients CCF16×r−11 and CCF16×r−10. Furthermore, in the case where the MSB in the compressed coefficient CCF16×r−11 and/or CCF16×r−10 has a positive value, “0” is inserted after the MSB of that compressed coefficient; otherwise, “1” is inserted after the MSB of that compressed coefficient. Thus, the decompressed coefficients DCF16×r−11 and DCF16×r−10 are generated. In order to deal with the coefficients CF16×r−7 and DCF16×r from which all bits have been deleted, the bit decoding section 10 generates decompressed coefficients DCF16×r−7 and DCF16×r whose eight bits are all “0”.
The inverse orthogonal transform section 11 performs an inverse orthogonal transform, i.e., an inverse process of the process which is performed by the orthogonal transform section 26, to multiply an inverse matrix of the aforementioned orthogonal transform matrix by the received decompressed coefficients DCFp×(r−1)+1 to DCFp×r, thereby generating a set of decoded pixel values DXVp×(r−1)+1 to DXVp×r. The resultant decoded pixel values DXVp×(r−1)+to DXVp×r, which hardly present any difference to the human eye as compared to the pixel values XVp×(r−l)+1 to XVp×r, are outputted to the image data reproduction section 8.
As described earlier, the present embodiment of the invention is directed to the case where an Hadamard transform is performed. Next, the processing to be performed by the inverse orthogonal transform section 11 in this specific case will be described more specifically. The following description also assumes that p=16. Under this assumption, the inverse orthogonal transform section 11 retains an inverse transform matrix H−1 of eq. 1 above. For conciseness, the currently-received decompressed coefficients DCFp×(r−1)+1 to DCFp×r are represented as a matrix expressed by eq. 5 below. The decoded pixel values DXVp×(r−1)+1 to DXVp×r which are obtained through the inverse transform of the Hadamard transform are represented as a matrix expressed by eq. 6 below.
y=[decompressed coefficient CF16×r−15, . . . , decompressed coefficient CF16×r]t eq. 5
z=[decoded pixel value XV16×r−15, . . . , decoded pixel value XV16×r]t eq. 6
In eq. 5 and eq. 6, “t” means transpose.
Under the definitions expressed by eq. 5 and eq. 6 above, the inverse orthogonal transform section 11 multiplies the matrix “y” by the inverse transform matrix H−1 of the Hadamard transform matrix H beginning from the right side thereof, as expressed by eq. 7 below.
z=H−1×y×4 eq. 7
As a result of the above processing, the image data reproduction section 8 sequentially receives q sets of decoded pixel values DXVp×(r−1)+1 to DXVp×r. The image data reproduction section 8 generates a reproduced image data RTD similar to that shown in
As described above, in accordance with the second implementation of the data compression section 2c, too, encoding and compression are performed for the fixed-length data block DBr composed of p pixel values XVp×(r−1)+1 to XVp×r arranged in line along the width direction HD, it is possible to minimize any delay time elapsing prior to the generation of the reproduced image data RTD in the receiver Rxc.
Although the above illustration is directed to the case where an Hadamard transform is performed in the orthogonal transform section 26, the present invention is not limited thereto. Alternatively, the orthogonal transform section 26 may perform a DCT or discrete sine transform (DST).
“First Application”
In recent years, there has been plenty of work directed to the research and development of driving assistant systems for assisting a driver in his/her driving of a vehicle by capturing an image of the surroundings of the vehicle via image capturing devices and providing such an image to the driver. Next, a driving assistant system TS1 incorporating the above-described transmitter Tx and receiver Rx will be described.
The image capturing devices 13, each of which is disposed so as to be able to capture an image of an area in the rear of the vehicle Vur, capture images of respectively different regions in the rear of the vehicle Vur, and generate captured image data CTD representing the captured images MG (see
As shown in
Each transmitter Tx performs the processing described in any of the earlier embodiments of the invention for the partial image data PTD it receives, thereby generating compressed block CBr. The respective compressed blocks CBr are transmitted to the receiver Rx via the transmission path N.
The receiver Rx performs the processing described in any of the earlier embodiments of the invention for each received compressed block CBr, thereby generating reproduced partial image data RPTD. Herein, as will be appreciated from the foregoing description, each reproduced partial data RPTD represents a reproduced partial image RMG which is substantially the same as the image MG represented by each partial image data PTD (as shown in
The image synthesis section 15 performs a synthesis process for both received reproduced partial image data RPTD to generate a merged image data MTD representing a single synthesized image MMG which is composed of the two partial images PMG. The merged image data MTD is outputted to the display section 16.
The display section 16 subjects the received merged image data MTD to display processing, thereby providing the aforementioned synthesized image MMG to the driver of the vehicle Vur.
Thus, in accordance with the present driving assistant system TS1, each image processing section 14 generates a partial image data PTD which is required on the receiver Rx side, so that the amount of data which is transmitted over the transmission path N can be minimized.
Moreover, in accordance with the transmitters Tx and the receiver Rx, as described earlier, the delay time which is incurred after the generation of captured image data CTD by the image capturing devices 13 and before the display processing of the merged image data MTD by the display section 16 can be minimized. By incorporating the transmitters Tx and the receiver Rx having such characteristics in the driving assistant system TS1, a driver is enabled to grasp the surroundings of the vehicle Vur in real time. As a result, the driver can drive the vehicle Vur with increased safety.
Next, with reference to
In
Δd=SP×DT eq. 8
Assuming that the vehicle Vur moves at a constant velocity, as seen from eq. 8 above, the distance Δd increases as the delay time DT increases. If the delay time DT=0, then the distance Δd=0, so that the points P0 and P1 will coincide. If there is substantial delay time DT, on the other hand, the vehicle Vur may collide into the obstacle BST before the display section 16 displays the sight of the collision in vain. From this perspective, it can be seen how useful it is to incorporate transmitters Tx and a receiver Rx having a sufficiently small delay time DT in the driving assistant system TS1.
In the above application, the driving assistant system TS1 comprises two sets of image capturing devices 13, image processing sections 14, and image transmitters Tx. However, the present invention is not limited to such a configuration. The driving assistant system TS1 may comprise one or more set of such elements. Although the above image capturing devices 13 are illustrated as being capable of capturing images of objects at the rear of the vehicle Vur, the image capturing devices 13 may alternatively be fixed on the vehicle Vur so as to be capable of capturing images of objects at the front and/or sides of the vehicle Vur, as necessary.
While the missing block recovery section 9 in the earlier-described embodiment is illustrated as approximately recovering the pixel values XVp×(r−1)+1 to DXVp×r which have been contained in a missing data packet DDPr from a correctly-generated set of the pixel value XVp×(r−1)+1 and the decoded pixel values DXVp×(r−1 )+2 to DXVp×r, it may be more preferable in the driving assistant system TS1 not to reproduce any missing data packets DDPr, but simply “black out” that portion, for example, thereby warning the driver of the occurrence of the missing data packets DDPr, because the driver is in need of accurate information concerning the surroundings of the vehicle Vur.
“Second Application”
In the field of FA (Factory Automation), there has been some research and development work on remote control systems for acting on an object via remote control. Next, a remote control system TS2 incorporating the above-described transmitter Tx and the receiver Rx will be described.
The image capturing device 17, which is disposed in the neighborhood of the object TG, captures an image of the object TG, and generates captured image data CTD representing the captured image. The generated captured image data CTD is outputted to the transmitter Tx. The transmitter Tx subjects the received captured image data CTD to the processing which has been described in any of the foregoing embodiments of the invention, thereby generating compressed block CBr as described above. The compressed block CBr is transmitted to the receiver Rx via the transmission path N.
The receiver Rx subjects the received compressed block CBr to the processing which has been described in the foregoing embodiment of the invention, thereby generating reproduced image data RTD. Herein, as will be appreciated from the foregoing description, the reproduced image data RTD represents an image of the object TG which is substantially the same as the image represented by the captured image data CTD. The reproduced image data RTD is outputted to the display section 18. The display section 18 subjects the received reproduced image data RTD to display processing, thereby providing an image representing the object TG to an operator.
While checking on the display section 18, the operator operates the control section 19 to instruct as to what sort of action to exert on the object TG. In response to the instruction from the control section 19, the control data generation section 110 generates control data CTLD representing an action to be exerted on the object, which is outputted to the control data sending section 111. The control data sending section 111 sends the received control data CTLD onto the transmission path N, which then is transmitted to the control data receiving section 112. The manipulator section 113 exerts an action on the object TG in accordance with the control data CTLD which is received by the control data receiving section 112.
As described above, in accordance with the transmitter Tx and the receiver Rx, the delay time which is incurred after the generation of captured image data CTD by the image capturing device 17 and before the display processing by the display section 18 can be minimized. By incorporating the transmitters Tx and the receiver Rx having such characteristics in the remote control system TS2, an operator is enabled to grasp the situation concerning the object TG in real time. As a result, the operator can properly act on the object TG from a remote place.
Although the above-described application assumes that the operator operates the control section 19 while looking at the display section 18, it is also possible to automatically control the operation of the robot (or the manipulator section 113) by applying image recognition techniques.
While the invention has been described in detail, the foregoing description is in all aspects illustrative and not restrictive. It is understood that numerous other modifications and variations can be devised without departing from the scope of the invention.
Number | Date | Country | Kind |
---|---|---|---|
2000-283015 | Sep 2000 | JP | national |
Number | Name | Date | Kind |
---|---|---|---|
4805017 | Kaneko et al. | Feb 1989 | A |
5051988 | Kawahigashi et al. | Sep 1991 | A |
5103306 | Weiman et al. | Apr 1992 | A |
5212549 | Ng et al. | May 1993 | A |
5243428 | Challapali et al. | Sep 1993 | A |
5272527 | Watanabe | Dec 1993 | A |
5281079 | Lemelson | Jan 1994 | A |
5424778 | Sugiyama et al. | Jun 1995 | A |
5432555 | Park | Jul 1995 | A |
5502491 | Sugiyama et al. | Mar 1996 | A |
5631744 | Takeuchi et al. | May 1997 | A |
5659362 | Kovac et al. | Aug 1997 | A |
5670935 | Schofield et al. | Sep 1997 | A |
5684536 | Sugiyama et al. | Nov 1997 | A |
5737481 | Gushima et al. | Apr 1998 | A |
5805293 | Mochizuki | Sep 1998 | A |
5926210 | Hackett et al. | Jul 1999 | A |
5995241 | Nakatani et al. | Nov 1999 | A |
6124995 | Kim | Sep 2000 | A |
6850647 | Gough et al. | Feb 2005 | B1 |
20040085447 | Katta et al. | May 2004 | A1 |
Number | Date | Country |
---|---|---|
0 586 225 | Mar 1994 | EP |
0 862 138 | Sep 1998 | EP |
0 949 818 | Oct 1999 | EP |
7-280911 | Oct 1995 | JP |
8-102953 | Apr 1996 | JP |
9-130787 | May 1997 | JP |
8810544 | Dec 1988 | WO |
9638319 | Dec 1996 | WO |
Number | Date | Country | |
---|---|---|---|
20020048322 A1 | Apr 2002 | US |