The present application is directed toward the field of image transfer over a network, and more particularly towards techniques for efficient transfer of source data, such as medical images, between a server and a client.
It has become more common for images to be stored, distributed, and viewed in digital form using computer technology. In the medical field, Picture Archival and Communication Systems or PACS have been in widespread use. In a typical PACS application, image data obtained by imaging equipment, such as CT scanners or MRI scanners, is stored in the form of computer data files. The size of a data file for an image varies depending on the size and resolution of the image. For example, a typical image file for a diagnostic-quality chest X-ray is on the order of 10 megabytes (MB). The image data files are usually formatted in a “standard” or widely accepted format. In the medical field, one widely used image format is known as DICOM. The DICOM image data files are distributed over computer networks to specialized viewing stations capable of converting the image data to high-resolution images on a display.
In medical imaging applications, images must be displayed at high resolution so that image details having potential diagnostic significance are visible. Also, in medical imaging applications, concurrent viewing of multiple images, captured over time, is desirable in order to enable the detection of changes that occur over a time period. The need for high resolution and multiple views translates into a need for high network bandwidth, large storage capacity, and significant processing power at the viewing stations. The traditional digitally encoded medical images, used in medical applications, usually require powerful and expensive computer systems to archive, distribute, manipulate, and display the medical images. Consequently, many current imaging systems are very expensive.
A technique for efficiently distributing large images over a network, such as medical images, has been developed by Dr. Paul Chang, M.D., and Carlos Bentancourt at the University of Pittsburgh. This technique operates in a client-server environment to deliver image data from the server to the client, as the image data is needed at the client (i.e., a just in time data delivery mechanism). To implement this just in time data delivery mechanism, a dynamic transfer syntax generates a flexible hierarchical representation of an image for storage at the server. The hierarchical representation consists of coefficients produced by a wavelet transform which can be used at the server to obtain image data from a particular area of an image. The transform coefficients preserve the data corresponding to a particular area of the image. To view portions of the image at the client, the client issues requests to the server for transform data that includes coefficient coordinates to identify coefficients in the hierarchical representation. The server provides the requested transform data to the client. The client then reconstructs the portion of the image from a combination of data already locally cached, if necessary, and the provided transform data. A complete description of the dynamic transfer syntax is contained in U.S. Pat. No. 6,711,297 entitled “Methods and Apparatus for Dynamic Transfer of Image Data”, inventors Paul Joseph Chang and Carlos Bentancourt, issued Mar. 23, 2004, which is expressly incorporated herein by reference.
In one embodiment of the system of the Chang '297 patent described above, the wavelet transform uses fixed point kernels with integer coefficient results in order to achieve lossless data presentation. However, this limitation to lossless compression generally precludes compression ratios of greater than 2 to 3 to 1 for image data. In some cases, the ability to obtain greater compression ratios is not necessary if the application has adequate bandwidth. However, in other applications, such as teleradiology applications, it is desirable to achieve compression ratios beyond those ratios generated with purely lossless compression. Another application class where this capability is useful is when the data sets are so large as to make transferring the entire lossless data set impractical (such as very large CT data sets—with thousands of images). It is desirable to increase compression ratios for limited bandwidth sites while preserving the ability to recover the original lossless data.
The present application discloses a system which efficiently transfers source data from a server to at least one client.
In one exemplary embodiment, the source data is decomposed into a hierarchical representation consisting of coefficients. The coefficients are quantized and compressed to form a compressed hierarchical representation, consisting of compressed coefficients. The compressed hierarchical representation may be used to reconstitute a “visually lossless” version of the source data for display at the client.
In order to reconstitute a visually lossless version of the source data at the client, the client requests compressed coefficients from the server, which has access to the compressed hierarchical representation. In response to the client request, the server transfers the compressed coefficients specified in the request. The client decompresses the compressed coefficients, and performs an inverse wavelet transform to generate the visually lossless version of the source data.
The full fidelity or completely lossless version of the source data can be reconstructed from the visually lossless version of the source data combined with residual data. An inverse wavelet transform is performed on the compressed coefficients to generate inverted visually lossless data and the residual data is generated from a difference between the inverted visually lossless data and the original source data. The generated residual data is stored in a partitioned and compressed format augmenting the visually lossless hierarchical representation.
To generate a full fidelity or completely lossless image, the client requests from the server residual data that corresponds to the visually lossless portion. The server has access to the generated residual data. The server transmits the residual data to the client, and the client uses the received residual data to upgrade the visually lossless version of the source data to a full fidelity or completely lossless version of the source data.
The compressed hierarchical representation and the residual data are optionally organized into partitions. For this option, the client requests and the server responses correspond to the respective partitions that organize the compressed hierarchical representation and the residual data.
As described below in more detail with respect to at least one exemplary embodiment, a “visually lossless” compression technique is used to deliver data in applications with limited bandwidth. In general, the term “visually lossless” refers to an image that has undergone a lossy compression technique but the loss of image information is not perceived by the human eye or the loss is deemed acceptable by the viewer. As described fully below, high quality visually lossless images are generated for rapid dissemination over limited bandwidth networks. Residual data is generated so that completely lossless images (e.g., original lossless images) can be reconstructed from visually lossless images at the client (e.g., a diagnostic station).
The compression techniques of the present disclosure have application for use in imaging, including medical imaging applications (e.g., mammogram, X-Ray, MRI, CATSCAN, etc.).
In
Source data processing 120 for generating a compressed hierarchical data representation includes a first step of “decomposition processing”. In general, “decomposition processing” refers to the use of a “transform” to generate a hierarchical representation including coefficients. The coefficients that are generated from the decomposition processing described above, are invertible to the original source data 110. The transform coefficients preserve the data corresponding to a particular area of the image. These transform coefficients along with coefficient coordinates that identify the coefficients in the hierarchical representation are hereinafter referred to collectively as “transform data”. Each level of the generated hierarchical data representation 220 is sufficient to reconstruct the source data at a given resolution. One specific example of decomposition processing is wavelet transform processing 210, which is described in more detail below in connection with
Appropriate hardware and software for decomposition processing are described more fully in U.S. Pat. No. 6,711,297, which is incorporated herein by reference. Such hardware and software are herein collectively referred to as decomposition processing means.
Source data processing 120 for generating a compressed hierarchical data representation further comprises software or hardware with executable instructions, or a combination thereof, for partitioning the hierarchical data presentation and quantizing and compressing the coefficients of the hierarchical representation, as described below in more detail in connection with
As further shown in
As shown in more detail in
In the present embodiment, a wavelet transform with preferably floating point kernels, is used to generate a hierarchical data representation 220 consisting of floating point coefficients. The hierarchical data representation 220 can be used at the server 100 to obtain image data from a particular area of an image. A wavelet transform with floating point kernels produces higher quality low resolution image data than a transform with fixed point kernels (for a given filter length). Also, a wavelet transform, with floating point kernels, is used for improved signal to noise ratios in compression applications. Thus, a wavelet transform, with floating point kernels, enables lossy compression at a high signal to noise ratio but at a level that is visually lossless. This approach permits attaining compression ratios for images between 3 and 20 or more to 1, and therefore is suitable for general distribution of images. The relevance of the benefits of a wavelet transform with floating point kernels to compression applications will be better understood from the discussion below of subsequent quantization and compression processing 230.
The related wavelet transform for the decomposition function may be generally defined as:
ŴI=C,
wherein: Ŵ defines the transform function for a transform that utilizes floating point kernels; I represents the source data (e.g., image); and C represents the transformed data (e.g., transformed image).
The use of the wavelet transform to generate the hierarchical data representation provides a scalable solution for transferring different portions of a large data file. When the source data is decomposed into the hierarchical data representation, sub-images and sub-resolution images, in the granularity of partitions of coefficients, can be extracted directly from the disk of the server. The image server then transmits only the physical coefficients required to reconstruct the exact size of the desired image for display at the client. Accordingly, a multi-resolution format is implicit in the hierarchical data representation.
The wavelet transform function generates mathematically independent information among the levels of the hierarchical representation. Accordingly, there is no redundant information in the hierarchical data representation. Thus, hierarchical data representation is not merely multiple replications of the source data at different resolutions, but it contains unique data at the different levels of the hierarchical representation. The mathematically independent nature of the wavelet transform permits minimizing the amount of data transferred over a network, by requiring only the transfer of “additional data” not yet transferred to the computer from the server necessary to construct a given image. The wavelet transform is lossless, in that no data from the original source data is lost in the decomposition into the hierarchical data representation.
In other embodiments, the wavelet transform may be used to generate multi-spectral (e.g., color) transform data. In general, multi-spectral transform data aggregates multiple components of the source image into a vector for the transform data. Also, the multi-spectral transform data may comprise any type of attribute for binding to the source data.
A “wavelet transform” has been described herein by example only for the purpose of providing a full and detailed written description of at least one specific embodiment. Decomposition processing is not by any means limited to wavelet transform processing and related wavelet filters and methods. Other methods for decomposition processing can be used which would be known or understood by one having ordinary skill in the art or which would inherently fall within the scope or spirit of this disclosure.
In general, to generate the hierarchical data representation 220, the wavelet transform is applied across the columns of the source data 110, and then this wavelet transform is applied across the rows of the source data 110 (i.e., or vice versa). The selection of the wavelet transform for wavelet transform processing 210 is dependent upon the particular characteristics of the hierarchical data structure desired. Each level of the hierarchical data structure is generated by recurring on the low-pass, “low low”, of the previous higher level. This recursion continues until a predetermined size is obtained. For example, in one embodiment, the lowest level in the hierarchical data structure for a source image consists of a low-pass component of approximately 128×128. However, any granularity of resolution may be generated for use in a hierarchical data structure without deviating from the spirit or scope of the invention. Also, any quadrant may be used in the recursion process with any desired transform.
Turning back to
The partitioned transform data is quantized and compressed in quantization and compression processing 230 (
{circumflex over (Q)}ŴI={circumflex over (Q)}R,
wherein, R represents a block of coefficients in the transform data, and {circumflex over (Q)} represents the quantized function. For this expression, if truncation toward zero is used, the quantization function may be expressed as follows.
If (R>0), then
else, (i.e., if R<0),
wherein, Q is a floating point value and the resultant is consistently truncated (i.e., either towards or away from zero), such that RQ is a multiple of the quantization bin width, Q.
The coefficients can be quantized in accordance with the corresponding level of the decomposition. Thus, for this embodiment, there is one quantization bin width, Q, for each level, but any number of quantization bin widths could be used. For this embodiment, the quantization values may be quantized as follows:
wherein, “L” and “Level” represent the level of the decomposition for the “N” level example, and Q1, Q2, Q3 and QN represent quantization bin widths for the respective levels. Although the above expression represents quantization based on levels of the decomposition, any combination of different quantization values may be used to quantize a set of coefficients without deviating from the spirit or scope of the invention.
Also, in a further scenario where coefficients are quantized based on the level of the decomposition, the high pass coefficients may be aggregated into a quantized coefficient block as follows:
Accordingly, for this example, the high-energy components for each coefficient block of the transform are quantized for compression processing. Although the above example quantizes components of the transform, any block of transform data may be quantized for subsequent compression.
The quantized coefficient block, RQ, is compressed to generate the transmission coefficient block in accordance with the expression:
RT=ÊRQ;
wherein, Ê defines the compression expression, and RT defines the compressed coefficient block for transmission. In general, the compression function, Ê, comprises a lossless coder. Any encoding rule may be used for the compression function, Ê, without deviating from the spirit or scope of the invention. For example, the compression function, Ê, may be implemented with a Rice encoding function or a Huffman encoding function, with or without run length encoding, arithmetic encoding, etc. As one example, the low low (“LL”) component of a level is encoded using Rice encoder, and quantized data (which typically include runs of zeros) is encoded using an arithmetic encoder.
As described more fully below, the compressed coefficient block, RT, is transmitted to the client. At the client, the inverse quantization and decoder functions are executed. This operation may be expressed as:
R={circumflex over (Q)}−1Ê−1RT.
The inverse transform, Ŵ−1 is performed on the coefficients, R (and any additionally required data already at the client), to generate the original data.
In the present embodiment, the coefficients and residual data are organized in partitions.
As shown in
As shown in
During set up of a communications protocol between the server and the client, the client may specify one or more source data or images (i.e., compressed coefficients) for the subsequent client request—server transfer process. In one embodiment, the server-client implements “event” communication services offered through the Common Object Request Broker Architecture (“CORBA”). However, any network protocol may be used to implement client—server network communications without deviating from the spirit or scope of the invention.
As further shown in
In response to the request, the server 100 extracts the partitions of compressed coefficients from the source data file 400, identified in the request, and transfers the compressed partitions of coefficients to the client 150. Thereafter, the client 150 can reconstruct a visually lossless version of the portion of the image using the compressed partitions of coefficients. Reconstruction is achieved through software (e.g., computer executable instructions), hardware such as a dedicated processor with executable processor instructions, or a combination thereof, operating on client 150 for performing the functions described herein. Such software and hardware are herein collectively referred to as visually lossless image reconstruction means.
If the client desires a full fidelity or completely lossless version of a portion of a source image stored at a client and the client has level 1 coefficient data, the client generates a request for partitions of residual data for that portion of the image, including image identification and pixel coordinates. Client 160 generates requests for residual data for level 0 coefficients. The server 100, in response to the request, extracts the partitions of residual pixel data for level 0, and transfers the residual data to client 160. In turn, the client 160 reconstructs the portion of the image, at full fidelity (a completely lossless version of the portion of the image), using the previously generated pixel data and the residual pixel data. Reconstruction is achieved through software (e.g., computer executable instructions) or hardware such as a dedicated processor with executable processor instructions, or a combination thereof, operating on client 160 for performing the functions described herein. Such software and hardware are herein collectively referred to as completely lossless image reconstruction means.
Those of skill would further appreciate that the various illustrative logical blocks, modules and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein may be implemented or performed with a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, (e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration).
Although the present invention has been described in terms of specific exemplary embodiments, it will be appreciated that various modifications and alterations might be made by those skilled in the art without departing from the spirit and scope of the invention.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/IB08/55198 | 12/10/2008 | WO | 00 | 7/26/2010 |
Number | Date | Country | |
---|---|---|---|
61016058 | Dec 2007 | US |