In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments that may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments, and it is to be understood that other embodiments may be utilized and that logical, mechanical, electrical and other changes may be made without departing from the scope of the embodiments. The following detailed description is, therefore, not to be taken as limiting the scope of the invention.
In various embodiments a method of providing an integrated interface for image zoom and montage comprising the steps of: (a) generating a montage using at least one neighboring image while zooming-out an image; and (b) displaying the zoomed-out image with the montage. The zooming-in in its general sense refers to expanding, enlarging or magnifying the image, and zooming-out refers to compressing, contracting, or reducing the image.
In another embodiment the invention provides zooming-in of an image from a zoomed-out image using a different tile. This would allow the user to rapidly transition between “bird's eye” views and “close-up” views, which could aid the user's understanding of the image data. Also in another embodiment, the invention provides fusing an image using a montage generated based on at least one neighboring image.
Thus various embodiments of integrating zoom with montage, and a display system providing the integrated montage and zoom, are disclosed. However, it should be noted that the invention is not limited to this or any particular application or environment. Rather, the technique may be employed in a range of applications, any display systems, such as a computer output surface or projecting mechanism that shows text and graphic images to a computer user, using a cathode ray tube (“CRT”), liquid crystal display (“LCD”), light-emitting diode, gas plasma, or other image projection technology, to mention a few. The invention also discloses a method and system for having real time interaction of zoom or pan interface.
In an embodiment, the neighboring images include images selected from nearby spatial locations of the zoomed-out image. While zooming-out an image, the montage is generated using the near-by images of the zoomed-out image.
In another embodiment, the neighboring images include images at nearby time points of the zoomed-out image, from a dynamic acquisition. In dynamically acquired images, the images having near-by time points or the images acquired simultaneously shall be used as neighboring images.
In yet another embodiment the neighboring images include images from similar locations of an object being imaged using different imaging protocols. If the invention uses different imaging systems for capturing the image, the neighboring images may be selected from different images obtained using different protocols and using different imaging systems. However the part being imaged will be the same. For example, the image of an “arm” of a patient is captured using different medical imaging devices. The different medical imaging device may include, X-ray imaging device, CT, MR, PET, etc, but not limited to these. Thus the slices captured for “arm” from different imaging devices may be used as neighboring images.
In yet another embodiment the neighboring images include images from different patients, but related according to some metric such as correlation.
Thus a montage is generated using neighboring images, the neighboring images being selected from nearby spatial locations of the zoomed out image, nearby time points of the zoomed out image from a dynamic acquisition, or different locations of an object being imaged using different imaging protocols or different patients. The montage is generated while the image is zoomed out. The zoomed-out image is displayed with the montage thus generated.
At block 120, a zoomed-in image is displayed using a selected image from the montage. The invention provides a unique way of zooming-in an image using a different tile than the zoomed-out tile. Exemplary steps involved in displaying the image are clearly illustrated in
At block 220, the zoomed-out image is displayed with the montage as the background. At block 230, the montage generated is panned. The term “pan,” “panning,” or the like means for a system user to traverse an image on a display in the horizontal and/or vertical direction (2-D roaming) using an interface device. Panning is performed to select an image from the montage. The image may be selected based on the choice of the user or any other predetermined specifications. At block 240, the selected image is placed at the center of the montage. At block 250, a zoomed-in image is displayed using the image selected from the montage.
The image database is stored in the memory 322. It should be understood that any type of computer accessible memory or storage device capable of storing the desired amount of data and/or code may be used. Moreover, the memory or storage device may comprise one or more memory devices, such as magnetic or optical devices, of similar or different memory types, which may be capable to store the montage generated by the display generator. The memory also may be stored with software necessary for the operation of the display system.
The means for generating montage further comprises a display generator 324. The display generator 324 is electronically coupled to the image database in order to retrieve the image data stored in the image data records. The display generator 324 performs selecting the images from the image data based on the user instructions and performs arranging the images into a montage.
The processor 320 further comprises means for panning and zooming-in the image 326. The means for panning and zooming-in the image 326 comprises a user interface for interacting with the display system 300. The user interface may comprise a standard computer keyboard or mouse. The interface utilizes user-friendly point-and-click interface software, which may be stored in the memory. The interface will preferably permit the user to select and view any of the images stored in the image database. The user will typically be permitted to control the selection and arrangement process that the display generator performs during operation of the display system 300.
The display device 330 is coupled with the processor 320 for displaying the processed image or the montage generated. The display device may be any device capable of displaying images, including a computer output surface or projecting mechanism that shows text and graphic images to a computer user, using a cathode ray tube (“CRT”), liquid crystal display (“LCD”), light-emitting diode, gas plasma, or other image projection technology, to mention a few. While zooming-out the image, the processor generates the montage using at least one neighboring image and the display device 330 displays the zoomed-out image with the montage generated. As the user zooms-out more, each image tile shrinks smaller so that more tiles fit in the montage.
The montage generated may be panned by the user or automatically to select an image. The image may be selected by the user or by the system by default. The selected image is placed at the center of the montage. Subsequent zooming-in enlarges the tiles around this new center. Thus the display system displays the zoomed-in image using the image selected from the montage. Zooming-in can be performed to the extent of vanishing the montage from view, and this allows the user to examine a different image slice than the one being viewed prior to zooming out.
In an embodiment of the invention, the processor may fuse the image with the montage generated. This is performed by using the montage stored in the memory 322. The montage may be fused or blended with the image at the display generator 324 and the display generator 324 may feed the blended image to the display device 330 for display. The blending is performed based on the opacity values selected by the user. A parameter called alpha specifies the opacity of a foreground image that is overlaid on a background image.
In another embodiment of the invention, the zoomed-out images are reformatted from a scale-space version of the image rather than the original. Scale space is computed through a process of pyramiding that consists of the two (2) steps of smoothing the image with a Gaussian kernel followed by down sampling. Smoothing the image with the Gaussian kernel is used to prevent spurious artifacts. This act is performed at the processor 320.
In an embodiment of the invention, the user has the option of conveniently resetting the zoom/pan, so that single image fills the view.
While the invention has been described with reference to preferred embodiments, those skilled in the art will appreciate that certain substitutions, alterations and omissions may be made to the embodiments without departing from the spirit of the invention. Accordingly, the foregoing description is meant to be exemplary only, and should not limit the scope of the invention as set forth in the following claims.