Patent Grant
9891784
This application claims the benefit of Korean Patent Application No. 10-2015-0004452, filed on Jan. 12, 2015, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein in its entirety by reference.
1. Field
One or more exemplary embodiments relate to an apparatus and method of displaying a medical image, and more particularly, to an apparatus and method of displaying a three-dimensional medical image that is generated by rendering volume data of an object.
2. Description of the Related Art
An ultrasound system having properties of being non-invasive and non-destructive has been widely used in a medical field to obtain information about the inside of an object. Since the ultrasound system may provide a doctor with a high resolution image of the inside of an object without a surgical operation for directly incising the object to observe the inside of the object, the ultrasound system is very important in the medical field.
In general, the ultrasound system in a state in which a probe is in contact with a surface of the object transmits an ultrasound signal to the object and receives an ultrasound signal (hereinafter, referred to as an echo signal) reflected from the object. The ultrasound system generates an ultrasound image of the object based on the echo signal received through the probe and displays a generated ultrasound signal through a display.
For example, the ultrasound system may generate and display a brightness mode (B Mode) image that represents the strength of the echo signal reflected from the object in terms of brightness or a Doppler mode (D Mode) image that represents a Doppler component extracted from the echo mode in terms of color or waveform. Also, the ultrasound system may generate volume data based on the echo signal reflected from the object and may generate and display a three-dimensional ultrasound image by rendering the volume data.
In providing a three-dimensional ultrasound image, a general ultrasound system generates and displays only three-dimensional ultrasound images of predetermined cross-sections. Accordingly, since a user receives only the three-dimensional ultrasound images rendered in a predetermined direction with regard to the predetermined cross-sections, it is difficult to precisely observe a desired portion in detail.
One or more exemplary embodiments include an apparatus and method of displaying a medical image that is generated by rendering volume data in a plurality of directions to observe the inside of an object at various angles.
Additional aspects will be set forth in part in the description which follows and, in part, will be apparent from the description, or may be learned by practice of the presented exemplary embodiments.
According to one or more exemplary embodiments, a method of displaying a medical image includes displaying a first image that is generated by rendering volume data of an object in a first direction, displaying on the first image a viewer tool for generating a second image, wherein the viewer tool indicates a section of the object, generating the second image by rendering sub-volume data included in the volume data in a second direction which is different from the first direction and indicated by the viewer tool, and displaying at least a part of the second image.
The generating of the second image may include determining the second direction based on at least one of a position, an angle, and a shape of the viewer tool displayed on the first image.
The viewer tool may include direction information indicating the second direction, and the generating of the second image may include determining the section of the object based on at least one of a position, an angle, and a shape of the viewer tool displayed on the first image, and determining which one of two directions perpendicular to the section is the second direction, based on the direction information.
The displaying of the viewer tool may include displaying viewing angle information indicating a range of the object represented by the second image that is generated based on the viewer tool, and the generating of the second image may include determining the second direction based on at least one of a position, an angle, and a shape of the viewer tool displayed on the first image, and generating the second image by rendering sub-volume data included in the volume data in the second direction indicated by the viewer tool based on the viewing angle information.
The viewer tool may include direction information indicating which one of two directions that are perpendicular to the section of the object indicated by the viewer tool is the second direction.
The first image may be generated by rendering in the first direction one of two pieces of sub-volume data obtained by dividing the volume data based on a first section of the object, and the second image may be generated by rendering in the second direction one of two pieces of sub-volume data obtained by dividing the volume data based on a second section of the object, in which the second section crosses the first section.
The first image may be generated by rendering in the first direction one of two pieces of sub-volume data obtained by dividing the volume data based on a first section of the object, and the second image may be generated by rendering in the second direction the other one of the two pieces of sub-volume data obtained by dividing the volume data based on the first section of the object, in which the second direction is opposite to the first direction.
The second image may be displayed on the viewer tool.
The second image may be generated by rendering the sub-volume data with a rendering parameter that is different from a rendering parameter applied to the first image.
The method may further include enlarging or contracting the second image based on a user input.
The method may further include changing a display mode based on a user input such that the viewer tool and the second image are not displayed on the first image.
The method may further include changing at least one of a position, an angle, and a shape of the viewer tool based on a user input, generating a third image by rendering the volume data based on the change of the at least one of the position, the angle, a size, and the shape of the viewer tool, and displaying the third image instead of the second image.
According to one or more exemplary embodiments, an apparatus for displaying a medical image includes a volume data acquirer acquiring volume data of an object, an image processor generating a first image by rendering the volume data in a first direction, a display displaying the first image and displaying on the first image a viewer tool for generating a second image, the viewer tool indicating a section of the object, and a controller determining a second direction based on the viewer tool, in which the image processor generates the second image by rendering sub-volume data included in the volume data in a second direction which is different from the first direction and indicated by the viewer tool, and the display displays at least a part of the second image.
The controller may determine the second direction based on at least one of a position, an angle, and a shape of the viewer tool displayed on the first image.
The viewer tool may include direction information indicating the second direction, and the controller may determine the section of the object based on at least one of a position, an angle, and a shape of the viewer tool displayed on the first image; and determine which one of two directions perpendicular to the section is the second direction, based on the direction information.
The display may further display viewing angle information indicating a range of the object represented by the second image that is generated based on the viewer tool, the controller may determine the second direction based on at least one of a position, an angle, and a shape of the viewer tool displayed on the first image, and the image processor may generate the second image by rendering sub-volume data included in the volume data in the second direction indicated by the viewer tool based on the viewing angle information.
The viewer tool may include direction information indicating which one of two directions that are perpendicular to the section of the object indicated by the viewer tool is the second direction.
The first image may be generated by rendering in the first direction one of two pieces of sub-volume data obtained by dividing the volume data based on a first section of the object, and the second image may be generated by rendering in the second direction one of two pieces of sub-volume data obtained by dividing the volume data based on a second section of the object, in which the second section crosses the first section.
The first image may be generated by rendering in the first direction one of two pieces of sub-volume data obtained by dividing the volume data based on a first section of the object, and the second image may be generated by rendering in the second direction the other one of the two pieces of sub-volume data obtained by dividing the volume data based on the first section of the object, in which the second direction being opposite to the first direction.
The second image may be displayed on the viewer tool.
The second image may be generated by rendering the sub-volume data with a rendering parameter that is different from a rendering parameter applied to the first image.
The apparatus may further include a user input unit that receives a user input, in which the display displays the second image which is enlarged or contracted based on the user input.
The apparatus may further include a user input unit that receives a user input, in which the display changes a display mode based on the user input such that the viewer tool and the second image are not displayed on the first image.
The apparatus may further include a user input unit that receives a user input, in which the display may display the viewer tool of which at least one of a position, an angle, and a shape is changed based on the user input, the image processor may generate a third image by rendering the volume data based on the viewer tool of which at least one of the position, the angle, a size, and the shape of the viewer tool is changed, and the display may display the third image instead of the second image.
According to one or more exemplary embodiments, a non-transitory computer-readable storage medium having stored thereon a program, which when executed by a computer, performs a method for display a medical image, in which the method may include displaying a first image that is generated by rendering volume data of an object in a first direction, displaying on the first image a viewer tool for generating a second image that is rendered in a second direction different from the first direction, wherein the viewer tool indicates a section of the object, generating the second image by rendering sub-volume data included in the volume data in the second direction indicated by the viewer tool, and displaying at least a part of the second image.
These and/or other aspects will become apparent and more readily appreciated from the following description of the exemplary embodiments, taken in conjunction with the accompanying drawings in which:
Reference will now be made in detail to exemplary embodiments, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to like elements throughout. In this regard, the present exemplary embodiments may have different forms and should not be construed as being limited to the descriptions set forth herein. Accordingly, the exemplary embodiments are merely described below, by referring to the figures, to explain aspects of the present description. As used herein, the term “and/or” includes any and all combinations of one or more of the associated listed items. Expressions such as “at least one of,” when preceding a list of elements, modify the entire list of elements and do not modify the individual elements of the list.
The present invention will now be described more fully with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown. Throughout the drawings, like reference numerals denote like elements. In the following description, when detailed descriptions about related well-known functions or structures are determined to make the gist of the present invention unclear, the detailed descriptions will be omitted herein.
When a part may “include” a certain constituent element, unless specified otherwise, it may not be construed to exclude another constituent element but may be construed to further include other constituent elements. Terms such as “˜portion”, “˜unit”, “˜module”, and “˜block” stated in the specification may signify a unit to process at least one function or operation and the unit may be embodied by hardware, software, or a combination of hardware and software. Also, as a computer software command to embody the present invention, hardware, software, or a combination of hardware and software may be used instead of a programmed processor/controller. Accordingly, the present invention is not limited by a specific combination of hardware and software.
In the present specification, when a constituent element “connects” or is “connected” to another constituent element, the constituent element contacts or is connected to the other constituent element directly or through at least one of other constituent elements. Conversely, when a constituent element is described to “directly connect” or to be “directly connected” to another constituent element, the constituent element should be construed to be directly connected to another constituent element without any other constituent element interposed therebetween.
In the present specification, an “object” may be a living thing or a non-living thing displayed on an image. Also, the object may be a part of a human and may include organs such as the liver, the heart, the womb, the brain, a breast, the abdomen, etc., or a fetus. Also, the object may include any one section of a human body.
Also, in the present specification, a “user” may be a medical expert including a doctor, a nurse, a clinical pathologist, a sonographer, or a medical imaging expert, but the present invention is not limited thereto.
Also, in the present specification, a “medical image” may include all images for diagnosis and treatment of a disease by which sectional and volume data of a part of a human body are restored from signals projected to the part, for example, a computed tomography (CT) image, a magnetic resonance imaging (MRI) image, or a positron emission tomography (PET), in addition to an ultrasound image. Also, the “medical image” may include all of two-dimensional (2D) image of a section of an object, a three-dimensional (3D) image of a space of the object, and a stereo image which enables a viewer watching an image to feel a sense of depth. The present inventive concept is described below in detail with reference to the accompanying drawings.
As illustrated in
Also, the general ultrasound system provides 3D ultrasound images that are generated by rendering volume data in a predetermined direction. For example, the general ultrasound system provides only 3D ultrasound images that are rendered in a direction in which the inside of a human body is viewed from the outside. Accordingly, the user who received the 3D ultrasound images from general ultrasound system has difficulty precisely observing the portion to observe at various angles.
Accordingly, various exemplary embodiments of the present inventive concept provide an apparatus and method of displaying a medical image which provides a user interface (UI) to set a direction in which volume data is rendered so that the user may observe the inside of the object at various angles.
The medical image display apparatus 200 according to the present exemplary embodiment may be embodied not only in a card type but also in a mobile type, as an apparatus for generating and displaying by rendering volume data. The medical image display apparatus 200 may be an apparatus developed for a medical use, but not limited thereto, and may include all apparatuses that may process and display an image. For example, the medical image display apparatus 200 may include a fax viewer such as a Picture Archiving and Communication System (PACS) viewer, hand-carried cardiac ultrasound (HCU) equipment, smartphones, laptop computers, personal digital assistants (PDAs), tablet PC, etc.
The medical image display apparatus 200 according to the present exemplary embodiment may include a volume data acquirer 210, an image processor 220, a display 230, and a controller 240. The medical image display apparatus 200 in
The volume data acquirer 210 may acquire volume data of the object.
The term “volume data” signifies data including information, for example, a position, a color, a density, etc., about a plurality of voxels forming a 3D space. A voxel may be a unit for defining a point in a 3D space. The volume data of the object may include space information about the space of an object or clinical information such as an anatomical shape of a tissue or an organ included of the object. For example, the volume data of the object may be ultrasound data acquired from an ultrasound echo signal reflected from the object in response to the ultrasound signal transmitted to the object.
In an example, the volume data acquirer 210 may acquire volume data by forming volume data of the object by using the ultrasound data acquired by the object. In another example, the volume data acquirer 210 may acquire volume data from an external device or server. In another example, the volume data acquirer 210 may acquire volume data that is previously stored.
The image processor 220 may generate a first image by rendering the volume data in a first direction. The first image may be a 2D image or a 3D image.
The term “rendering” is technology to form and display a projection image of a 3D discretely sampled data set such as the volume data. For example, a method of rendering volume data includes a ray casting method in which a part of the object is detected by casting an imaginary light ray toward the object.
The expression “rendering the volume data in a predetermined direction” signifies generating a 3D image obtained by projecting the volume data in a predetermined direction. In other words, the expression “rendering the volume data in a predetermined direction” signifies generating an image of the object by using an imaginary camera that watches the object in a predetermined direction in an imaginary space where the object is located.
The first direction may be a direction that is preset as a default value or determined based on a user input in the medical image display apparatus 200.
Also, the image processor 220 may further generate a second image by rendering the volume data in a second direction that is determined through a viewer tool displayed through the display 230.
The image processor 220 may generate a plurality of images by rendering the volume data in different directions with respect to one section of the object.
For example, the image processor 220 may generate the first image by rendering in the first direction one of two pieces of sub-volume data obtained by dividing the volume data based on the first section of the object. The image processor 220 may generate the second image by rendering the other sub-volume data in the second direction that is the opposite direction to the first direction.
Alternatively, the image processor 220 may generate a plurality of images by rendering the volume data in different directions based on different sections of the object. For example, the image processor 220 may generate the first image by rendering in the first direction one of two pieces of sub-volume data obtained by dividing the volume data based on the first section of the object.
The image processor 220 may generate the second image by rendering the volume data based on a second section of the object shown by the viewer tool. The image processor 220 may generate the second image by rendering in the second direction one of two pieces of sub-volume data obtained by dividing the volume data based on the second section that crosses the first section.
The image processor 220 may generate the second image by using a rendering parameter that is different from a rendering parameter applied to the first image. The rendering parameter may include opacity, level of detail, brightness, and a rendering method. For example, the image processor 220 may generate the first and second images by rendering the volume data in different methods. Alternatively, the image processor 220 may render the volume data by using different brightness parameters so that an average brightness of the second image is brighter than the brightness of the first image.
Also, the image processor 220 may generate the second image, as if the user may receive an image reflected from a concave mirror or a convex mirror inserted into the inside of the object. The image processor 220 may provide various ways of distortion effects, by applying various filters to the second image generated by rendering the volume data. The distortion effect signifies giving a sense of refraction to an image by enlarging or contracting at least a part of the image, or moving the positions of pixels.
The display 230 displays a medical image generated by the medical image display apparatus 200. The display 230 may display not only the medical image generated by the medical image display apparatus 200 but also various pieces of information processed by a medical diagnosis apparatus 100 through a graphical user interface (GUI).
The display 230 may include at least one of a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a 3D display, and an electrophoretic display. The medical image display apparatus 200 may include two or more of the display 230 according to an embodiment shape of the medical image display apparatus 200.
The display 230 may display the first image generated by the image processor 220. The display 230 may display on the first image the viewer tool for generating the second image rendered in the second direction that is different from the first direction. The display 230 may display at least a part of the second image generated by the image processor 220 based on the viewer tool. For example, the display 230 may generate the first image by rendering the volume data in a direction from the outside of the object toward the inside of the object, and the second image by rendering the volume data in a direction from the inside of the object toward the outside of the object.
The display 230 according to the present exemplary embodiment may display on the first image the viewer tool for observing the object from a different viewpoint from the first image of the object. The image processor 220 may further generate the second image by rendering the volume data of the object in the second direction that is different from the first direction, based on the viewer tool displayed on the first image. The display 230 may show through the second image an inner portion of the object that is not displayed through the first image, by further displaying the second image.
According to the viewer tool according to an example embodiment, the user may variously set a viewpoint to observe the inside of the object. Accordingly, according to the present exemplary embodiment, the user may observe the inside of the object at various angles.
The viewer tool may include direction information indicating the second direction to render the volume data. For example, the medical image display apparatus 200 may indicate the direction information by using at least one of characters, signs, shapes, and colors. The viewer tool, which is displayed on the first image, may show the second section that crosses, is parallel to, or is the same as the first section that the first image represents. The second direction may be perpendicular to the second section. The medical image display apparatus 200 may generate the second image by rendering the volume data in a direction perpendicular to the second section of the object determined by the viewer tool. The direction information included in the viewer tool may indicate in which of two directions perpendicular to the second section the volume data is rendered.
The viewer tool may show a section of the object to determine two pieces of sub-volume data, by dividing the volume data. The medical image display apparatus 200 may determine which of the two pieces of sub-volume data determined based on the viewer tool is to be rendered, based on the direction information included in the viewer tool.
The display 230 may display at least a part of the second image generated by the image processor 220 based on the viewer tool. The medical image display apparatus 200 may select at least a partial area of the second image based on the position, size, and shape of the viewer tool on the first image. The display 230 of the medical image display apparatus 200 may display the selected partial area on a screen.
The display 230 may separately display the first image and the second image in different areas on the screen. Alternatively, the display 230 may display the first image and the second image in one area so that the second image is displayed overlapping the first image. For example, the display 230 may display the second image on the viewer tool.
Also, the display 230 may further display viewing angle information about a range of the object indicated by the second image generated based on the viewer tool. The image processor 220 may generate the second image by rendering the sub-volume data included in the volume data in the second direction indicated by the viewer angle, based on the viewing angle information.
The controller 240 controls an overall operation of the medical image display apparatus 200 and controls the volume data acquirer 210, the image processor 220, and the display 230 to display a medical image based on an operation state of the medical image display apparatus 200.
The controller 240 may determine the second direction to render the volume data based on the viewer tool. The controller 240 may determine the second direction based on at least one of the position, angle, and shape of the viewer toll displayed on the first image.
The controller 240 may determine one section of the object based on at least one of the position, angle, and shape of the viewer tool displayed on the first image. The controller 240 may determine one of two direction perpendicular to the section as the second direction, based on the direction information presented by the viewer tool.
Also, as illustrated in
The user input unit 250 signifies a unit for receiving an input of data to control the medical image display apparatus 200. The user input unit 250 may receive a user input to adjust a medical image to be displayed through the medical image display apparatus 200. For example, the user input unit 250 may receive a user input to enlarge or contract a displayed image, a user input to change a display mode, a user input to change at least one of the position, angle, size, and shape of the viewer tool, and an user input to adjust a range of the object indicated by the second image generated based on the viewer tool. The user may change a position of an area to observe of the object or an angle to observe the inside of the object, by changing at least one of the position, angle, size, and shape of the viewer tool.
For example, the display 230 may enlarge or contract a displayed image, based on the use input received through the user input unit 250. The display 230 may display the second image that is enlarged or contracted based on the user input.
Alternatively, the display 230 may change the display mode based on the user input received through the user input unit 250. The display 230 may change the display mode to prevent the viewer tool and the second image from being displayed on the first image, based on the user input.
Alternatively, the display 230 may display the viewer tool of which at least one of the position, angle, size, and shape is changed based on the user input to the viewer tool. As at least one of the position, angle, size, and shape of the viewer tool is changed, the image processor 220 may change a section of the object that is a reference for generating the second image. Also, the image processor 220 may change the second direction for rendering the volume data to generate the second image, as at least one of the position, angle, size, and shape of the viewer tool. The image processor 220 may generate a third image by rendering the volume data based on the viewer tool in which at least one of the position, angle, size, and shape is changed. The third image may be displayed instead of the second image.
Also, the user input unit 250 may receive from the user display setting information to set a method of displaying medical images on the screen. The display setting information may include information about setting of the number of divided screens to display an image or the position or size of an area where the image is displayed. The display 230 may change a screen structure based on the display setting information received through the user input unit 250.
Also, the user input unit 250 may receive from the user the viewing angle information about the range of the object indicated by the second image generated based on the viewer tool. The display 230 may change a spatial area of the object indicated by the second image to increase or decrease based on the viewing angle information received through the user input unit 250.
The user input unit 250 may include a hardware structure such as a keypad, a touch panel, a touch screen, a trackball, a jog switch, etc., but not limited thereto. The user input unit 250 may further include various input devices such as an electrocardiogram measurement module, a respiration measurement module, a voice recognition sensor, a gesture recognition sensor, a fingerprint recognition sensor, an iris recognition sensor, a depth sensor, a distance sensor, etc.
The communication unit 260 is connected to a network 1030 in a wired or wireless manner and communicates with an external device or server. For example, the communication unit 260 may receive the volume data of the object from the external device or server, and transfer the received volume data to the volume data acquirer 210.
The memory 270 stores various pieces of information processed by the medical image display apparatus 200. For example, the memory 270 may store medical data related to a medical image or diagnosis of the object, or an algorithm or program performed in the medical image display apparatus 200.
The memory 270 may be embodied by various types of storage media such as flash memory, hard disk, EEPROM, etc. Also, the memory 270 may run a web storage or a cloud server that performs a storing function on the web. The memory 270 may store the volume data of the object, and transfers the stored volume data to the volume data acquirer 210.
The volume data acquirer 210 according to the present exemplary embodiment may transmit an ultrasound signal to the object and receive an ultrasound echo signal from the object, thereby acquiring the volume data.
As illustrated in
The probe 211 may transmit an ultrasound signal to the object according to a control signal transmitted by the ultrasound transceiver 213 and receive an ultrasound echo signal reflected from the object, thereby forming a receiving signal. The probe 211 may transmit the receiving signal to the ultrasound transceiver 213.
The ultrasound transceiver 213 supplies a driving signal to the probe 211 so that the probe 211 transmits an ultrasound signal to the object. Also, the ultrasound transceiver 213 forms ultrasound image data by using the receiving signal received from the probe 211.
The volume data acquirer 210 may form the volume data by combining the ultrasound image data about a plurality of sections of the object formed by the ultrasound transceiver 213.
As illustrated in
The medical image display apparatus 200 may generate the first image 510 showing the inside of the object based on a first section 420. The medical image display apparatus 200 may determine two pieces of sub-volume data 411 and 413 by dividing the volume data 410 based on the first section 420 of the object.
The medical image display apparatus 200 may generate the first image 510 of
The medical image display apparatus 200 according to the present exemplary embodiment may provide the viewer tool for setting a region of interest (ROI) so that the user may observe the ROI to observe at various angles.
As illustrated in
For example, the user may select a reference position to set the viewer tool 610 by locating the viewer tool cursor at a desired position on the first image 510 and selecting the position. The user may set the size and shape of the ROI by moving the viewer tool cursor 615 to another position from the reference position. The medical image display apparatus 200 may set the at least one of the position, size, and shape of the viewer tool 610 based on a user input to move the viewer tool cursor 615. When the ROI is set, the viewer tool 610 may display the position, size, and shape of the set ROI on the first image 510.
The medical image display apparatus 200 may determine the second direction to render the volume data based on the viewer tool 610 set on the first image 510. For example, as illustrated in
The medical image display apparatus 200 may generate the second image by rendering the other one of the two sub-volume data 411 and 413 in the second direction. The medical image display apparatus 200 may generate the second image based on at least one of the position, angle, size, and shape of the viewer tool 610. For example, the medical image display apparatus 200 may generate the second image showing tissues 403 and 404 in the sub-volume data 413 corresponding to the position, size, angle, and shape of the viewer tool 610.
For example, the medical image display apparatus 200 may determine the second section of the object based on the viewer tool 610. The second section may be the same as or different from the first section that is used as a reference for generating the first image 510. The medical image display apparatus 200 may determine the second section of the object based on the position, angle, size, and shape of the viewer tool 610.
The medical image display apparatus 200 may divide the volume data into two sub-volume data based on the second section, and generate the second image by rendering one of the two sub-volume data in the second direction. For example, the second direction may be perpendicular to the second section, and may be a direction determined based on direction information included in the viewer tool 610.
The medical image display apparatus 200 may select at least a partial area of the second image on the first image 510, based on the position, size, and shape of the viewer tool 610. The medical image display apparatus 200 may display the selected partial area on the screen.
As illustrated in
As illustrated in
The medical image display apparatus 200 may determine the second direction to render the volume data 410 based on the viewer tool 610. For example, as illustrated in
The medical image display apparatus 200 may generate the second image by rendering the volume data 410 in the second direction based on the second section 920.
For example, the medical image display apparatus 200 may generate the second image by dividing the volume data 410 into two pieces of sub-volume data 911 and 913 based on the second section 920 and rendering one of the two sub-volume data 911 and 913, that is, the sub-volume data 913, in the second direction. For example, the second direction is perpendicular to the second section and may be determined based on direction information included in the viewer tool 610.
The medical image display apparatus 200 may select at least a partial area of the second image on the first image 510 based on the position, size, and shape of the viewer tool 610. The medical image display apparatus 200 may display a selected partial area on the screen.
As illustrated in
As illustrated in
Referring to
In S1110, the medical image display apparatus 200 may display the first image generated by rendering the volume data of the object in the first direction.
The medical image display apparatus 200 may divide the volume data into two pieces of sub-volume data based on the first section of the object. The medical image display apparatus 200 may generate the first image by rendering one of the two sub-volume data in the first direction. The first image may be perpendicular to the first section.
In S1120, the medical image display apparatus 200 may display the viewer tool on the first image. The viewer tool may be displayed on the first image to generate the second image rendered in the second direction that is different from the first direction.
The viewer tool may include the direction information indicating the second direction to render the volume data. The viewer tool may show the second section of the object to determine the two sub-volume data as the medical image display apparatus 200 divides the volume data. The viewer tool may show the second direction to render one of the two sub-volume data to generate the second image.
The medical image display apparatus 200 may further display the viewer tool and the viewing angel information about the range of the object displayed on the second image that is generated based on the viewer tool.
In S1130, the medical image display apparatus 200 may generate the second image by rendering the sub-volume data included in the volume data in the second direction shown by the viewer tool.
First, the medical image display apparatus 200 may determine the second direction based on the viewer tool. For example, the medical image display apparatus 200 may determine the second direction based on at least one of the position, angle, and shape of the viewer tool.
The medical image display apparatus 200 may determine the second section of the object based on at least one of the position and shape of the viewer tool displayed on the first image. The second section may be the same section as the first section or a section crossing the first section. The medical image display apparatus 200 may determine one of the two directions perpendicular to the second section, as the second direction, based on the direction information of the viewer tool. For example, when the second section is the same section as the first section, the medical image display apparatus 200 may determine a direction opposite to the first direction as the second direction.
Next, the medical image display apparatus 200 may generate the second image by rendering the volume data in the determined second direction.
The medical image display apparatus 200 may generate the second image by dividing the volume data into two pieces of sub-volume data based on the second section and rendering one of the two sub-volume data in the second direction.
The medical image display apparatus 200 may generate the second image by rendering the sub-volume data included in the volume data in the second direction indicated by the viewer tool based on the viewing angle information. The medical image display apparatus 200 may generate the second image representing a spatial area of the inside of the object determined based on the viewing angle information changed by the user input.
The medical image display apparatus 200 may generate the second image by rendering the volume data using a rendering parameter that is different from the rendering parameter applied to the first image.
In S1140, the medical image display apparatus 200 may display at least a part of the second image.
The medical image display apparatus 200 may separately display the first image and the second image in different areas on the screen. Alternatively, the medical image display apparatus 200 may display the first image and the second image in one area so that the second image is displayed overlapping the first image. For example, the medical image display apparatus 200 may display the second image on the viewer tool.
Also, the medical image display apparatus 200 may further include receiving a user input to change at least one of the position, angle, size, and shape of the viewer tool. The medical image display apparatus 200 may generate the third image by rendering the volume data based on the viewer tool in which at least one of the position, angle, size, and shape is changed. The medical image display apparatus 200 may display the third image instead of the second image.
As described above, the medical image display apparatus 200 according to the present exemplary embodiment may enable the user to observe the ROI at various angles based on the viewer tool. Also, the medical image display apparatus 200 according to the present exemplary embodiment may enable the user to observe the ROI simultaneously at various angles, by displaying, on a single screen, 3D images rendered in a plurality of directions with respect to the ROI of the object.
Also, the medical image display apparatus 200 according to the present exemplary embodiment may generate an image of the ROI by rendering the volume data in a predetermined rendering method. Accordingly, according to the present exemplary embodiment, the image of the ROI is highlighted so as to be distinguished from the other images.
Also, the medical image display apparatus 200 according to the present exemplary embodiment may display on a single screen a main image generated by rendering the volume data in a predetermined direction and a sub-image generated by rendering the volume data in a direction opposite to a predetermined direction. Accordingly, according to the present exemplary embodiment, the user may easily compare the main image and the sub-image while simultaneously observing the same section of interest in different directions.
As illustrated in
The medical image display apparatus 200 may display the viewer tool 610 on the first image 1210. The viewer tool 610 may display an arrow 611 as the direction information.
The medical image display apparatus 200 may generate a second image 1220 by determining the second direction based on the viewer tool 610 and rendering the volume data in the second direction. The medical image display apparatus 200 may determine the second section of the object based on the position and shape of the viewer tool 610. In
The medical image display apparatus 200 may generate the second image 1220 by rendering the volume data in the direction indicated by the arrow 611 of the viewer tool 610. A point 1221 in the second image of
Also, the medical image display apparatus 200 may select at least a partial area of the second image 1220 based on the size and shape of the viewer tool 610 and display only the selected partial area on the screen. In
Alternatively, the medical image display apparatus 200 according to the present exemplary embodiment may change at least one of the position, angle, size, and shape of the viewer tool 610. The medical image display apparatus 200 may update the second image according to a change in the viewer tool 610, by rendering the volume data based on the viewer tool 610 that is changed according to the user input.
As illustrated in
Also, the medical image display apparatus 200 may change the position of the viewer tool 610 based on the user input to move the imaginary wheels 1321, 1322, and 1323 along the axes. As the position of the viewer tool 610 is changed, the medical image display apparatus 200 may change the position of the second section that is a reference for generating the second image. Or, as the position of the viewer tool 610 is changed, the medical image display apparatus 200 may change a selection area selected in the second image. The medical image display apparatus 200 may display on the screen the selection area changed in the second image.
According to the present exemplary embodiment, the viewer tool 610 may show a section that divides a spatial area of the object that the volume data represents. The medical image display apparatus 200 may generate the second image by rendering the volumes data based on the second section of the object that the viewer tool 610 indicates. The medical image display apparatus 200 may generate the second image by rendering in the second direction one of two pieces of sub-volume data obtained by dividing the volume data based on the second section.
As illustrated in
The GUI 1301 may include an image 1330 indicating the volume data and an image 1340 indicating a section corresponding to the viewer tool 610. The GUI 1301 may provide information about which section used for dividing the volume data into the sub-volume data the second image generated based on the viewer tool 610 corresponds. The GUI 1301 may include an auxiliary line 1341 that provides information about the position and angle of the section corresponding to the viewer tool 610 in the volume data.
The medical image display apparatus 200 may change at least one of the position, angle, and shape of the viewer tool 610 based on the user input to change at least one of the position, angle, and shape of the image 1340 indicating a section corresponding to the viewer tool 610.
For example, when the angle of the viewer tool 610 is changed by using the image 1340 of
Also, the medical image display apparatus 200 may change the position of the viewer tool 610 based on the user input to move the image 1340. As the position of the viewer tool 610 is changed, the medical image display apparatus 200 may change the position of the second section that is a reference for generating the second image or change a selection area selected in the second image. The medical image display apparatus 200 may display on the screen a changed selection area in the second image.
As illustrated in
The medical image display apparatus 200 may display the viewer tool 610 on the first image 1410. The viewer tool 610 may display the arrow 611 as the direction information. The medical image display apparatus 200 may determine the second direction based on the viewer tool 610 and generate a second image 1420 by rendering the volume data in the second direction. The medical image display apparatus 200 may determine the second section based on the position and shape of the viewer tool 610. In
The medical image display apparatus 200 may generate the second image 1420 by rendering the volume data in the direction indicated by the arrow 611 of the viewer tool 610. A point 1421 of the second image 1420 of
Also, the medical image display apparatus 200 may select at least a partial area of the second image 1420 based on the size and shape of the viewer tool 610, and display only a selected partial area on the image. In
As illustrated in
The medical image display apparatus 200 may display the viewer tool 610 on the first image 1430. The viewer tool 610 may display the arrow 611 as the direction information. The medical image display apparatus 200 may determine the second direction based on the viewer tool 610 and generate a second image 1440 by rendering the volume data in the second direction. The medical image display apparatus 200 may determine the second section based on the position and shape of the viewer tool 610. In
The medical image display apparatus 200 may generate the second image 1440 by rendering the volume data in the direction indicated by the arrow 611 of the viewer tool 610. A point 1441 of the second image 1440 of
Also, the medical image display apparatus 200 may select at least a partial area of the second image 1440 based on the size and shape of the viewer tool 610, and display only a selected partial area on the image. In
As illustrated in
Alternatively, the medical image display apparatus 200 according to the present exemplary embodiment may change the range of the object, that is, a viewing angle, indicated by the second image generated based on the viewer angle based on the user input.
As illustrates in
Referring to
The medical image display apparatus 200 may change the viewing angle information based on the user input. As illustrated in
The medical image display apparatus 200 may change the viewing angle information that is a range of the object indicated by the second image generated based on the viewer tool 610, based on the user input through the GUI 1530.
As illustrated in
Alternatively,
As illustrated in
Also, according to the present exemplary embodiment, the medical image display apparatus 200 mage generate the second image 1620 by rendering the volume data by using rendering parameters that are different from the rendering parameters applied to the first image 1610. For example, the medical image display apparatus 200 may generate the second image 1620 so that the second image 1620 that is an image of the ROI is highlighted compared to a surrounding area. Accordingly, the medical image display apparatus 200 according to the present exemplary embodiment may enable the user to more intuitively and precisely observe the ROI.
Alternatively, the medical image display apparatus 200 according to the present exemplary embodiment may change the display mode based on the user input.
As illustrated in
Referring to
Also, when the slide button 1731 is located in the middle of the GUI 1730, the medical image display apparatus 200 may display the first image 1710 and the second image 1720 by blending the same. For example, the medical image display apparatus 200 may display an image obtained by blending the first image 1710 and the second image 1720, by adjusting transparency of at least one of the first image 1710 and the second image 1720, based on the position of the slide button 1731.
Alternatively, the medical image display apparatus 200 according to the present exemplary embodiment may contract or enlarge a displayed image based on the user input.
As illustrated in
Referring to
The medical image display apparatus 200 according to the present exemplary embodiment may be applied to an ultrasound system for generating and displaying an ultrasound image. Accordingly, a method of displaying a medical image according to an exemplary embodiment may be performed by an ultrasound system 2000 illustrated in
For example, the volume data acquirer 210 of
Also, the probe 211 of
The respective elements included in the ultrasound system 2000 of
According to the present exemplary embodiment, the probe 2010, the ultrasound transceiver 2020, and the image processor 2030 may acquire ultrasound image data about an object 20. According to the present exemplary embodiment, the ultrasound image data may be 2D ultrasound image data or 3D ultrasound image data about the object 20.
According to the present exemplary embodiment, a transmitter 2021 included in the ultrasound transceiver 2020 may include a pulse generator 2023, a transmission delaying unit 2024, and a pulser 2025, as illustrated in
The transmitter 2021 may supply a driving signal to the probe 2010. The pulse generator 2023 generates a pulse to form a transmission ultrasound wave according to a predetermined pulse repetition frequency (PRF). The transmission delaying unit 2024 applies to the pulse a delay time to determine transmission directionality. Each pulse to which the delay time is applied corresponds to each of a plurality of piezoelectric vibrators included in the probe 2010. The pulser 2025 applies the driving signal, or a driving pulse, to the probe 2010 at a timing corresponding to each pulse to which the delay time is applied.
According to the present exemplary embodiment, a receiver 2022 included in the ultrasound transceiver 2020 may include an amplifier 2026, an analog-to-digital converter (ADC) 2027, a reception delaying unit 2028, and a summing unit 2029, as illustrated in
The receiver 2022 may generate ultrasound data by processing a response signal received from the probe 2010. The amplifier 2026 amplifies the response signal for each channel. The ADC 2027 performs analog-to-digital conversion on the amplified response signal. The reception delaying unit 2028 applies a delay time to determine reception directionality to the digitally converted response signal. The summing unit 2029 sums the response signal processed by the reception delaying unit 2028, thereby generating ultrasound image data.
According to the present exemplary embodiment, the probe 2010 may include a part of whole of elements included in the transmitter 2021 and the receiver 2022 of
The image processor 2030 generates an ultrasound image through a scan conversion process on the ultrasound image data generated by the ultrasound transceiver 2020. Alternatively, the ultrasound image may include not only a gray scale image obtained by scanning the object in an amplitude mode “A mode”, a brightness mode “B mode”, and a motion mode “M mode”, but also a Doppler image representing a moving object by using a Doppler effect. The Doppler image may include a blood flow Doppler image indicating a flow of blood, which is referred to as a color Doppler image, a tissue Doppler image indicating a movement of a tissue, and a spectral Doppler image indicating a movement velocity of the object as a waveform.
A B-mode processor 2033 processes the ultrasound image data by extracting a B mode component from the ultrasound image data. An image generator 2032 may generate an ultrasound image in which strength of a signal is presented in brightness based on the M mode component extracted by the B-mode processor 2033.
The image processor 2030 may include an elasticity processor (not shown). An elasticity processor (not shown) processes elasticity data by extracting a velocity component of a shear wave, for example, a shear ware coefficient, from the elasticity data. The image generator 2032 may generate an elasticity image in which the velocity of a shear wave is represented in color, based on the velocity component of a shear wave, for example, a shear ware coefficient, extracted by the elasticity processor.
Also, a Doppler processor 2034 may extract a Doppler component from the ultrasound image data. The image generator 2032 may generate a Doppler image in which a movement of the object is represented in color or waveform based on the extracted Doppler component.
The image generator 2032 according to the present exemplary embodiment may generate a 3D ultrasound image through a volume rendering process on the volume data, and generate an elasticity image that images a degree of deformation of the object 20 according to pressure.
Furthermore, the image generator 2032 may provide various pieces of additional information in text or graphics on the ultrasound image. For example, the image generator 2032 may add at least one of annotations related to a part or whole of the ultrasound image to the ultrasound image. In other words, the image generator 2032 may analyze the ultrasound image and recommend at least one of annotations related to a part or whole of the ultrasound image based on a result of the analysis. Also, the image generator 2032 may add additional information corresponding to the ROI selected by the user to the ultrasound image.
Alternatively, the image processor 2030 may extract the ROI from the ultrasound image by using an image processing algorithm. For example, the image processor 2030 may extract the ROI from the elasticity image based on the shear wave coefficient. The image processor 2030 may add a color to the ROI, or a pattern or edge thereto.
The user input unit 2700 may signify a device used by a user, for example, a sonographer, to input data to control the ultrasound system 2000. For example, the user input unit 2700 may include a key pad, a dome switch, a touch pad such as a capacitive overlap method, a resistive overlay method, an infrared beam method, a surface acoustic wave method, an integral strain gauge method, or a piezoelectric method, a trackball, a jog switch, etc., but not limited thereto. For example, the user input unit 2700 may further include various input devices such as an electrocardiogram measurement module, a respiration measurement module, a voice recognition sensor, a gesture recognition sensor, a fingerprint recognition sensor, an iris recognition sensor, a depth sensor, a distance sensor, etc.
According to the present exemplary embodiment, the user input unit 2700 may sense not only a real-touch but also a proximity touch. The user input unit 2700 may sense a touch input, for example, touch and hold, tap, double tap, flick, etc., with respect to the ultrasound image. Also, the user input unit 2700 may sense a drag input from a position where a touch input is sensed. Alternatively, the user input unit 2700 may sense multiple touch inputs, for example, pinch, with respect to at least two positions on the ultrasound image.
According to the present exemplary embodiment, the user input unit 2700 may receive an input of interest elasticity information from the user. For example, the user input unit 2700 may receive a range of a shear wave coefficient as the interest elasticity information. The user input unit 2700 may receive inputs of a center shear wave coefficient and an application range as the interest elasticity information. The user input unit 2700 may receive a selection of an interest elasticity range from an elasticity range list including a plurality of elasticity ranges.
According to the present exemplary embodiment, the user input unit 2700 may receive information about a size of interest from the user. The user input unit 2700 may receive a request for deleting a boundary line of at least one of a plurality of tumor of interests corresponding to the interest elasticity information. According to the present exemplary embodiment, the user input unit 2700 may receive an input to change the interest elasticity information.
The controller 2300 controls an overall operation of the ultrasound system 2000. For example, the controller 2300 may generally control the probe 2010, the ultrasound transceiver 2020, the image processor 2030, the user input unit 2700, the display 2400, the memory 2500, and the communication unit 2600.
The display 2400 displays and outputs the information processed by the ultrasound system 2000. For example, the display 2400 may display the ultrasound image, or an UI or GUI related to a control panel (not shown).
The display 2400 may display the elasticity image acquired by using a shear wave. The display 2400 may display the elasticity image by overlapping the elasticity image on the B mode image. The display 2400 may display a tumor of interest in the elasticity image. For example, the display 2400 may display a boundary line on the tumor of interest. The display 2400 may provide measurement information about measurement of a tumor of interest. When a plurality of tumors of interest are detected, the display 2400 may provide the measurement information corresponding to each of the tumors of interest.
When the display 2400 and a touch pad in a layer structure form a touch image, the display 2400 may be used as an input device in addition to an output device. The display 2400 may include at least one of a liquid crystal display, a thin film transistor-liquid crystal display, an organic light-emitting diode, a flexible display, a 3D display, and an electrophoretic display. The ultrasound system 2000 may include two or more displays 2400 according to an embodiment type of the ultrasound system 2000.
The memory 2500 may store a program for processing the controller and store input/output data, for example, ultrasound image data, elasticity data, interest elasticity information, elasticity range list, examinee information, probe information, body marker, additional information, etc.
The memory 2500 may include a storage medium of at least one type of a flash memory type, a hard disk type, a multimedia card micro type, a card type memory, for example, SD or XD memory, random access memory (RAM) static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable ROM (EEPROM), programmable ROM (PROM), magnetic memory, a magnetic disc, an optical disc, etc. Also, the ultrasound system 2000 may run a web storage or a cloud server that performs a storing function of the memory 2500 on the Internet.
The communication unit 2600 may include one or more elements that enable communication between the ultrasound system 2000 and a server 3000, between the ultrasound system 2000 and a medical apparatus 4000, and between the ultrasound system 2000 and a mobile terminal 5000. For example, the communication unit 2600 may include a short-range communication module 2610, a wired communication module 2620, a mobile communication module 2630, etc.
The short-range communication module 2610 refers to a module for short-range communication within a predetermined distance. Short-range communication technology may include Wi-Fi, Bluetooth, BLE, ultra-wideband (UWB), ZigBee, near field communication (NFC), Wi-Fi direct (WFD), infrared data association (IrDA), etc.
The wired communication module 2620 refers to a module for communication using an electric signal or optical signal. Wired communication technology according to an exemplary embodiment may include a pair cable, a coaxial cable, a fiber optic cable, an Ethernet cable, etc.
The mobile communication module 2630 transceives a wireless signal with respect to at least one of a base station, the external devices 4000 and 5000, and the server 3000. The wireless signal may include a voice call signal, a video call signal, or various types of data according to text/multimedia message transceiving.
The communication unit 2600 is connected to a network 30 in a wired or wireless manner and communicates with the external device, for example, the medical apparatus 4000 or the mobile terminal 5000, or with the server 3000. The communication unit 2600 may exchange data with a hospital server or other medical apparatuses in a hospital connected through the Picture Archiving and Communication System (PACS). Also, the communication unit 2600 may communicate data according to the Digital Imaging and Communications in Medicine (DICOM) Standard.
The communication unit 2600 may transceive data related to diagnosis of the object 20, for example, ultrasound image, ultrasound image data, Doppler image data of the object 20, or a medical image imaged by other medical apparatus such as CT, MRI, X-ray, etc., via the network 30. Furthermore, the communication unit 2600 may receive information such as a diagnosis history or treatment schedule of a patient, from the server 3000.
The invention can also be embodied as computer readable codes on a computer readable recording medium. The computer readable recording medium is any data storage device that can store data which can be thereafter read by a computer system. Examples of the computer readable recording medium include read-only memory (ROM), random-access memory (RAM), CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.
It should be understood that exemplary embodiments described herein should be considered in a descriptive sense only and not for purposes of limitation. Descriptions of features or aspects within each exemplary embodiment should typically be considered as available for other similar features or aspects in other exemplary embodiments.
While one or more exemplary embodiments have been described with reference to the figures, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope as defined by the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2015-0004452 | Jan 2015 | KR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 6733458 | Steins | May 2004 | B1 |
| 8617075 | Tsujita et al. | Dec 2013 | B2 |
| 20070279436 | Ng | Dec 2007 | A1 |
| 20100113931 | Lee | May 2010 | A1 |
| 20110235066 | Sakuragi | Sep 2011 | A1 |
| 20110282207 | Hashimoto | Nov 2011 | A1 |
| 20120113111 | Shiki et al. | May 2012 | A1 |
| 20120314842 | Kargar | Dec 2012 | A1 |
| 20130195323 | Liu | Aug 2013 | A1 |
| 20130328874 | Smith-Casem et al. | Dec 2013 | A1 |
| 20140003693 | Lee et al. | Jan 2014 | A1 |
| Number | Date | Country |
|---|---|---|
| 2208467 | Jul 2010 | EP |
| 10-1014559 | Feb 2011 | KR |
| 10-2011-0124037 | Nov 2011 | KR |
| 10-1126222 | Mar 2012 | KR |
| 10-2014-0002998 | Jan 2014 | KR |
| 10-2014-0038777 | Mar 2014 | KR |
| Entry |
|---|
| Communication dated Jun. 9, 2016, issued by the European Patent Office in counterpart European Application No. 15177325.6. |
| Number | Date | Country | |
|---|---|---|---|
| 20160202875 A1 | Jul 2016 | US |
