Patent Application
20130187954
1. Field of the Invention
The present invention relates to an image data generation apparatus, and an image data generation method, and more particularly to a technique of generating display image data for image diagnosis capable of improving work efficiency of pathological diagnosis.
2. Description of the Related Art
Recently, in a pathological field, a virtual slide system enabling pathological diagnosis on a display by imaging a test sample placed on a prepared slide and digitizing an image is gaining attention as an alternative for an optical microscope that is a tool of pathological diagnosis. By digitizing a pathological diagnostic image using the virtual slide system, an optical microscope image of a conventional test sample can be handled as digital data. As a result, it is expected to obtain merits such as speeding up of remote diagnosis, explanation to a patient for which an digital image is used, shared use of rear cases, promotion of education efficiency and practical training efficiency. Generally, in a case where a whole test sample is digitized, the data volume thereof becomes considerably large, that is, several hundreds of millions of pixels to several billions of pixels. Therefore, a micro image (detail enlarged image) to a macro image (whole overhead image) can be observed by performing enlarging/reducing processing on a diagnosis image viewer, thereby enabling provision of various conveniences.
On the other hand, there are provided various viewers enabling immediate display of user's desired images from low magnification images to high magnification images, not only in the pathological field. For example, in an ultrasonic diagnostic device disclosed in Japanese Patent Application Laid-open No. H6-78927, which region of a tomographic image has been enlarged and displayed can be clearly grasped by clearly specifying the corresponding relation between the tomographic image and an enlarged image of the partial region. Furthermore, in an ultrasonic diagnostic device disclosed in Japanese Patent Application Laid-open No. 2004-121652, in a case where an ultrasonic image is enlarged and displayed in a stepwise fashion, reduce processing is performed for an original image, and a generated reference image is prepared. This allows to easily understand the relation between the respective images, and to easily display the original image.
Moreover, in an information processing device disclosed in Japanese Patent Application Laid-open No. 2011-112523, when different partial images of an image of an object to be observed, which is obtained by a microscope, are continuously output, the corresponding information of the positions and the resolution of the output partial images are regularly sampled. Then, a history image representing the track of the positions and the observation magnification of the partial image is generated and displayed in a whole image, thereby enabling an observation history to be grasped while looking the whole image.
Generally, in pathological diagnosis, work referred to as screening where interest regions are designated or marked while observing an low magnification image is first performed, and thereafter, detailed observation of the respective interest regions is performed using a high magnification image. Japanese Patent Application Laid-open Nos. H6-78927 and 2004-121652 clearly specify the corresponding relation between the low magnification image and the high magnification observation image that is the interest regions, but do not disclose the corresponding relation about designated order or time between a plurality of interest regions. Therefore, when screening is performed in pathological diagnosis, the designated progress of the interest regions is unclear, and excess and deficiency of the designation of the interest regions occur. Then, redesignation work is required, and work efficiency is reduced.
On the other hand, Japanese Patent Application Laid-open No. 2011-112523 discloses a history image capable of grasping output order or time of a plurality of partial images, but does not clearly specify the corresponding relation between a generated history image and the interest regions desired to be marked. Therefore, the generated history image includes unnecessary information when performing screening, and the interest regions, which are desired to be observed in detail, cannot be clearly determined. Furthermore, corresponding relation can be represented by stopping the screening for a certain period of time every time the partial image including the interest region is output, and generating the history image, however work efficiency is reduced. Moreover, Japanese Patent Application Laid-open No. 2011-112523 discloses a history image capable of grasping the range of enlarged and output partial images, but does not clearly specify a technique of re-outputting the partial images on the basis of the history. Therefore, from the result of the screening, position redesignation and magnification designation are required in order to output and observe the enlarged images of the interest regions in detail, and work efficiency is reduced due to a cumbersome screen operation.
Although the pathological diagnosis is taken as an example, as long as the system is a system that performs screening using a low magnification image and performs detailed observation using a high magnification image, similar problems occur for the system in any field.
The present invention has been conceived in order to solve the aforementioned problems, and the object thereof is to provide a technique capable of associating screening work for designating a plurality of interest regions with detailed observation, and independently performing the screening work and the detailed observation, capable of visually observing progress where the interest regions have been designated during the screening, and capable of improving the work efficiency of image observation.
The present invention in its first aspect provides an image data generation apparatus generating data for displaying information of an interest region designated by a user along with an image on a display device, the image data generation apparatus including: a display data generation unit that generates, from data of an original image, data of a display image corresponding to all or a part of a region of the original image, and generates data of a display screen including the display image; and a position data acquisition unit that, when a user designates an interest region with respect to the display image displayed on the display device on the basis of the data of the display screen, acquires data of a position designated as the interest region, wherein in a case where a plurality of interest regions are designated with respect to the original image, the position data acquisition unit creates list information, in which data of respective positions of the plurality of interest regions is associated with designation order, and in a case where data of a position of a newly designated interest region is acquired by the position data acquisition unit, the display data generation unit generates data for superimposing and displaying, on the display image, a mark indicating the position of the newly designated interest region, and in a case where data of a position of an interest region designated last time exists in the list information, the display data generation unit generates data for superimposing and displaying, on the display image, also a path connecting the position of the interest region designated last time and the position of the newly designated interest region.
The present invention in its second aspect provides an image data generation apparatus generating data for displaying information of an interest region designated by a user along with an image on a display device, the image data generation apparatus including: a display data generation unit that generates, from data of an original image, data of a display image corresponding to all or a part of a region of the original image, and generates data of a display screen including the display image; and a position data acquisition unit that acquires list information in which designation order and data of respective positions of a plurality of interest regions designated by a user with respect to the original image are associated with each other and listed, wherein the position data acquisition unit acquires, from the list information, the data of the respective positions of the plurality of interest regions and the designation order, and the display data generation unit generates data for superimposing and displaying, on the display image, marks indicating the respective positions of the plurality of interest regions, on the basis of the data of the respective positions of the plurality of interest regions and the designation order acquired from the list information.
The present invention in its third aspect provides an image data generation apparatus generating data for displaying information of an interest region designated by a user along with an image on a display device, the image data generation apparatus including: a display data generation unit that generates, from data of an original image, data of a display image corresponding to all or a part of a region of the original image, and generates data of a display screen including the display image; and a position data acquisition unit that acquires list information in which designation order and data of respective positions of a plurality of interest regions designated by a user with respect to the original image are associated with each other and listed, wherein the position data acquisition unit acquires, from the list information, the data of the respective positions of the plurality of interest regions and the designation order, and the display data generation unit generates data for superimposing and displaying, on the display image, marks indicating the respective positions of the plurality of interest regions, and paths sequentially connecting the positions of the plurality of interest regions, on the basis of the data of the respective positions of the plurality of interest regions and the designation order acquired from the list information.
The present invention in its fourth aspect provides an image processing system including: an imaging apparatus or an image storage device; a display device; and an image data generation apparatus generating data for performing display on the display device by using data of an image acquired from the imaging apparatus or the image storage device, wherein the image data generation apparatus is the image data generation apparatus according to the present invention.
The present invention in its fifth aspect provides an image data generation method of generating data for displaying information of an interest region designated by a user along with an image on a display device, the image data generation method including: a display data generation step in which a computer generates, from data of an original image, data of a display image corresponding to all or a part of a region of the original image, and generates data of a display screen including the display image; and a position data acquisition step in which, when a user designates an interest region with respect to the display image displayed on the display device on the basis of the data of the display screen, the computer acquires data of a position designated as the interest region, wherein in the position data acquisition step, in a case where a plurality of interest regions are designated with respect to the original image, list information is created in which data of respective positions of the plurality of interest regions is associated with designation order, and in the display data generation step, in a case where data of a position of a newly designated interest region is acquired in the position data acquisition step, data for superimposing and displaying, on the display image, a mark indicating the position of the newly designated interest region is generated, and in a case where data of a position of an interest region designated last time exists in the list information, data for superimposing and displaying, on the display image, also a path connecting the position of the interest region designated last time and the position of the newly designated interest region is generated.
The present invention in its sixth aspect provides an image data generation method of generating data for displaying information of an interest region designated by a user along with an image on a display device, the image data generation method including: a display data generation step in which a computer generates, from data of an original image, data of a display image corresponding to all or a part of a region of the original image, and generates data of a display screen including the display image; and a position data acquisition step in which the computer acquires list information in which designation order and data of respective positions of a plurality of interest regions designated by a user with respect to the original image are associated with each other and listed, wherein in the position data acquisition step, the data of the respective positions of the plurality of interest regions and the designation order are acquired from the list information, and in the display data generation step, data for superimposing and displaying, on the display image, marks indicating the respective positions of the plurality of interest regions is generated on the basis of the data of the respective positions of the plurality of interest regions and the designation order acquired from the list information.
The present invention in its seventh aspect provides an image data generation method of generating data for displaying information of an interest region designated by a user along with an image on a display device, the image data generation method including: a display data generation step in which a computer generates, from data of an original image, data of a display image corresponding to all or a part of a region of the original image, and generates data of a display screen including the display image; and a position data acquisition step in which the computer acquires list information in which designation order and data of respective positions of a plurality of interest regions designated by a user with respect to the original image are associated with each other and listed, wherein in the position data acquisition step, the data of the respective positions of the plurality of interest regions and the designation order are acquired from the list information, and in the display data generation step, data for superimposing and displaying, on the display image, marks indicating the respective positions of the plurality of interest regions, and paths sequentially connecting the positions of the plurality of interest regions is generated on the basis of the data of the respective positions of the plurality of interest regions and the designation order acquired from the list information.
The present invention in its eighth aspect provides a non-transitory recording medium recording a program for causing a computer to execute the respective steps of the image data generation method according to the present invention.
According to the present invention, it is possible to associate screening work for designating a plurality of interest regions with detailed observation and independently perform the screening work and the detailed observation, to visually observe progress where the interest regions have been designated during the screening, and to improve the work efficiency of image observation.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
A first embodiment achieving the present invention will be described with reference to the figures.
(Configuration of Image Processing System)
The imaging apparatus 101 has a function of imaging a two-dimensional image, and outputting acquired two-dimensional image data to an external device. The imaging apparatus may be a digital microscope device in which a digital camera is mounted on an ocular unit of a general optical microscope. A solid-state image sensing device such as a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS) is employed to acquire the two-dimensional image data.
The image data generation apparatus 102 has a function of generating, from the two-dimensional image data acquired from the imaging apparatus 101, display data for displaying an image or information suitable for pathological diagnosis. The image data generation apparatus 102 can be configured by a general-purpose computer or a workstation enabling high speed arithmetic processing, which includes hardware resources such as a central processing unit (CPU), a RAM, a storage device, an operation unit, and an I/F. The storage device is a high-capacity information storage device such as a hard disk drive, and stores a program or data for achieving respective processing described later, an operating system (OS), and the like. The CPU loads programs and data necessary for the RAM from the storage device and runs the programs, thereby achieving the aforementioned respective functions. The operation unit is configured by a keyboard, a pointing device, or the like, and utilized in order for an operator to input various instructions. The display device 103 described later may have a function of operation input by employing a touch panel.
The display device 103 is a monitor having a function of displaying the image or information suitable for the pathological diagnosis on the basis of the display data generated by the image data generation apparatus 102, and can be configured by a CRT, a liquid crystal display, or the like.
In an example of
The memory 201 is a storage device that has an area for temporarily storing captured (picked-up) image data acquired from external by the image data generation apparatus 102, or display data generated inside, or a work area used for performing various processing with the CPU 204. Here, a DRAM device such as a double-data-rate 3 (DDR 3) is used. The storage 202 is a non-volatile storage device that stores a program or data for allowing the CPU 204 to perform the respective processing performed by the image data generation apparatus 102, image data or a list desired to be stored in the image data generation apparatus 102, and setting data. Here, a device such as an HDD or an SSD is used. The I/F 203 is an interface device for acquiring the captured image data from external, outputting the display data to external, or acquiring operation information from external with the image data generation apparatus 102. Here, a device such as a USB, a Gigabit Ethernet (registered trademark) and DVI can be used. The CPU 204 is an arithmetic processing device for running a program, which manages whole behavior of the image data generation apparatus 102, such as initial setting, control of various devices, and image data processing. Here, a CPU of a general-purpose computer or a workstation is used. The internal bus 205 is an internal bus connecting the aforementioned respective devices. Here, a serial bus such as a PCI Express is used.
The image data input unit 301 has a function of inputting the captured (picked-up) image data acquired by the imaging apparatus 101, and outputting the same to the image memory 302. While imaging apparatus recognition means (not shown) desirably automatically recognizes the connected imaging apparatus 101 and changes the format of the input captured image data, the format of the input captured image data may be set by a user.
The image memory 302 stores the captured image data associated with position coordinates. For example, in a case where an captured image with N×N pixels is present, position coordinates of the pixel at a left upper corner of the captured image are defined as (0, 0), position coordinates on a right hand side of the position coordinate of the pixel at the left upper corner are defined as (1, 0), position coordinates of the pixel at a left lower corner are defined as (0, N), and position coordinates of the pixel at a right lower corner are defined as (N, N). Respective image data corresponding to a plurality of position coordinates is stored in the image memory 302 as the captured image data associated with the position coordinate. The respective captured image data is stored, for example, from address number 0 of the image memory 302 in order of the position coordinates. Accordingly, the address numbers and the position coordinates on the captured image data can be designated on a one-on-one basis. Furthermore, the captured image data stored in the image memory 302 may be either black and white image data or color image data. However, the color image data is configured by three RGB image data for the respective position coordinates. Moreover, the captured image data is preferably configured from a plurality of hierarchical image data (i.e., a plurality of image data with different resolution at the same angle of view) corresponding to an observation magnification. Accordingly, any hierarchical image data corresponding to the position coordinates on the captured image data can be input and output by designating the observation magnification and memory address.
The display image data acquisition unit 303 acquires, from the image memory 302, image data on a region designated by the position data acquisition unit 307 described later. In a case where the designated region is rectangular, the designation of the aforementioned region may be represented by using the position coordinates on four corners of the designated region, represented by a combination of the left upper position coordinates and the right lower position coordinates, or represented by position coordinates, which are the head of the region, and the numbers (region widths) of the respective horizontal and vertical pixels.
The display data generation unit 305 has a function of generating data (referred to as display data) for displaying, on the display device 103, a display screen configured by a display image, a pointer image, a list image, an interest region mark, and the like. Display regions or display position coordinates on the display device 103 of respective parts (images) are designated on the basis of information acquired by the position data acquisition unit 307.
The display image is an image of an object to be observed, which is generated from the captured image data. The pointer image is an image of a mouse pointer. The list image is an image indicating a list of the information of the interest region designated by the user. The interest region mark is information for clearly specifying a position, at which the interest region is designated, to the user, and is superimposed and displayed at a display position on the display image, corresponding to the position coordinates on the captured image data of the interest region. In a case where the designation of the interest region is a second point or subsequent point, the display data generation unit 305 generates data of a path for clearly specifying the position relation between an interest region mark designated last time and an interest region mark designated this time, together with the interest region marks. The path may be either a straight line or a curved line connecting between two points. In a case of the curved line, it is desirable to consider a position and a distance of an interest region mark designated before last time but one. While the curved line may be obtained by any method, a curved line with a long distance and a small curvature radius is not desirable because paths interfere with each other and this makes determination of the user difficult.
The display data output unit 306 has a function of outputting the display data to the display device 103. The format of the output display data corresponds to various signals such as a RGB signal and a brightness color-difference signal. Furthermore, the format thereof also arbitrarily corresponds to the resolution (the number of pixels) of the display device 103. These setting can desirably changed by recognizing the connecting display device 103 with display device recognition means (not shown), but may be set by the user.
The operation information input unit 308 has a function of acquiring, from the operation unit (not shown), operation/setting information such as movement of the mouse pointer, operation determination, and number input, which is operated and set by the user, and outputting the same to the position data acquisition unit 307.
The position data acquisition unit 307 has a function of generating the following data on the basis of the operation/setting information such as movement of the mouse pointer, operation determination, and number input, which is operated and set by the user. That is, the position data acquisition unit 307 generates an acquisition region and a display magnification of the display image data, and the list image data. Moreover, the position data acquisition unit 307 generates the display regions on a screen of the display image and the list image, and the display position coordinates on the screen of the pointer image and the interest region mark.
The interest region mark setting unit 313 is a part of the function configuring the position data acquisition unit 307, and has a function of performing the following on the basis of the display position coordinates of the pointer image on the screen, and the display region on the screen, the display magnification, and acquisition region of the display image data. That is, the interest region mark setting unit 313 has a function of acquiring the position coordinates on the captured image data in the interest region, generating a list associated with the display magnification, generating the list image data, and generating the display position coordinates of the interest region mark on the screen of the display device 103. The position coordinates of the interest region, acquired herein are position coordinates on the captured image data corresponding to a position representative of the inside of the interest region, are here defined as position coordinates which is the head of the region. The display position coordinates of the interest region mark are generated as the display position coordinates on the display image in the display device 103, which correspond to the position coordinates on the captured image data of the interest region.
The acquisition data region calculation unit 312 is a part of the function configuring the position data acquisition unit 307, and has a function of calculating the acquisition region of the display image data by the position coordinates and the display magnification on the captured image data, of the image to be displayed, and the display region in the display device 103.
The image data generation apparatus 102 of
(Screening Work)
As shown in
The whole image display unit 402 displays a whole image 407 where the whole of captured image data acquired from the imaging apparatus 101 is reduced in size, and a region designation frame 408 of the display image according to the observation magnification displayed on the observation image display unit 403.
The observation image display unit 403 displays display image based on the position coordinates on the captured image data designated by the user at a display magnification indicated by the observation magnification display unit 404 as an observation image 409.
The observation magnification display unit 404 displays a display magnification set by the user.
The interest region information display unit 405 displays an image of a list of the interest region. In this list image, information of the interest regions designated/set by the user is arranged in order of listing/setting. Each item of the list corresponds to a single interest region, and information displayed on each item includes position coordinates on the captured image data of the interest region, an interest region display magnification, and the like. The user can select the item in the list image by using the operation unit. In
The interest region can be designated by moving a pointer 406 to a corresponding position on the observation image 409 with the operation unit, and performing a predetermined determination operation such as pressing of a bottom. In this case, the position coordinates of the interest region is obtained by converting position coordinates on the observation image 409 designated by the pointer 406 to the position coordinates on the captured image data. Alternatively, the interest region may be designated and set by directly inputting position coordinate values on the captured image data or display position coordinate values on the display screen 401.
When the interest region is designated, an interest region display magnification input unit 412 for inputting a display magnification (interest region display magnification) used when observing the interest region in detail is displayed.
In a case where the operation unit (not shown) instructs movement of the display region, the position coordinates on the captured image data, corresponding to the display region after the movement are updated, and a new observation image 418 is generated and displayed on the basis of the updated position coordinates and the display magnification. Furthermore, a region designation frame 419 is displayed at a position corresponding to a display region of the new observation image 418 on the whole image 407. Following the movement of the display region, destination display position coordinates of the first point and second point interest region marks 410 and 414 and the path therebetween are also updated. However, in
When the third point interest region is designated in
The display data generation unit 305 generates data for drawing the straight lines (paths) connecting between the interest region marks in order of designation. As a result, a part 426 of a path between the interest region marks 420 and 414, which is present in the display region of the observation image display unit 403 is displayed on the observation image 409. A path 427 between the interest region marks 421 and 415 is displayed similarly.
In the screening work, the aforementioned display behavior is repeated until the designation of all the interest regions is terminated, and the designation of a plurality of the interest regions can be easily and intensively performed. Furthermore, order of designation of the interest regions is clearly specified with the paths, and hence it is possible to designate the interest regions while confirming the history of the designation of the interest regions.
In Step S501, display data for displaying the whole image 407 is generated. The display data generation unit 305 performs resolution conversion of captured image data read out from the image memory 302 so as to conform to the display region of whole image display unit 402, and generates whole image data. Processing of Step S501 can be increased in speed or omitted by previously preparing data for display after the resolution conversion for the whole image 407.
In Step S502, image data of a region of an observation image to be displayed is read out from the image memory 302, and observation image data is generated. As a result, the display data for displaying as in
In Step S503, the interest region mark setting unit 313 determines whether or not the data of the interest region already listed in the list is present. In a case where the already listed data is present, the process advances to Step S504. In a case where the already listed data is not present, that is, the user has never designated the interest region, the process advances to Step S505.
In Step S504, the interest region mark setting unit 313 and the display data generation unit 305 update the interest region mark and the path. First, the interest region mark setting unit 313 acquires position coordinates on captured image data of the respective interest regions from the list, and calculates display position coordinates on the display screen of interest region marks corresponding to these. The calculated display position coordinates of interest region marks are given to the display data generation unit 305 in order of listing in the list. The display data generation unit 305 calculates display position coordinates of straight lines (paths) for connecting the adjacent interest region marks in the order of listing in the list (i.e., order of designation). Then, the display data generation unit 305 extracts only coordinates existing in the display region of the observation image display unit 403 from the calculated coordinate values, and generates and updates the data of the interest region marks and the paths on the basis of these. In a case where the display position coordinates of the interest region marks are associated with the interest regions and stored in the list, the display position coordinates may not be recalculated, and display position coordinates read in from the list may be used.
In Step S505, the position data acquisition unit 307 determines whether the display region of the observation image is changed or not (the display region is moved or not, the display magnification is changed or not, etc.). In a case where the display region of the observation image is changed, the process returns to Step S502, and the display data is updated. In a case where the display position coordinates of the interest region mark are stored in the list, the display position coordinates are also updated, or the display position coordinates are cleared. In a case where the display region is not changed, the process advances to Step S506.
In Step S506, the position data acquisition unit 307 (interest region mark setting unit 313) determines whether or not the user designates the interest region. In a case where the interest region is not designated, the process advances to Step S509. In a case where the interest region is designated, the process advances to Step S507.
In a case where the interest region is designated, the interest region mark setting unit 313 acquires position coordinates on the captured image data, corresponding to the interest region designated by the user in Step S507. Moreover, the interest region mark setting unit 313 acquires interest region display magnification set by the user for detailed observation of the interest region. A configuration, in which a default value of the interest region display magnification is set, and used as it is, may be employed. For example, in a case where screening is performed by using a 5-fold to 10-fold observation image, when a 20-fold image is set for the detailed observation, the set value can be used in a case where change is particularly made. In a case where observation at 40-fold is required, the interest region display magnification should set to 40-fold only for a relevant interest region. Then, the interest region mark setting unit 313 adds the information (the position coordinates on the captured image data, and the interest region display magnification) of this interest region as a new item in the list. At this time, the list is generated so as to have such a data configuration that the order of designation of the interest regions is seen (for example, a data configuration in which items are arranged in the order of designation, or the like). Then, the interest region mark setting unit 313 generates the list image data on the basis of the information of the updated list, in order to add information of a newly designated interest region to the interest region information display unit 405, and gives the same to the display data generation unit 305. Thus, display of the interest region information display unit 405 is updated.
In Step S508, the interest region mark setting unit 313 and the display data generation unit 305 update an interest region mark and a path corresponding to the newly designated interest region. The detailed process will be described later with reference to
In Step S509, it is determined whether or not the screening work is completed. In a case where the screening work is completed, that is, in a case where the work is transferred to detailed observation with a high magnification, or in a case where instruction on read-in of another specimen image is made in order to start screening work of another specimen, this display data generation processing is terminated. In a case where it is determined that the screening work is continuously performed, the process returns to Step S505, and transfers to a waiting state of change of the display region of the observation image.
Here, a configuration, in which the list thus created is stored in the storage 202, may be employed. Furthermore, a configuration, in which this list is output to external of the device (other computer, a network storage, or the like) through the I/F 203, may be employed. Moreover, the list can be desirably associated with a file of the captured image data and managed. As an association method, information of the file of the captured image data may be listed in the list, or information of the list may be listed in the file of the captured image data.
The list can be thus stored and output, thereby allowing another user to use the information of the interest region obtained by the screening work. Alternatively, the information of the interest region obtained by past screening work can be used later. Moreover, a configuration, in which the display position coordinates of the interest region mark once generated are listed in the list, while corresponding to each item in the list, is employed, thereby eliminating the need to recalculate the display position coordinates every time the display position coordinates are read out from the list. However, in this case, when the magnification or the display region in the display device 103 is changed, it is necessary to update the display position coordinates stored in the list, or to clear (delete) the display position coordinates.
In Step S601, the interest region mark setting unit 313 calculates display position coordinates on the observation image 409 in the display device 103 and display position coordinates on the whole image 407 from the position coordinates on the captured image data of the new interest region designated in Step S506. The calculated display position coordinates on the respective images are given to the display data generation unit 305.
In Step S602, the interest region mark setting unit 313 determines whether or not the data of the interest region designated last time is present in the list. In a case where the data is present, that is, in a case of designation of the second point or subsequent point interest region, the process advances to Step S603. In a case where the data is not present, that is, in a case of designation of the first point interest region, the process advances Step S604.
In S603, the interest region mark setting unit 313 acquires, from the list, position coordinates on the captured image data, corresponding to an interest region designated last time. Then, the interest region mark setting unit 313 calculates display position coordinates on the observation image in the display device 103, and display position coordinates on the observation image in the display device 103, of the straight line (path) connecting the two points on the basis of both of the coordinates. Similarly, the interest region mark setting unit 313 calculates display position coordinates on the whole image 407. The display position coordinates of the interest region designated last time are already calculated, and in a case where the coordinate values thereof can be acquired from the interest region mark setting unit 313 or the list, a configuration, in which the already calculated coordinate values are acquired (utilized) in Step S603, may be employed.
In Step S604, the display data generation unit 305 generates and updates the data of the interest region mark and the path on the basis of the coordinates existing in the display region of the observation image display unit 403, among the respective coordinates calculated in Step S601 and Step S603. Similarly, the data of the interest region mark and the path on the whole image 407 are generated and updated.
The form of the path to be displayed may be an arrow directed from the interest region designated last time to the interest region newly designated. A starting point side and an ending point side of the path can be easily visually distinguished by the arrow, and hence the user can easily grasp the progress of designating the interest regions. A display form capable distinguishing the starting point side and the ending point side is not limited to the arrow, and can be achieved also by line width, change in color (gradation, or the like), and animation.
(Detailed Observation)
As shown in
The whole image display unit 402 displays the whole image 407 obtained by reducing the size of the whole of captured image data acquired from the imaging apparatus 101. Then, interest region marks and a path 818 created and updated during screening work are displayed on the whole image 407.
The interest region information display unit 405 displays information of interest regions generated during screening in a list. In this list 801, the information of the interest regions designated/set by the user is arranged in order of designation/setting. Each item of the list corresponds to the single interest region, and the information displayed on each item includes position coordinates on the captured image data of the interest region, an interest region display magnification, and the like. The user can select the item, for which detailed observation is performed, from the list 801 by using the operation unit. A configuration, in which the user does not select the item from the list 801, but can directly input coordinates or a display magnification of the region to be desired to be observed in detail, with reference to the information displayed in the list, may be employed.
The observation image display unit 403 displays an enlarged display image of a region corresponding to the interest region selected from the list by the user. In
The observation magnification display unit 404 displays an interest region display magnification for detailed observation of the interest region selected from the list by the user. In
In detailed observation work, the aforementioned display behavior is repeated by selection of the interest region by the user, and a plurality of the interest regions can be simultaneously observed in detail. Furthermore, the whole image, in which a path showing the order of the designation of the interest regions during screening is superimposed and displayed, is arranged along with the image for observation, thereby enabling detailed diagnosis to be performed while confirming the progress of the screening.
In Step S901, the position data acquisition unit 307 determines whether or not the user selects any of the interest regions from the list of the interest regions. In a case where the interest region is selected, the process advances to Step S902. In a case where the interest region is not selected, the processing of the display image data generation for the detailed observation is terminated.
In Step S902, the position data acquisition unit 307 acquires, from the list of the interest regions, position coordinates on the captured image data, corresponding to the interest region selected by the user. Then, in Step S903, the position data acquisition unit 307 acquires, from the list of the interest regions, an interest region display magnification, corresponding to the interest region selected by the user.
In Step S904, the display image data acquisition unit 303 reads out, from the image memory 302, image data corresponding to a region of the enlarged display image selected by the user, on the basis of a region calculated from the position coordinates and the interest region display magnification acquired by the position data acquisition unit 307. Thereafter, the display data generation unit 305 generates display data reflecting enlarged observation image data through the display image data acquisition unit 303.
Finally, in Step S905, it is determined whether an interest region is additionally selected or not. In a case where the additional selection is made, the process returns to Step S902, and generation of the enlarged display image data for detailed observation is thereafter repeated. In a case where the additional selection is not made, the processing of the display image data generation for the detailed observation is terminated. A screen of the detailed observation shown in
The list provided for the display screen during the detailed observation is generally generated based on the list information, which is created during screening work performed at a stage prior to detailed observation and is stored in the memory 201. However, a configuration, in which list information generated in screening work performed in the past or in screening work performed at another place (another device) can be read in from the storage 202, or can be acquired from external of the device through the I/F 203, may be employed. Moreover, a configuration, in which the captured image data is re-entered in a case where the captured image data stored in the memory 201 does not coincide with the captured image data associated with the list, is desirably employed.
The list information can be thus read out and input, thereby enabling another user to perform detailed observation by using the information of the interest region obtained by the screening work. Alternatively, the detailed observation can be performed by reutilizing the information of the interest region obtained by past screening work. Here, in a case of performing the detailed observation work on the basis of the information thus read out from the list, it is desirable to generate and update the data of the interest region mark and the path on the whole image and the list prior to Step S901, and to enable the display of the display image shown in
With the configuration and the behavior according to the first embodiment, described above, the user can designate a plurality of the interest regions in the screening work of pathological diagnosis while confirming the history of the designation of the interest regions. Furthermore, the user can select a plurality of the interest regions in detailed observation work while confirming the progress of the screening. As a result, excess and deficiency of the screening or the selection of the interest regions can be prevented, a screen operation is simplified, and work efficiency can be improved. Furthermore, a designation process of a screening worker can be reproduced, and hence a list generated by an experienced pathologist can be applied to a pathology education of a freshman, for example.
A second embodiment achieving the present invention will be described with reference to the figures.
According to the first embodiment, the example of displaying the path showing the order of designation of the interest regions during the screening work or the detailed observation work is shown. According to the present embodiment, a function of drawing interest region marks and paths one by one (or group by group in a case where the number of the interest regions is large) in accordance with order of designation of the interest regions is provided. That is, this function is a function of reproducing and displaying the progress of the designation of the interest regions during screening work by a user. At this time, it is preferable to adjust such that not only the order of the drawing, but also a time interval of display (drawing) of each interest region mark conforms to a time interval of actual designation of each interest region in the screening work. This enables the intention of the screening worker to be faithfully reproduced. In the following second embodiment, an example of reproducing both the order of the designation of the interest regions and the time interval of the designation is shown.
The system of
The image server 1201 has a function of storing the two-dimensional image data of the specimen imaged by an imaging apparatus 101 having a function of imaging the two-dimensional image. The image data generation apparatus 1202 has a function of acquiring the imaged two-dimensional image data from the image server 1201, and generating a display data for displaying information and an image suitable for pathological diagnosis. A configuration of the image data generation apparatus 1202 achieving this function is similar to that of the image data generation apparatus 102 illustrated in
The time acquisition unit 311 is a part of the function configuring the position data acquisition unit 307, and has a function of acquiring a time when an interest region mark setting unit 313 acquires position coordinates corresponding to the interest region. Furthermore, the time acquisition unit 311 has a function of selecting an interest region from a list to read out the acquisition time of the interest region, and setting a time for reading out the position coordinates corresponding to the interest region, and selecting a next interest region.
The interest region mark setting unit 313 has a function of generating a list in which the respective acquisition times are associated with the position coordinates corresponding to the interest regions, in addition to the function illustrated in
The image data generation apparatus 1202 of
As shown in
Serial numbers are generated and listed in the item number column 1101 in order of acquisition of the interest regions. While
First, in Step S1301, the display data generation unit 305 generates a whole image data, and updates display data for displaying the whole image on an observation image display unit 403. This flow may be the same processing as Step S501 of
In Step S1302, the time acquisition unit 311 determines whether or not a data line associated with a next item number is present in the list. In a case where the next data line is present, the data line is selected, and the process advances to Step S1303. In a case where the next data line is not present, reproduction and display of the path are terminated.
In Step S1303, the time acquisition unit 311 reads out an acquisition time from the data line selected in Step S1302.
In Step S1304, the time acquisition unit 311 sets a timer on the basis of the acquisition time read out in Step S1303, and a countdown is started. A difference from an acquisition time of last time is used as a start value of the timer. Alternatively, the difference may be a value multiplied by any constant.
In Step S1305, the time acquisition unit 311 determines whether or not the timer value reaches 0. In a case where the timer value reaches 0, the interest region mark setting unit 313 reads out position coordinates on the captured image data, corresponding to the interest region, from the data line selected in Step S1302, and the process advances to Step S1306. In a case where the timer value is not 0, the process returns to Step S1305, and monitoring of the timer value is repeated.
In Step S1306, the display data generation unit 305 generates display image data including an updated interest region mark and a path, and updates the display data. This flow is performed based on the position coordinates on the captured image data, which are read out in Step S1305, similarly to Steps S601 to S604 of
In a case where only designation order is reproduced, processes regarding the time in Steps S1303 to S1306 are not performed, the flow of reading out the position coordinates on the captured image data, corresponding to the interest region should be employed in Step S1306, after selecting the data line in Step S1302. In this case, the interest region mark and the path may be updated at a constant time interval.
Alternatively, a configuration, in which the observation image display unit 403 can display an image for detailed observation of the position coordinates on the captured image data, which are read out in Step S1305, in place of the whole image, may be employed. In this case, in place of selection by a user, selection in Step S1302 should be made in the flow of
With the configuration and the behavior according to the second embodiment, described above, the user can reproduce and confirm the progress of sequentially designating a plurality of the interest regions in the screening work in order of the designation at a time interval according to the time of the designation. As a result, the designation process such as “how long does a screening worker spend to designate for which interest region” can be faithfully reproduced, and hence can be more adequately applied to pathology education.
A third embodiment achieving the present invention will be described with reference to the figures.
According to the second embodiment, the example of sequentially drawing the interest region marks and the paths on the display image in the order of designation of the interest regions or according to time intervals is shown. According to the present embodiment, a function of selecting a path of a specific interest region from among a sequence of interest regions designated by a screening worker, and displaying the same is provided.
A device configuration can be achieved by a device configuration equivalent to those described in the first and second embodiments, and the detailed description thereof will be omitted.
As shown in
The whole image display unit 402 displays a whole image 701 where the whole of captured image data acquired from an imaging apparatus 101 is reduced in size. Then, an interest region mark created and updated during screening work is displayed on the whole image 701. In an initial state, no path is displayed.
The interest region information display unit 405 displays information of an interest region generated during screening in a list. The user can select the item, for which detailed observation is performed, from the list by using the operation unit. A configuration, in which the user does not select the item from the list, but can directly input coordinates of the region, for which path display is desired to be performed, with reference to the information displayed in the list, may be employed.
Here, the observation image display unit 403 displays an enlarged display image 803 corresponding to the selected interest region at a corresponding interest region display magnification. While the enlarged display image 803 is displayed by using the whole display region of the observation image display unit 403, an enlarged display image of a region connected to the additionally displayed path may be displayed simultaneously (so as to be arranged together spatially) as in
Processes of Steps S1401 to S1403 can be achieved by those of Steps S1301, S1302, and S1306 of the flowchart at the time of reproduction and display in the order of the designate of the paths according to the second embodiment of the present invention illustrated in
In Step S1402, a position data acquisition unit 307 determines whether or not a data line associated with a next item number is present in the list. In a case where the next data line is present, the data line is selected, and the process advances to Step S1403. In a case where the next data line is not present, addition of a sequence of the interest region marks is terminated, and the process advances to Step S1404.
In Step S1403, the position data acquisition unit 307 reads out position coordinates on the captured image data, corresponding to the interest region, from the data line selected in Step S1402. Then, a display data generation unit 305 generates display image data including an updated interest region mark, and updates the display data. At this point, a path is not added to the display image. Thereafter, the process returns to Step S1402, and search of the next data line in the list is thereafter repeated.
In Step S1404, display image data, in which the path connected to the interest region selected by the user is added, is generated and updated. This process will be later described in detail with reference with
In Step S1501, the position data acquisition unit 307 determines whether or not the user selects any of the interest regions from the list of the interest regions. In a case where the interest region is selected, the process advances to Step S1502. In a case where the interest region is not selected, the processing of the generation of the display image data, in which the path is added, is terminated.
In Step S1502, the position data acquisition unit 307 acquires, from the list of the interest regions, position coordinates on the captured image data, corresponding to the interest region selected by the user.
In Step S1503, the position data acquisition unit 307 determines whether or not data line associated a last item number is present in the list. In a case where the last item number is present, the data line is selected, and the process advances to Step S1504. In a case where the last item number is not present, the process advances to Step S1505.
In Step S1504, the position data acquisition unit 307 acquires, from the list of the interest region, position coordinates on the captured image data, corresponding to an interest region selected as the last line.
In Step S1505, the position data acquisition unit 307 determines whether or not data line associated a nest item number is present in the list. In a case where the next item number is present, the data line is selected, and the process advances to Step S1506. In a case where the last item number is not present, the process advances to Step S1507.
In Step S1506, the position data acquisition unit 307 acquires, from the list of the interest region, position coordinates on the captured image data, corresponding to an interest region selected as the next line.
In Step S1507, the display data generation unit 305 generates path data connected to the interest region selected by the user on the basis of the coordinates acquired in Steps S1502, S1504 and S1506 by the position data acquisition unit 307, and generates the display image data. Then, display data reflecting the display image data is updated. Thus, paths connecting the interest region selected by the user and interest regions before and after the interest region are displayed.
Finally, in Step S1508, it is determined whether an interest region is additionally selected or not. In a case where the additional selection is made, the process returns to Step S1502, and generation of the display image data, in which the path is added, is thereafter repeated. In a case where the additional selection is not made, the processing of the generation of the display image data, in which the path is added, is terminated.
In a case where the observation image display unit 403 displays an enlarged display image 803 corresponding to the selected interest region, processes of Steps S902 to S904 shown in
With the configuration and the behavior according to the third embodiment, described above, the user can focus on the specific interest region, and confirm the path designated by a screening worker. As a result, it is possible to review or expect order of the designation of the specific interest regions from among a sequence of the interest regions, and to compare the same with order of actual designation, thereby enabling more adequate application to pathology education.
A fourth embodiment achieving the present invention will be described with reference to the figures.
According to the present embodiment, a function of determining importances of paths of interest regions on the basis of conditions upon designation of interest regions designated by a screening worker, and clearly specifying the results thereof is provided.
For example, in a case where during screening work, a time interval up to designation of a next interest region is longer than a time interval up to designation of another region, there is a high possibility that an important feature confusing the screening worker's determination as to the designation of the interest region exists around the region. According to the present embodiment, an example of performing importance determination by employing the time interval up to the designation of the next interest region as a condition will be described.
A device configuration can be achieved by a device configuration equivalent to those described in the first and second embodiments except for the position data acquisition unit 307, and the detailed description thereof will be omitted.
The importance determination unit 2201 has a function of reading out an acquisition time listed on a data line newly updated this time and an acquisition time listed on a last data line from a list at the time of designation of the interest region, and determining importances from a difference between both of the acquisition times. Furthermore, the importance determination unit 2201 has a function of sequentially selecting data lines from the list when reproducing and displaying interest region marks and paths, and reading out an acquisition time listed on the selected data line and an acquisition time listed on a last data line, and sequentially determining importances from a difference between both of the acquisition times. Similarly to the second embodiment, the acquisition time used here may be a time from any point of time.
An interest region mark setting unit 313 has a function of determining a method for displaying an interest region mark and a path reflected on a whole image 407 by a display data generation unit 305 in accordance with the importance determined by the importance determination unit 2201, in addition to the function illustrated in
Processing shown in
In Step S1601, the time acquisition unit 311 acquires a time of designation of an interest region (this time acquisition time) by a user, ant lists the same in the list. Other processing is similar to that of Step S507.
In Step S1602, the importance determination unit 2201 determines whether or not a data line associated with a last item number is present in the list. In a case where the last data line is present, the process advances to Step S1603. In a case where the last data line is not present, the determination of the importance is terminated.
In Step S1603, the importance determination unit 2201 reads out the this time acquisition time listed in Step S1601, and an acquisition time (last time acquisition time) listed on the last data line determined in Step S1602.
In Step S1604, the importance determination unit 2201 deducts the last time acquisition time from the this time acquisition time, and the process advances to Step S1605 in a case where the difference is larger than a predetermined value (threshold), while the determination of the importance is terminated in a case where the difference is the threshold or less.
In Step S1605, the importance determination unit 2201 sets an “importance” flag. This flag is cleared at the time of designation of a next interest region.
Thereafter, in Step S604 shown in
On the other hand, in a case where the importance is determined and reflects when the interest region mark and the path are reproduced and displayed, Steps S1602 to S1605 should be performed in the process of Step S1306 in
A determination condition of the importance used in the present embodiment is not limited to a time interval up to the designation of the next interest region, described above.
For example, the determination condition may be based on an interest region display magnification set at the time of the designation of the interest region. There is a high possibility that when the interest region display magnification is changed to be set to a higher magnification, a portion around the region is a region which should be distinguished from other region.
Alternatively, a UI and a function capable of adding attribution to the interest region at the time of the designation of the interest region may be provided, and the importance may be determined on the basis of the added attribution. Attribution information should be recorded in the list for each interest region. The attribution is information on a property or a state of the interest region, and as long as the information is information useful for detailed observation or diagnosis, any information may be added as the attribution information. For example, the user may select the information among from options such as “normal”, “abnormal”, “pending”, and the like. There is a high possibility that when the attribution is changed, a portion around the region is a region which should be distinguished from other region. Therefore, in a case where attribution added to an interest region on a starting point side of a path is compared with attribution added to an interest region on an ending point side, and there is a difference between them, the “importance” flag should be set on this path. Alternatively, the stage (level) of the importance may be changed depending on a way to change the attribution (for example, the level of the importance is made different in accordance with a case of change from “normal” to “abnormal” and a case of change from “abnormal” to “pending”, etc.).
As the UI for inputting (selecting) the attribution, a configuration, in which an attribution input unit is displayed at the time of the designation of the interest region may be employed, similarly to the interest region display magnification illustrated in
Moreover, the determination condition of the importance used in the present embodiment may be combination of various conditions, and for example, there is a high possibility that a feature which should be further distinguished exists on a portion around a region where the attribution is changed into “abnormality” after a long period of time. In a case of using various such determination conditions, the information read out from the list by the importance determination unit 2201 should be properly changed in Step S1603.
With the configuration and the behavior according to the fourth embodiment, described above, the user can grasp the feather (importance) of the interest region designated by the screening worker. As a result, a region having a high possibility that the screening worker focuses distinctively from other region can be grasped from the whole image, and work efficiency can be further improved.
A fifth embodiment achieving the present invention will be described with reference to the figures.
According to the present embodiment, a function of enabling selection of a path of an interest region designated by a screening worker, and displaying the enlarged image around the selected path is added.
A device configuration can be achieved by a device configuration equivalent to those described in the first to fourth embodiments, and the detailed description thereof will be omitted.
The path list of
As shown in
Serial numbers are generated and listed in the path number column 1701 in order of display of the paths (order of the designation of second point and subsequent interest regions). In the starting point coordinate column 1702, the starting point coordinates at the display positions of the paths in the display device 103 are associated with path numbers and listed for each line. In the ending point coordinate column 1703, the ending point coordinates at the display positions of the paths in the display device 103 are associated with the path numbers, and listed for each line. In the importance column 1704, the importances of the paths are associated with the path numbers, and listed for each line. Alternatively, feature amounts of the path may be associated with the path numbers and listed in the path list. For example, a distance or a direction of the path is relevant to the feature amount of the path.
This path list 1700 can be generated by listing the display coordinates of the path calculated in Step S603 of
Thus, the display information of the path calculated once is recorded in the path list, thereby enabling utilization for determination as to which path is selected, when the path is selected by a mouse pointer, for example. However, when a display magnification or a display region in the display device 103 is changed, the display position coordinates in the list information, or the feature amount needs to be updated.
Determination as to whether or not the path exists at a position of the mouse pointer should be made by using a method of enabling selection of an already drawn figure like a method used in general drawing device and the like.
As shown in
Here, an observation image display unit 403 displays enlarged display images 808 and 809 corresponding to the interest region marks connected to the path 1802 at respective corresponding interest region display magnifications.
With this screen configuration, a user can easily compare detained image of specific continuous interest regions.
In Step S1901, a position data acquisition unit 307 determines whether or not the user selects any path from the path list. In a case where the path is selected, the process advances to Step S1902. In a case where the path is not selected, the flow of
In Step S1902, the position data acquisition unit 307 acquires a path number of the path selected from the path list.
In Step S1903, the position data acquisition unit 307 acquires, from the list of the interest regions, position coordinates and an interest region display magnification on a captured image data, corresponding to an interest region listed in an item number coincident with the path number acquired in Step S1902.
In Step S1904, the position data acquisition unit 307 acquires, from the list of the interest regions, position coordinates and an interest region display magnification on captured image data, corresponding to an interest region listed on a next line of the item number coincident with the path number acquired in Step S1902.
Finally, in Step S1905, a display image data acquisition unit 303 reads out image data corresponding to regions of two enlarged display images by an image memory 302 on the basis of the position coordinates and the interest region display magnification acquired by the position data acquisition unit 307. Thereafter, a display data generation unit 305 reflects two enlarged observation image data and updates display data through the display image data acquisition unit 303. A screen of the detailed observation shown in
With this screen configuration, the user can easily confirm the detained image of the specific interest regions.
The display image data displaying as in
That is, in Step S1905, the data acquisition unit 307 determines a display magnification (resolution conversion rate of read out image data) and a region read out from the image memory 302, from position coordinates on the captured image data of the two portions acquired in Steps S1903 and S1904, such that the both fall in the observation image display unit 403. Then, the display image data acquisition unit 303 reads out a target image data on the basis of the region determined by the position data acquisition unit 307 from the image memory 302, and performs resolution conversion. Thereafter, the display data generation unit 305 reflects enlarged observation image data and updates display data through the display image data acquisition unit 303. A screen of the detailed observation shown in
With the configuration and the behavior according to the fifth embodiment, described above, the user can select the specific path from among a sequence of the interest regions designated by the screening worker, and confirm the enlarged images around the path. Thus, the user can select the region, for which detailed observation is performed, on the basis of the feature of the path. As a result, the interest region can be effectively selected, and work efficiency can be further improved.
Furthermore, the aforementioned effect further is improved by combining this embodiment with the fourth embodiment. For example, the user can select the path while grasping the importance, and can easily compare and confirm a detailed image of a region having a high possibility that the screening worker focuses distinctively from other region, or of a portion around the region.
A sixth embodiment achieving the present invention will be described with reference to the figures.
According to the present embodiment, a position data acquisition unit 307 records not only position coordinates of an interest region designated by a user but also display track information (display track data) which is a track of movement (scroll) of a display region. Then, a function of providing a display track is provided in addition to the interest region or a path. When the display track is recorded through the whole screening work, data volume becomes enormous, and therefore the display track should be partially recorded. Preferably, determination as to whether the display track needs to be recorded should be made on the basis of information (interest region display magnification, or the like) set to the interest region, the importance of the path described in the fourth embodiment, or the like every time the user designates a new interest region.
For example, in a case where the user sets an interest region display magnification to be higher than other region during screening work, there is a high possibility that screening until a next interest region is designated is performed on a track different from before. According to the present embodiment, an example, in which the display track information is acquired by employing change in setting of the interest region display magnification as a condition, will be described.
A device configuration can be achieved by a device configuration equivalent to those described in the first to fourth embodiments, and the detailed description thereof will be omitted.
As shown in
Furthermore,
Here, the acquisition of the display track information during the screening according to the sixth embodiment of the present invention can be achieved as described below.
That is, the flow of the acquisition of the interest region illustrated in
Furthermore, the display of the display track can be achieved as follows.
That is, the additional flow of the interest region mark described in Steps S1401 to S1403 of
On the other hand, in a case of reflecting the display track when reproducing and displaying the interest region mark and the path, the aforementioned track display flow (corresponding to Steps S1401 to S1403 of
With the configuration and the behavior according to the sixth embodiment, described above, as to a region having a high possibility that the screening worker focuses distinctively from other region, the relation between the track following for screening and the path can be grasped. As a result, the interest region can be efficiently selected, and work efficiency can be further improved.
Furthermore, the relation between a portion determined as an interest region by the screening worker, and a portion not determined as the interest region can be grasped, thereby enabling more adequate application for pathology education.
Moreover, the aforementioned effect is further improved by combining this embodiment with the fourth embodiment and the fifth embodiment. For example, as shown in
The display of the enlarged display image 2103 of the display track can be achieved as described below.
That is, processing similar to the generation and display flow of the display data during detailed observation illustrated in
The present invention provides a system or a device with a recording medium (or a storage medium) recording a program cord of a software achieving all or a part of the functions according to the aforementioned embodiments. Then, a computer (or CPU or MPU) of the system or the device reads out and executes the program cord stored in the recording medium, thereby enabling the functions. In this case, the program cord itself, which is read out from the recording medium, achieves the functions of the aforementioned embodiments, the recording medium recording the program cord configures the present invention. Furthermore, the computer executes the read-out program cord, an operating system (OS) running on the computer and the like performs a part of or all of actual processing. A case where the aforementioned functions are achieved by the processing is also included in the present invention. Moreover, the program cord read out from the recording medium is written in a memory provided on a function expansion card inserted into the computer or a function expansion unit connected to the computer. Thereafter, a case where a CPU provided on the function expansion card or the function expansion unit, or the like performs a part of or all of the actual processing on the basis of the instruction of the program cord, and the functions according to the aforementioned embodiments are achieved by the processing are also included in the present invention. In a case where the present invention is applied to the aforementioned recording medium, the recording medium stores a program cord corresponding to the flowchart previously described.
The configurations described in the first to seventh embodiments can be used by combining the embodiments with each other.
Accordingly, a person skilled in the art will easily conceive of appropriately combining various techniques in the aforementioned respective embodiments to configure a new system, and a system obtained by such various combination also belongs to the category of the present invention.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2012-013323, filed on Jan. 25, 2012 and Japanese Patent Application No. 2012-251760, filed on Nov. 16, 2012, which are hereby incorporated by reference herein in their entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2012-013323 | Jan 2012 | JP | national |
| 2012-251760 | Nov 2012 | JP | national |
