The present invention relates to a technique for acquiring biological information of a subject under inspection using an inspection apparatus such as a medical imaging apparatus and providing the information to the medical imaging apparatus.
In the examination of a subject (mostly a patient) using a medical imaging device such as an MRI (Magnetic Resonance Imaging) apparatus, in order to reduce the influence of artifacts caused by pulsations and breathing movements of the subject, image capturing synchronized with these movements is widely performed. Regarding pulsation and breathing motion of a subject, usually, a measurement device such as an electrocardiograph or a breathing balloon for measuring breathing motion is attached to the subject under examination, and a signal from the measurement device is taken into the imaging apparatus, thereby controlling imaging. When the breathing motion can be accurately grasped, it is also possible to correct the measurement data or the image by using the movement.
However, a measurement device such as a breathing balloon needs to be prepared for setting it to a patient, and the detection of breathing motion may not be performed with high accuracy due to a matter of setting or individual difference of the subject.
On the other hand, in an MRI apparatus, for example, there has been widely used a technique of performing signal acquisition with a target at a position where there is movement of a diaphragm or the like in order to detect breathing movement and monitoring breathing movement. However, in this technique, the imaging time must be extended because the navigation echo should be generated and collected separately from the nuclear magnetic resonance signal for imaging.
On the other hand, Japanese Pat. No. 6,714,006 (Patent Document 1) discloses a camera system for automatically measuring physiological parameters such as pulsation and breathing of a subject, and describes that the camera system is applied to an MRI apparatus and the physiological parameters are automatically measured during examination by the MRI apparatus. In this camera system, before the subject is conveyed into the bore (examination space) of the inspection apparatus, biometric parameters of the subject are measured by a digital camera installed outside the bore to determine a predetermined range of the subject, and then the predetermined range is photographed by a camera arranged distal to the conveyance side with respect to the subject conveyed into the bore to determine a region of interest using the biometric parameters, and physiological parameters are calculated from photographic data of the region of interest.
As described above, in order to obtain physiological information (motion information) such as breathing motion and pulsation from a subject without contact, it is important to set a region of interest that best reflects the physiological information (motion information), and in particular, in the case of performing respiration gated imaging, it is essential to set the region of interest correctly. However, in Patent Document 1, the region of interest is determined based on the biometric parameter, specifically, the distance between the sternum and the right clavicle of the subject, and if the region of interest is determined only by the biometric parameter such as the size of the subject, the region of interest may be too wide, or narrow. If the region of interest is narrow, positional deviation may occur, and appropriate setting of the region of interest is not secured. In addition, in the technique described in Patent Document 1, in order to acquire biometric parameters, it is necessary to install a camera outside the bore in addition to the camera that photographs the area including the region of interest. That is, two cameras are essential.
It is an object of the present invention to solve the above-mentioned problems of the prior art, and to provide a technique capable of accurately and automatically setting a region of interest for acquiring biological information based on a biological information signal obtained from a subject placed in an examination space in a non-contact manner, thereby making it possible to efficiently perform imaging of a medical image using biological information (motion information), and to reduce time and labor of a doctor, a technician, or the like.
In order to solve the above-mentioned problems, a medical imaging apparatus of the present invention is provided with a biological signal processing system, and the biological signal processing system is provided with a non-contact type biological information measuring unit that is installed in or near an examination space of the medical imaging apparatus and measures a state of a subject under examination, and a signal analyzing unit that processes a biological information signal measured by the biological information measuring unit and calculates a motion of the subject. The biological information measuring unit acquires the biological information signal from a predetermined range of the subject, and the signal analyzing unit includes a region-of-interest selection unit that selects a region of interest, in which the movement of the subject is to be acquired, from among the plurality of regions using biological information signals for each of the plurality of regions included in the predetermined range, and calculates the movement of the subject using biological information signals measured from the region of interest selected by the region-of-interest selection unit.
The medical imaging system of the present invention includes an inspection apparatus, a biological signal processing system including the above-described biological information measuring unit and signal analyzing unit, and an apparatus for displaying a GUI.
Further, the biological signal processing method of the present invention is a biological signal processing method which processes a biological information signal of a subject under examination measured by a non-contact type biological information measuring apparatus installed in or in the vicinity of an examination space of an inspection apparatus, and calculates a movement of the subject. The method includes a step of generating a biological information signal for each of a plurality of regions using the biological information signal measured from a predetermined range of the subject, a step of calculating an index relating to the strength or noise or a movement for each of the biological information signals of the plurality of regions, and a step of selecting a region of interest of the subject for which a movement is calculated, based on the index.
According to the present invention, it is possible to set a position or a region in which the movement of the subject can be accurately captured by the biological information measuring apparatus as a region of interest, and to obtain the movement of the subject with good accuracy by processing the biological information signal collected from the region of interest.
Embodiments of a biological signal processing system and a medical imaging system including the same of the present invention will be described below with reference to the drawings.
As shown in
The biological signal measuring apparatus 200 is an apparatus for acquiring biological information of a subject under examination using the imaging apparatus 100 without contact, and is composed of a camera, or a distance sensor using an ultrasonic wave, an electromagnetic wave such as infrared rays or millimeter waves, or the like (hereinafter, collectively referred to as a sensor including a camera), detects information relating to a distance between the sensor and the subject or a movement of the subject, and outputs the information as a biological information signal. When the sensor is a camera, video data (time-series image data) obtained by photographing a predetermined range of the subject by the camera is a primary biological (biometric) information signal, and therefrom biological information (secondary biological information signal) representing changes in pixel values and pixel positions over time is generated. The measuring apparatus utilizing electromagnetic waves, ultrasonic waves or the like is provided with a source for generating electromagnetic waves or ultrasonic waves such as millimeter waves and a receiving unit for receiving the reflected waves thereof, and processes the reflected waves from a predetermined range of the subject to generate a biological information signal for each position within the predetermined range.
The signal processing apparatus 300, by using the biological information signal output by the biological signal measuring apparatus 200, performs various processes for determining a site (region of interest) of a subject from which biological information (movement of a subject) to be provided to the imaging apparatus 100 should be acquired. In order to realize it, the signal processing apparatus 300 includes a signal analyzing unit 310. The signal analyzing unit 310 includes a biological information calculation unit 311 that calculates biological information for each of a plurality of positions or regions within the predetermined range from a biological information signal sent from the biometric signal measuring apparatus 200, a region-of-interest selection unit 313 that analyzes the biological information for each position or region and determines a position or region where the most accurate biological information is obtained, an index calculation unit 312 that calculates an index related to noise or intensity of the biological information, and the like.
A signal processing device associated with the imaging apparatus 100 for processing the measurement signal and the image signal in the imaging apparatus 100 may also serve as the signal processing apparatus 300, or the signal processing apparatus 300 may be an independent device for processing the signal of the biological signal measuring apparatus 200. Here, the system including the biological signal measuring apparatus 200 and the signal analyzing unit 310 of the signal processing apparatus 300 is referred to as a biological signal processing system 20. The function of the signal processing apparatus 300 including the signal analyzing unit 310 is realized by a computer including a memory and a processing unit such as a CPU or a GPU reading an analysis program. However, some or all of the functions of the signal processing apparatus 300 may be realized by hardware such as an ASIC, and such a case is also included in the present embodiment. Although not essential, the computer for realizing the function of the signal processing apparatus 300 includes a display 400 for displaying a processing result or a GUI, an input device such as a pointing device or a keyboard, a storage device for storing the above-described analysis program, the processing result, data necessary for processing, and the like, similarly to a general computer. Note that the display 400 for displaying the GUI may be a display provided in the imaging apparatus 100.
The specific content of the biological information signal will be described later, but in the case where the biological information is periodic movement such as pulsation or breathing of the subject, the secondary biological information signal will be a signal having a periodicity having a predetermined amplitude as shown in
The biological signal processing system 20 of the present embodiment first determines an optimal position (region of interest) of a subject for obtaining biological information using the biological information signal measured by the biological signal measuring apparatus 200, and then calculates biological information using the biological information signal obtained from the determined region of interest. In order to determine a region of interest, biological information signals obtained from a plurality of regions are used to calculate indicators related to noise and the intensity of motion. Based on the calculated one or a plurality of indices, a region in which the noise is small and the motion intensity is large is selected from a plurality of regions, and the region is set as a region of interest. There are various ways to set a size and to divide a plurality of regions, and the region of interest to be selected is not only one, but may be 2 or more.
Hereinafter, an embodiment in which the imaging apparatus 100 is an MRI apparatus and the sensor of the biological signal measuring apparatus 200 is the camera 210 will be described.
In the present embodiment, the sensor of the biological signal measuring apparatus 200 is a single camera, acquires a video signal as a primary biological information signal, and calculates a signal representing a positional variation obtained from an optical flow of an image as a secondary biological information signal. The overall configuration of the apparatus is the same as that shown in
As illustrated, the signal analyzing unit 310 includes an optical flow calculating unit 311A that receives a video signal (primary biological information signal) from the sensor (camera) 210 as the biological information calculation unit 311 of
For example, as shown in
The optical flow is a change in the pixel position between frames for each pixel expressed by a velocity vector, it can be calculated using a gradient method such as, for example, a Lucas-Kanade. By calculating this between each frame, as the integrated value of the velocity vector, it is possible to obtain a variation with respect to the time axis of each pixel (S2). If the body axis direction of the subject 103 is taken as the Y direction, this variation can be obtained as a component (vector absolute value of the Z component) in a direction (Z direction) orthogonal to the Y direction. An example of the variation thus obtained is shown in
Next, among the variations of each pixel (for each position in the image), in order to determine the position of the variation that reflects the body movement to be determined most accurately, the index calculation unit 312 divides the image into a plurality of regions, and calculates indices of noise and movement for variations in each region (S3). In the present embodiment, the standard deviation (SD) of the variation value and the band power (BP) of the variation are calculated as indices. SD is an indicator of noise and is used to eliminate areas where large noise is present. BP is an index for knowing whether or not a large motion has occurred in the frequency band related to the body motion.
To calculate the band power, FFT unit 312A performs Fourier transform the variation with respect to the time axis as shown in
In Equation (1), f is the frequency, x is the sampling measurement signal, N is the number of measurement points, Δt is the sampling interval.
In step S4, the index calculation unit 312 calculates the indices SD and BP for each region, and the region-of-interest selection unit 313 selects, based on the index calculated by the index calculation unit 312, a region in which the noise is small and the motion is large as a region of interest (S4). Here, the minimum unit of the region for which the index calculation unit 312 calculates the index is each pixel, but one region may be selected from roughly divided regions, and the region may be further subdivided to select one or a plurality of regions from the divided regions. Further, areas where the brightness of the image is too high or the brightness is too low may be excluded by setting a threshold in advance, since the calculation of the optical flow and the index is likely to be unsuccessful for those areas. When calculating BP, time information of a certain length or more is required. Compared to BP, SD can be calculated in a relatively short time. Therefore, an index SD may be used first to select a region of interest to start acquisition of biological information, and then an index BP may be used to narrow down the region of interest. Thus, it is also possible to shorten the time until the start of acquisition of biological information.
Further, the calculation formula of the band power described above is not limited to Equations (1), (2). Appropriate calculation equations are used according to the measurement data.
For example, as shown in
Here, an example has been described in which the region is narrowed down by two stages, i.e., a large region and a region with high accuracy, but the region may be set up in one stage if it is desired to set up the region in a shorter time even if the accuracy is somewhat low, or may be set up in three or more stages if it is desired to set up a region with higher accuracy. In addition, although an example in which an optical flow map is formed and then divided into regions has been described here, a region may be limited a little by using an image before the optical flow processing and thereafter the optical flow process may be performed. For example, a region in which there is no obvious living body may be excluded from the region in which the optical flow processing is performed at the stage of the image before the optical flow processing.
The region-of-interest selection unit 313 may select the region of interest using the indices SD and BP by the following method. For example, when the BP is larger than that of other regions, it is considered that periodic body movements are reflected. As for SD, it can be a high value in both cases of large noise other than body movement and cases of large movement. But, it is considered to be a noise when SD is large in spite of small BP or when SD is protruding in comparison with other areas. Therefore, for example, the region of interest selection unit 313 can select a region of interest by first selecting a plurality of regions having a large BP, and excluding a region in which BP or SD exceeds a predetermined threshold.
Further, in order to improve the accuracy of the index value, in particular, the SD, the time-series data of the variation may be subjected to filtering processing or regression processing of an arbitrary order before the index value is calculated. Further, since the waveform of the biological information does not necessarily have perfect periodicity, a typical waveform of the biological information as a target may be prepared and a similar shape may be searched for. When searching for a similar shape, AI such as machine learning may be applied. In addition, a frequency analysis technique using Wavelet transform may be adopted.
As an example,
In the above example, the region-of-interest selection unit 313 selects the region of interest using BP and SD, but it is also possible to use an index such as a ratio of BP and SD. Among factors that increase SD, frequency distribution shown in
As shown in
When the region of interest is determined by the region-of-interest selection unit 313, the signal analyzing unit 310 uses the video signal sent from the cameras 210 to calculate the variation of the region of interest by the optical flow calculator 311A when the imaging apparatus 100 starts imaging (examination), and sends the calculated variation to the imaging apparatus 100 as biological data (S5). Imaging apparatus (medical imaging apparatus) 100, as shown in
According to the present embodiment, since the optical flow is calculated from the video signal acquired by the camera, the region of interest in which the biological information with large motion and small noise is accurately obtained is determined, and the biological information from the region of interest is acquired, the biological information (motion of the subject) can be accurately acquired without depending on the region-of-interest setting skill of the examiner or the individual difference of the subject. In addition, since the camera is used, the labor and time required for setting a sensor to the subject or the like can be eliminated.
Although the case where the imaging apparatus 100 is an MRI apparatus has been described as an example, the imaging apparatus may be a medical imaging apparatus other than the MRI apparatus, or the biological signal processing system of the present invention may be applied to an examination apparatus such as an endoscope or an ultrasonic apparatus inserted into the body. In that case, the examination space is to be construed in a broad sense to include supports and struts for supporting the examination apparatus.
It is also possible to display, as a GUI, a graph of the temporal change of body movement such as breathing or the like as shown in
In the first embodiment, the optical flow is calculated using the video signal from the camera, and the optical flow is analyzed to select the region of interest. In the present embodiment, however, a stereo camera is used to analyze the distance from the camera calculated by utilizing the deviation of the image by the left and right cameras to the subject, and select the region of interest.
In the present embodiment, the configuration of the signal processing apparatus 300 is the same as that of the first embodiment except that the optical flow calculator 311A of
Stereo camera 220, when the body axis direction of the subject is the Y direction and the lateral direction perpendicular thereto is X the direction, as shown in
Signal analyzing unit 310 (distance calculator), when inputting the video signal of the left camera and the right camera of the stereo camera 220, detects the deviation S of the images, for each frame of both video signals, and calculates the distance D from the focal point to the object using the focal length f of both cameras and the reference length B (distance between the focal point and the focal point) according to the following equation (3). [Equation 3]
D=B×f/S (3)
The deviation amount S of the images of the left camera and the right camera can be calculated by a method such as block matching. In this case, one image is made to a standard and divided into a plurality of regions. For each region, a region having a high correlation in the other image is determined, and the deviation S between the images is calculated for each frame. Thus, the variation of the distance D for each region is obtained. This variation, similar to the graph shown in
When the variation of the distance between the subject and the camera is calculated by the distance calculator, index calculation unit 312 calculates an indicator of the magnitude of noise and movement using the distance variation for each region. The index may be the standard deviation SD of the variation, the band power BP, or a combination thereof, similarly to Embodiment 1, and can be calculated in the same manner as Embodiment 1. After that, the region-of-interest selection unit 313 selects a region having a small noise and a large motion using the index calculated for each region by the index calculation unit 312, calculates a variation from the selected region of interest, and outputs the variation to the imaging apparatus as a biological information signal, similarly to Embodiment 1.
According to the present embodiment, by using a stereo camera, a distance directly reflecting the body movement can be obtained so that it is possible to perform accurate region selection.
In Embodiment 1 and Embodiment 1, the case where only one body movement (for example, breathing movement) is obtained as the biological information signal has been described. It is also possible to obtain breathing movement and beating at the same time. As shown in
Therefore, when two body movements are monitored, the index calculation unit 312 calculates BP for each of the two body movements by varying the integration range (band f1-f2) represented by the equation (2), and the region-of-interest selection unit 313 selects the region of interest for each type of body movement based on the calculated BP and SD of the entire variation.
Embodiment 1 and Embodiment 2 use the video signal from the camera, as the biological information measuring apparatus 200. It is also possible to use a ranging meter or the like using a non-contact infrared or millimeter wave. Also in such a case, by obtaining the variation of the distance for each region, it is possible to perform the analysis and selection of the region of interest of the variation using the indicator similarly to Embodiments 1 and 2.
The biological signal processing system of the present embodiment is characterized in that it deals with not only breathing motion and pulsation but also motion and abnormal situation of a subject that may occur suddenly. The method of acquiring the biological signal may employ any of the methods of the above-described embodiments. Signal processing apparatus 300 (signal analyzing unit 310) of the present embodiment, as shown in
Hereinafter, the operation of the biological signal processing system according to the present embodiment will be described with reference to the flow of
Also in the present embodiment, biological information is acquired from the biological signal measuring apparatus 200 (S1), motion information (variation) is calculated (S2), and an index indicating the magnitude of noise and the magnitude of motion is calculated (S3). When the region of interest is selected based on the index (S4), imaging is started, and during imaging, the biological information obtained from the selected region of interest among the biological information acquired by the biological information measuring apparatus 200 is analyzed to calculate motion information (S5). Signal processing apparatus 300 displays this information on the display apparatus 400 provided in the signal processing apparatus 200 or a display device provided in the imaging apparatus 100, and transmits the information to the inspection apparatus (imaging apparatus) 100. In the case of synchronous imaging, as shown in
Imaging is performed while referring to such movement information. When there is a large movement, for example, as shown on the right side of the graph of
Thereafter, it is as described above that the imaging is continued while referring to the biological information from the newly set region of interest (S8)
According to the present embodiment, even when an unexpected movement other than the usual body movement of interest occurs, it is possible to immediately update the region of interest, and it is possible to continuously acquire a highly accurate biological information signal.
Further, the amount of change (corresponding to the differential value of the variation) may be used for detecting the unexpected movement, rather than the variation (coordinate information) as shown in
The methods of the above-described embodiments and variations can be applied to various imaging fields such as medicine. In addition, the present invention is not limited by the above embodiments and modifications.
Number | Date | Country | Kind |
---|---|---|---|
2020-219210 | Dec 2020 | JP | national |