The present invention relates to an apparatus and a method for determining respiration signals from a subject, wherein image data determined from the subject in a field of view is received and the respiration signals are determined on the basis of movement pattern determined in the image data.
Vital signals of a subject or a patient and in particular the respiration rate of a subject can be monitored remotely using contactless sensors such as a video camera. A general method for determining a respiration rate from image data by means of pattern detection is known e.g. from WO 2012/140531 A1. Since the subject to be measured can be located freely in the field of view of the camera and since the relevant area from which the vital signs should be derived can be located freely in the field of view of the camera, the subject and the relevant area have to be detected and defined for extraction of the desired vital sign information such as the respiration rate of the subject. Further, different movement pattern indicative for vital sign information and not indicative for vital sign information have to be identified and distinguished for a precise remote measurement of the vital sign information.
The traditional identification of the region of interest in general is based on detection of human being, e.g. the face or the chest or by using background segmentation. For identification of a human being and for measuring the vital signs from the remote image detection measurement such as a pulse or a respiration rate from a region of interest, US 2009/0141124 suggests to detect the contour of an infrared video segment to select the region of interest representing a portion of the subject to be measured.
Further, WO 2012/093320 A2 discloses a video detection device for detecting vital sign information from a subject, in particular photo-plethysmography signals from the subject, wherein the video data is divided in different blocks in order to select a region of interest which is in this case the skin of the subject in order to determine the vital sign information automatically in the field of view.
The traditional method for measuring the respiration is the inductive plethysmography wherein the respiration is detected by a breathing band measuring changes in the chest or abdomen cross-sectional area by placing a wire turn around the torso of the subject. Typically two breathing bands are used in order to distinguish thoracic and abdominal breathing. To measure the respiration of the subject precisely and to identify special injuries or paralysis, the independent measurement of the thoracic and abdominal breathing is necessary.
The disadvantage of the known methods for measuring respiration signals from a subject is that only one respiration signal can be determined remotely from the subject wherein only a coarse respiration analysis is possible or that the systems which measure precisely different respiration signals from the subject are uncomfortable for the user due to the use of contact measurement sensors.
It is an object of the present invention to provide an improved apparatus and a corresponding improved method for determining respiration signals from a subject which is more precise and more comfortable for the user.
According to one aspect of the present invention, an apparatus for determining respiration signals from a subject is provided, comprising:
According to another aspect of the present invention a method for determining respiration signals from a subject is provided, comprising the steps of:
According to still another aspect of the present invention, a computer readable non-transitory medium is provided having instructions stored thereon which, when carried out on a computer, cause the computer to perform the steps of the method according to the present invention.
The present invention is based on the idea to measure different respiration signals from one subject on the basis of a contactless measurement and to provide an improved and precise respiration measurement which is comfortable due to the contactless measurement for the user. The different respiration signals are determined on the basis of movement pattern determined from image data captured from the subject to be measured, wherein the movement pattern of different areas in the field of view are used to determine the different respiration signals. Hence, the movement of different portions of the subject corresponding to the respiration of the subject can be determined independently such that e.g. the thoracic and abdominal breathing can be determined independently and comfortable for the user so that the whole breathing information can be determined with low technical effort. On the basis of the different breathing signals, additional diagnostics can be performed so that the vital sign detection becomes more precise.
Preferred embodiments of the present invention are defined in the dependent claims. It should be understood that the claimed method has similar and/or identical preferred embodiments as the claimed apparatus and as defined in the dependent claims.
In a preferred embodiment, the processing unit is adapted to define a plurality of image sections in the image data and to determine one alternating signal corresponding to the vital sign information from each of the image sections on the basis of movement pattern detection. This is a possibility to identify the vital sign information from the whole field of view with low technical effort.
In a preferred embodiment, the processing unit is adapted to define the different image sections as an array of image sections in the image data. This is a simple solution to analyze the whole image data and to analyze the whole field of view in order to determine the different vital sign information of the subject.
In a preferred embodiment, the apparatus further comprises a frequency analysis unit for determining spectral parameter of the alternating signals determined from the different image sections and a selection unit for selecting different image sections on the basis of the spectral parameter as the different areas to determine the different respiration signals. This is a reliable possibility to determine different regions of interest in the field of view from which vital sign information can be derived.
In a preferred embodiment, the spectral parameter determined from the different image sections is a spectral energy of the alternating signals. This is a possibility to distinguish vital sign information from disturbing signals and noise with high reliability.
In a preferred embodiment, the selection unit is adapted to select the image sections if the spectral energy of a predefined frequency band of the alternating signals exceeds a threshold level. This is a possibility to analyze the spectral parameter with low technical effort.
In a preferred embodiment, the different respiration signals are determined on the basis of motion vectors derived from different portions of the subject. By means of the motion vector derived from different portions of the subject, the different respiration signals corresponding to e.g. thoracic and abdominal respiration can be determined.
In a preferred embodiment, the different respiration signals are time-dependent alternating signals having different waveforms. This is a possibility to determine additional diagnostic information from the subject in addition to the simple respiration rate.
In a preferred embodiment, the different respiration signals are time-dependent alternating signals having a phase shift to each other. This is a possibility to distinguish different respiration signals of the subject in order to determine additional diagnostic information.
In a preferred embodiment, the evaluation unit is adapted to determine a signal difference of the different respiration signals as additional respiration information from the subject. This is a solution to automatically determine additional respiration information beyond the respiration rate for additional diagnostics.
In a preferred embodiment, the evaluation unit is adapted to determine the phase shift of the different respiration signals and to combine the different respiration signals to one general respiration signal considering the determined phase shift. This is a possibility to determine a single respiration signal having an increased preciseness and a higher reliability.
In a further preferred embodiment, the evaluation unit is adapted to determine an array of respiration signals on the basis of the different respiration signals derived from the different image sections to provide a spatial respiration map of the subject. This is a possibility to determine the whole respiration information from the subject in order to provide additional diagnostic possibilities.
In a further preferred embodiment, the selection unit is adapted to determine a weight factor for each of the selected different image sections and wherein the evaluation unit is adapted to determine the different respiration signals on the basis of the alternating signals of selected image sections weighed by means of the respective weight factor. This is a possibility to consider a signal strength of the alternating signals in order to increase the preciseness of the determined respiration signal, since disturbing signals or noisy signals are less considered than those signals which have a high strength.
It is further preferred if the selection unit is adapted to perform the selection on a regular basis and wherein the weight factor for each of the selected image sections is determined on the basis of a frequency of selection of the respective image section. This is a possibility to determine the signal strength and the weight factor with low technical effort.
As mentioned above, the present invention provides a possibility to determine different vital sign information from one subject on the basis of contactless remote measurements by using image data determined from a field of view including the subject to be measured. Since the alternating signals are determined on the basis of movement pattern determined from different areas of the field of view, respiration signals from different portions of the subject, e.g. the thorax and the abdomen can be determined corresponding to different respiration techniques in order to increase the preciseness of the respiration detection and to determine additional information from the respiration of the subject. Hence, additional diagnostics can be performed and the detection of the respiration has a higher reliability and is more precise and can be determined comfortable on the basis of contactless measurements.
In still another aspect of the present invention an apparatus for determining respiration signals from a subject is presented, comprising:
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. In the following drawings
The apparatus 10 is connected to an image detection device 22, e.g. a monochromatic camera which can be used for recording image frames of the subject 12. The image frames can be derived from electromagnetic radiation 24 emitted or reflected by the subject 12. For extracting the image information from the image data, e.g. a sequence of image frames, the image detection device 22 is connected via an interface 28 to an image processing unit 30. The image detection device 22 may be part of the apparatus 10 or may be an external camera 22 such that the image data 26 is merely provided to the interface 28 in order to provide the image data 26 to the apparatus 10 in general.
The image detection 22 is adapted to capture images belonging to at least a spectral component of the electromagnetic radiation 24. The image detection device 22 may provide continuous image data or a discrete sequence of image frames captured from a field of view including the subject 12 to be measured.
The image processing unit 30 is adapted to receive the image data 26 from the image detection device 22 via the interface 28, to evaluate the image data 26 in general and to detect different regions of interest of the subject 12, e. g. the thorax 18 and the abdomen 20 as indicative portions of the respiration of the subject 12. In order to detect the region of interest, e.g. the thorax 18 and/or the abdomen 20, the image processing unit 30 is adapted to divide the captured images in sections or areas of the field of view and to evaluate the image sections separately in order to determine the region of interest. The image processing unit 30 divides the captured images into the image sections and detects motion vectors from the different sections corresponding to the motion of the subject in the field of view including the motion of the thorax region 18 and/or the abdomen region 20 of the subject 12 as indicative portions of the respiration. The motion vectors are determined by means of pattern detection in the image sections or by means of edge detection in the image sections. A method for edge or pattern detection and for deriving the motion vectors from the captured image frames is for example disclosed by WO 2012/140531 A1.
The imaging processing unit 30 is connected to an analysis unit 32. The image processing unit 30 determines alternating signals from the motion vectors from each of the image sections and provides the alternating signals to the analysis unit 32.
The analysis unit 32 determines a spectral parameter of each of the alternating signals by means of a frequency analysis unit included in the analysis unit 32 as described in detail in the following. The spectral parameter of each of the sections in the image data 26 are analyzed by a selection unit which is part of the analysis unit 32. The selection unit selects those sections of the image data from which an alternating signal is derived which is supposed to correspond to a respiration signal. The selection unit selects the sections on the basis of the respective spectral parameter. The spectral parameter is a frequency spectrum or a spectral energy distribution of each of the alternating signal. Since the respiration signal of the subject has a characteristic spectral energy distribution or a characteristic frequency, the selection unit can select the sections which comprise the respiration signals of the subject 12, and, therefore, the selection unit identifies the thorax 18 and/or the abdomen 20 of the subject 12 in the image data 26 for determining different respiration signals.
The selection unit also determines a weight factor for each of the different image sections dependent on the frequency analysis as described in the following. The weight factor is in general dependent on the frequency how often each of the image section is selected. Hence, the weight factor represents a factor corresponding to a signal strength of the alternating signals so that the respective alternating signal from each of the image sections can be considered according to the signal quality.
The analysis unit 32 is connected to an evaluation unit 34 and provides the alternating signals to the evaluation unit 34 for determining respiration signals corresponding to the respiration of the subject 12. The evaluation unit 34 receives the alternating signals determined from the different image sections and the respective weight factors for the different image sections from the analysis unit 32 and calculates the different respiration signals on the basis of the alternating signals, the weight factors and the different regions from which the alternating signals are derived. Hence, the respiration signals are calculated on the basis of the image data 26 and can be determined entirely contactless, wherein the respiration signals can be derived independently from different portions, e.g. the thorax 18 and the abdomen 20 of the subject 12.
The so-calculated respiration signals can be provided to a display 36 to display the measured respiration signals continuously or frequently.
Hence, the thoracic and abdominal breathing can be determined entirely contactless and independently from each other so that the respiration measurement becomes more precise and additional information can be derived from the respiration of the subject 12 in order to diagnose additional injuries such as spinal cord injuries or diaphragmic paralysis.
Over time as indicated by an arrow 40, the indicative portions 18, 20 are moved between a contracted position indicated by reference numerals 18a, 20b and 18c and an extracted position indicated by 20a, 18b and 20c. Essentially, based on the motion pattern, for instance the respiration rate or the respiration rate variability or the respiration volume can be assessed by means of pattern or edge detection in the captured image sequence. While the indicative portions 18, 20 are pulsating over time, the head 16 as a non-indicative portions remains substantially motionless. It should be understood that the thorax 18 and the abdomen 20 are examples as indicative portions for the respiration and that also other portions of the subject 12 can be detected in order to determine additional respiration signal such as movements at the lower rib of the subject 12.
Certainly, also the head 16 undergoes diverse motion over time. However, these motions do not correspond to the periodic pulsation of the thorax 18 or the abdomen 20 and can be distinguished by means of the frequency analysis unit.
The alternating signals S are derived from each of the image sections of the field of view wherein a plurality of image sections comprise vital sign information such as a breathing rate and many image sections may comprise disturbing signals which are not related to vital sign information of the subject 12 or other alternating signals which comprise mostly high-frequency noise. In order to identify those image sections from which vital sign information can be derived, the analysis unit 32 comprises the frequency analysis device to perform a frequency analysis of the alternating signals S. The frequency analysis is preferably performed by filtering the alternating signals S and/or by performing a Fourier Transformation, in particular a Fast Fourier Transformation (FFT) of the alternating signal S. From the alternating signals S, a frequency spectrum is derived in order to identify the image section including vital sign information corresponding to the respiration of the subject 12 as described in the following.
On the basis of these data, which are determined for each of the image sections 48, the selection unit selects those image sections which include the respiration signals and may combine those selected image sections 48 to the region of interest, which is in
On the basis of the different alternating signals S which are derived from the image sections 48 of the region of interest 50, the evaluation unit 34 determines the different respiration signals corresponding to the breathing motion of the thorax 18 and the abdomen 20. The analysis unit 32, in particular the selection unit of the analysis unit 32 determines a weight factor for each of the selected image sections 48 of the region of interest 50 in order to weight the alternating signals S of the different sections 48 on the basis of the signal quality. The weight factor determined by the analysis unit 32 may be calculated on the basis of the frequency how often the respective image section is selected by the selection unit. In other words, the alternating signals S from those image sections 48 which are selected more often as a selected image section 48 are given more weight and the image sections 48 selected less often are given less weight to calculate the respective respiratory signals.
The alternating signals S comprising identical or corresponding wave forms are combined (by the evaluation unit 32) to a single respiration signal since these alternating signals S are considered to be derived from the same indicative portion 18, 20. If the alternating signals from different sections 48 have a larger difference, e.g. phase shift, those alternating signals S are considered to be derived from different indicative portions 18, 20 and are not combined directly to one respiration signal. The combination steps are performed by the evaluation unit 32.
It is also possible to determine the respiration signals of each of the image sections 48 separately and to determine by means of the evaluation unit 34 a spatial respiration map of the subject 12 and the region of interest 50.
a shows a timing diagram comprising three different respiration signals R1, R2 and R3 which are derived by motion vector detection contactless from different portions of the subject 12. The regions of the subject 12 from which the respiration signals R1, R2, R3 are derived are schematically shown in the captured images of
The first respiration signal R1 is determined from a region of interest 52 including the thorax 18 or the chest 18 as indicated in
The respiration signals R1 and R2, R3 have their peaks corresponding to the movement of the respective indicative portion 18, 20 at different points in time t1, t2 and are phase shifted to each other as indicated by Δt1 and Δt2. The phase shift Δt1, Δt2 corresponds to the alternating movement of the thorax 18 and the abdomen 20 due to the respiration of the subject 12. Hence, the different respiration signals R1, R2, R3 can be derived independently by means of the apparatus 10 contactless and remotely and additional information like the phase shift Δt1, Δt2 can be determined from the remote measurement.
On the basis of the additional information like the phase shift Δt1, Δt2 additional diagnostics can be performed in order to determine certain injuries of the subject 12.
In a certain embodiment, the phase shift Δt1, Δt2 of the respiration signals R1, R2, R3 is determined and a general respiration signal is determined by combining the different respiration signals R1, R2, R3 derived from the different regions of interest 52, 54, 56 indicative portions 18, 20 wherein the phase shift is considered and the signals are respectively shifted so that the respiration signals R1, R2, R3 are in phase before the signals are combined. By means of this combination, a reliable respiration signal can be determined even if the single respiration signals R1, R2, R3 have a poor signal strength.
In a simple embodiment of the invention, the image data is evaluated on the basis of the different rows of the grid 46 wherein one alternating signal S of one image section 48 of each of the rows is selected having the highest signal strength and the respective respiration signal R1, R2, R3 is determined for each of the rows on the basis of the one selected image section 48. This can reduce the technical effort of the apparatus 10 and the calculation time for determining the respiration signals R1, R2, R3.
At step 76, the different respiration signals R1, R2, R3 are displayed by means of the display 36.
At step 78, the method 60 ends. Hence the method 60 can determine different respiration signals R1, R2, R3 from the one subject 12 based on motion detection of the different indicative portions 18, 20.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.
In the claims, the word “comprising” does not exclude other elements or steps, and the indefinite article “a” or an does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
Furthermore, the different embodiments can take the form of a computer program product accessible from a computer usable or computer readable medium providing program code for use by or in connection with a computer or any device or system that executes instructions. For the purposes of this disclosure, a computer usable or computer readable medium can generally be any tangible device or apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution device.
In so far as embodiments of the disclosure have been described as being implemented, at least in part, by software-controlled data processing devices, it will be appreciated that the non-transitory machine-readable medium carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure.
The computer usable or computer readable medium can be, for example, without limitation, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, or a propagation medium. Non-limiting examples of a computer readable medium include a semiconductor or solid state memory, magnetic tape, a removable computer diskette, a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk, and an optical disk. Optical disks may include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W), and DVD.
Further, a computer usable or computer readable medium may contain or store a computer readable or usable program code such that when the computer readable or usable program code is executed on a computer, the execution of this computer readable or usable program code causes the computer to transmit another computer readable or usable program code over a communications link. This communications link may use a medium that is, for example, without limitation, physical or wireless.
A data processing system or device suitable for storing and/or executing computer readable or computer usable program code will include one or more processors coupled directly or indirectly to memory elements through a communications fabric, such as a system bus. The memory elements may include local memory employed during actual execution of the program code, bulk storage, and cache memories, which provide temporary storage of at least some computer readable or computer usable program code to reduce the number of times code may be retrieved from bulk storage during execution of the code.
Input/output, or I/O devices, can be coupled to the system either directly or through intervening I/O controllers. These devices may include, for example, without limitation, keyboards, touch screen displays, and pointing devices. Different communications adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems, remote printers, or storage devices through intervening private or public networks. Non-limiting examples are modems and network adapters and are just a few of the currently available types of communications adapters.
The description of the different illustrative embodiments has been presented for purposes of illustration and description and is not intended to be exhaustive or limited to the embodiments in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art. Further, different illustrative embodiments may provide different advantages as compared to other illustrative embodiments. The embodiment or embodiments selected are chosen and described in order to best explain the principles of the embodiments, the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various embodiments with various modifications as are suited to the particular use contemplated. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
13162887.7 | Apr 2013 | EP | regional |
This application claims the benefit of U.S. provisional application Ser. No. 61/809,964 filed Apr. 9, 2013 and European provisional application serial no. 13162887.7 filed Apr. 9, 2013, both of which are incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
61809964 | Apr 2013 | US |