The present invention is related to the field of data processing, more specifically to the treatment and analysis of images, in particular the segmentation of medical images, and concerns an automatic segmentation process of a 3D medical image by one or several neural networks through structured convolution according to the geometry or structuration of the 3D medical image.
A three-dimensional image made from a medical imaging device such as a scanner, MRI, ultrasound, CT or SPEC type image is composed of a set of voxels, which are the basic units of a 3D image. The voxel is the 3D extension of the pixel, which is the basic unit of a 2D image. Each voxel is associated with a grey level or density, which can be considered to be the result of a 2D function F(x, y) or a 3D function F(x, y, z), where x, y and z denote spatial coordinates (see
In 3D images, voxels can be seen in 2D according to various axes or planes. The three main axes or planes in medical images are the axial, sagittal and frontal ones (
Typically, a 2D or 3D medical image contains a set of anatomical and pathological structures (organs, bones, tissues, . . . ) or artificial elements (stents, implants, instruments, . . . ) that clinicians have to delineate in order to evaluate the situation and to define and plan their therapeutic strategy. In this respect, organs and pathologies have to be identified in the image, which means labelling (for example coloring) each pixel of a 2D image or each voxel of a 3D image. This process is called segmentation.
There are many known methods to perform a segmentation, in particular automatic methods making use of algorithms, especially AI algorithms.
In this context, numerous variations of neural networks have been used in the state of the art, all based on standard non-specific architectures, resulting globally in a waste of inappropriate resources and a lack of efficiency and accuracy.
The main aim of the present invention is to propose a new method and a new system intended for overcoming the aforementioned limitations.
Therefore the present invention relates to an automatic segmentation method of features, such as anatomical and pathological structures or instruments, which are visible in a 3D medical image of a subject, composed of voxels,
said method being characterised in that it consists
in providing a global software means or arrangement combining N different convolutional neural networks or CNNs, with N≥2, and having a structured geometry or architecture adapted and comparable to that of the image volume,
and in analysing voxels forming said volume of the 3D image according to N different reconstruction axes or planes, each CNN being allocated to the analysis of the voxels belonging to one axis or plane.
The invention will be better understood using the description below, which relates to several preferred embodiments, given by way of non-limiting examples and explained with reference to the accompanying drawings, wherein:
Shown on
According to the invention, said method consists in providing a global software means or arrangement combining N different convolutional neural networks or CNNs, with N≥2, and having a structured geometry or architecture adapted and comparable to that of the image volume, and in analysing voxels forming said volume of the 3D image according to N different reconstruction axes or planes, each CNN being allocated to the analysis of the voxels belonging to one axis or plane.
Thus, the invention provides for a structured organization and a coordinated working together of multiple CNNs taking into account the very geometry, structuration and content of a medical image.
This specific composite computing system (possibly grouped in a single framework) which combines N different CNNs (with N≥2, preferably N≥3) according to N different reconstruction axes or planes of the 3D image volume allows to extend the use of known CNN for analising and segmenting 2D images, to 3D images.
Typically, a known CNN algorithm which may be used within the method and the system of the present invention is “U-Net” (see for example: “U-Net: Convolutional Networks for Biomedical Image Segmentation”; O Ronneberger et al.; MICCAI 2015, Part III, LNCS 3951, pp 234-“241, Springer IPS).
“U-Net” may be implemented in connection with other known architectures such as “ResNet” or “DenseNet”.
Advantageously, the inventive method may consist, as also shown on
By segmenting the 3D image volume for analysing purposes and by merging (combining) the results of these partial analyses into a single 3D image segmentation, the invention allows to realise complex segmentation procedures with limited resources and to deliver quickly accurate and somehow cross-checked results.
The combination or merging of the results of the N analyses and segmentations may be performed:
According to a first embodiment of the invention shown on
In this case, the very internal structure of the so formed single meta-CNN integrates the 3D axes image analysis, as well as the combination of the information resulting from the different 2D analyses and segmentations.
Of course, such a multiple parallel 2D treatment with a combination of the results of these treatments can also be managed by a parallel arrangement of N independent CNNs, not integrated in a single framework (
According to a second alternative embodiment of the invention, shown on
In this case, the CNN 2D sequential treatment scheme is integrated in the global architecture of the algorithmic framework or in the structured geometry of the CNN arrangement, the final combination of the information of the sequence being thus intrinsically integrated in the structure of said framework or arrangement.
In relation to a preferred embodiment of the invention, shown on
Advantageously, the first operational step is carried out by N different CNNs operating in parallel or sequentially, wherein each of these CNNs automatically segments, independently from the others, anatomical and pathological structures, or instruments, that can be seen in the 3D medical image, each CNN analysing all the voxels forming the volume of the 3D image according to a different reconstruction plane for each one of the N different CNNs.
According to a first alternative implementation of the invention, shown in
According to a second alternative implementation of the invention; shown in
In a 3D image volume, the neighbouring voxels of a given voxel may concern three different groups of voxels, namely (as shown on
As illustrated in
In addition to the foregoing or alternatively to it, the N reconstruction planes may also comprise:
The present invention also encompasses, as symbolically shown in
Said system is characterised in that it comprises at least one computer device hosting, in a structured arrangement, and allowing the coordinated working, in a sequential or a parallel organization, of N different convolutional neural networks (CNN), with N≥2, each CNN, being adapted and configured to perform, automatically and independently from the other CNNs, a segmentation of anatomical and pathological structures, or instruments, that can be seen in a 3D medical image to be treated, by analysing voxels forming said volume of the 3D image according to N different reconstruction axes or planes, each CNN being allocated to the analysis of the voxels belonging to one axis or plane.
Preferably, said system also comprises means to combine, and possibly display, the results of the analyses and segmentations performed by said N different CNNs.
According to a first alternative, the N different CNNs may be arranged in a serial architecture in such a way that each CNNi of said N different CNNs analyses all the voxels forming the volume of the 3D image according to a specific reconstruction plane for each of the N different CNNs and uses the results provided by the previous network CNNi-1 when it exists (
According to a second alternative, the N different CNNs may be arranged in a parallel architecture, possibly within a single algorithmic framework, the results of said N different CNNs being combined in a final stage (
Of course, the invention is not limited to the at least one embodiment described and represented in the accompanying drawings. Modifications remain possible, particularly from the viewpoint of the composition of the various elements or by substitution of technical equivalents without thereby exceeding the field of protection of the invention.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2019/050542 | 1/10/2019 | WO | 00 |