Patent Application
20250029255
The present disclosure relates to medical technology, and in particular to an image processing method, apparatus, and device.
An endoscope is a commonly used medical instrument, consisting of a light guide structure and a group of lenses. After an endoscope enters a target object, the endoscope may be used to collect a visible light image and a fluorescence image of a designated location inside the target object, and generate a fused image based on the visible light image and the fluorescence image. The fused image can clearly display the normal tissue and the diseased tissue at the designated location inside the target object, i.e., the normal tissue and the diseased tissue at the designated location inside the target object can be distinguished based on the fused image, such that the target object can be inspected and treated based on the fused image, which can accurately determine which tissue needs to be cut.
When a fused image is generated based on a visible light image and a fluorescence image, the fluorescence image is superimposed on the visible light image for display, which may result in a part of the visible light image to be covered, causing certain visual obstacles, and subsequently resulting in a poor effect of the fused image and failure to clearly display the normal tissue and the diseased tissue at a designated location inside a target object, which affects the quality and efficiency of a doctor's operation and provides poor use experiences.
In view of this, the present disclosure provides an image processing method to improve the quality and efficiency of a doctor's operation.
The present disclosure provides an image processing method, including: acquiring a visible light image and a fluorescence image corresponding to a designated location inside a target object, where the designated location includes diseased tissue and normal tissue; determining a to-be-cut boundary corresponding to the diseased tissue from a to-be-detected image, where the to-be-detected image is the fluorescence image, or the to-be-detected image is a fused image of the visible light image and the fluorescence image; and generating a target image, where the target image includes the visible light image and the to-be-cut boundary.
The present disclosure provides an image processing apparatus, including: an acquisition module, configured to acquire a visible light image and a fluorescence image corresponding to a designated location inside a target object, where the designated location includes diseased tissue and normal tissue; a determination module, configured to determine a to-be-cut boundary corresponding to the diseased tissue from a to-be-detected image, where the to-be-detected image is the fluorescence image, or the to-be-detected image is a fused image of the visible light image and the fluorescence image; and a generating module, configured to generate a target image, where the target image includes the visible light image and the to-be-cut boundary.
The present disclosure provides an image processing device, including: a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is configured to execute the machine-executable instructions to implement the image processing method disclosed in the examples of the present disclosure.
The present disclosure provides a machine-readable storage medium storing computer instructions, and when the computer instructions are called by one or more processors, the above image processing method is executed by the one or more processors.
It can be seen from the above technical solution that in embodiments of the present disclosure, the to-be-cut boundary corresponding to the diseased tissue (i.e., the boundary of the focus region) may be determined. and a target image is generated based on the visible light image and the to-be-cut boundary, i.e., the to-be-cut boundary is superimposed on the visible light image for display. Since the to-be-cut boundary is superimposed on the visible light image for display instead of superimposing the fluorescent image on the visible light image for display, the problem of most parts of the visible light images being covered is avoided, the situation of visible light images being covered by fluorescent development is improved, and visual obstructions are avoided or alleviated. The effect of the target image is better, which can clearly display the normal tissue and diseased tissue at the designated location inside the target object, improving the quality and efficiency of the doctor's operation and providing a better user experience. Since the target image has a to-be-cut boundary, the doctor may know the to-be-cut boundary corresponding to the diseased tissue, and may cut according to the to-be-cut boundary, thereby improving the quality and efficiency of the doctor's operation.
In order to more clearly illustrate embodiments of the present disclosure or the technical solutions in the related art, the accompanying drawings required in the embodiments of the present disclosure or the related art are briefly introduced below, and the accompanying drawings in the following description are merely some embodiments recorded in the present disclosure, and for those of ordinary skill in the art, other drawings may also be obtained based on these drawings of embodiments of the present disclosure.
The terms used in examples of the present disclosure are intended to describe the particular examples only, and not to limit the present disclosure. The terms determined by “a”, “the” and “said” in their singular forms in the present disclosure and the appended claims are also intended to include their plural forms, unless clearly indicated otherwise in the context. It should also be understood that the term “and/or” as used herein is and includes any and all possible combinations of one or more of the associated listed items.
It is to be understood that, although terms “first”, “second”, “third” and the like may be used in the present disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same category with each other. For example, without departing from the scope of the present disclosure, first information may be referred as second information; and similarly, second information may also be referred as first information. Depending on the context, the word “if” as used herein may be interpreted as “when” or “upon” or “in response to determining”.
Referring to
For example, an endoscope may be inserted into a designated location (i.e., the location to be inspected, that is, the region inside the patient that needs to be inspected, and this designated location is not limited) inside a target object (e.g., a patient or other subjects), to collect images of the designated location inside the target object, and output the images of the designated location inside the target object to a display device and a storage device. Users (e.g., medical staff, etc.) observe the images displayed on the display device to inspect abnormal parts such as a bleeding part or a tumor part at designated locations inside the target object. Users may access images stored in the storage device to conduct postoperative reviews and surgical training, etc.
The endoscope may collect images of designated locations inside the target object and input the images to the camera system host. The light source may provide a light source for the endoscope, i.e., illumination light is emitted from the front end of the endoscope, such that the endoscope may collect relatively clear images inside the target object. After receiving an image, the camera system host may input the image to a storage device, to store the image. In subsequent processes, the user may access the image in the storage device, or access a video (a video composed of a large number of images) in the storage device. After receiving the image, the camera system host may also input the image to a display device, to display the image, and the user may observe the image displayed by the display device in real time.
As illustrated in
The light source may include an illumination control unit and an illumination unit. The illumination control unit is configured to receive an indication signal from the camera system host and control the illumination unit to provide illumination light to the endoscope based on the indication signal.
The camera system host is configured to process the image data received from the endoscope and transmit the processed image data to the display device and the storage device, and the display device and the storage device are external devices of the camera system host.
The camera system host may include an image input unit, an image processing unit, an intelligent processing unit, a video encoding unit, a control unit, and an operating unit. The image input unit is configured to receive the signal sent by the endoscope and transmit the received signal to the image processing unit. The image processing unit is configured to perform image signal processing (ISP) operation on the image input by the image input unit, including but not limited to brightness conversion, sharpening, fluorescence dyeing, scaling, etc. The image processed by the image processing unit is transmitted to the intelligent processing unit, the video encoding unit, or the display device. The intelligent processing unit is configured to perform intelligent analysis on the image, including but not limited to scene classification (e.g., a thyroid surgery scene, a hepatobiliary surgery scene, and an ear-nose-throat surgery scene) based on deep learning, instrument head detection, gauze detection, and dense fog classification (dense fog classification means that the use of electric knife to cut tissue during operation will produce smoke, which will affect the field of vision. By classifying the concentration of smoke, for example, no fog type, light fog type, thick fog type, and subsequently being linked to the insufflator or defogging algorithm for defogging processing.). The image processed by the intelligent processing unit is transmitted to the image processing unit or the video encoding unit, and the image processing unit processes the image processed by the intelligent processing unit in a manner including but not limited to brightness conversion, box superimposing (box superimposing means superimposing a box pattern on the image, such as an recognition box, to mark a target recognition result, i.e., the superimposed box serves as a mark and a prompt), and scaling. The video encoding unit is configured to encode and compress the image and transmit the compressed image to the storage device. The control unit is configured to control various modules of the endoscope system, including but not limited to the illumination method of the light source, the image processing method, the intelligent processing method and the video encoding method. The operating unit may include but is not limited to a switch, a button, and a touch panel, etc., which is configured to receive an external indication signal and output the received indication signal to the control unit.
During the actual use process. it is found that, visible light images are not able to distinguish diseased tissue (such as cancerous tissue) enough, and doctors often need to judge and remove diseased tissue based on experiences, which easily causes problems such as excessive resection of normal tissue or unclean removal of diseased tissue. In order to distinguish normal tissue from diseased tissue, in a possible implementation, a white light endoscope and a fluorescence endoscope may be inserted into a designated location inside a target object, and images of the designated location inside the target object may be collected. For the convenience of distinction, the images collected by the white light endoscope are referred to as visible light images (also referred to as white light images), and the images collected by the fluorescence endoscope are referred to as fluorescence images.
By injecting fluorescent developer (such as indocyanine green (ICG)) into a designated location inside the target object in advance, the diseased tissue absorbs more fluorescent developer, and when the excitation light irradiates the diseased tissue, the fluorescent developer generates fluorescence, making the diseased tissue highlighted in the fluorescence image collected by the fluorescence endoscope. Based on the fluorescence image, normal tissue and diseased tissue may be distinguished, helping the medical staff to accurately distinguish diseased tissue.
Visible light images may be images generated based on visible light, as illustrated in
In summary, it may be seen that, in the actual use of the endoscope system, visible light images and fluorescence images of the designated location inside the target object may be collected, i.e., the camera system host may obtain the visible light images and the fluorescence images. After obtaining the visible light image and the fluorescence image, a fused image may be generated based on the visible light image and the fluorescence image. The fused image can clearly show the normal tissue and the diseased tissue at the designated location inside the target object, and the normal tissue and the diseased tissue at the designated location inside the target object may be distinguished based on the fused image.
When collecting fluorescence images of a designated location inside the target object, there are a plurality of development modes, and the forms of fluorescence images collected in different development modes are different. For example, during use, the fluorescence endoscopes are divided into positive development modes (a positive development mode may also be referred to as a positive direction development mode) and negative development modes (a negative development mode may also be referred to as a negative direction development mode). In a positive development mode, the location of the diseased tissue shows fluorescence, i.e., a development region of the fluorescence image corresponds to the diseased tissue. In a negative development mode, the locations outside the diseased tissue show fluorescence, and the locations of the diseased tissue do not show fluorescence, i.e., a non-development region of the fluorescence image corresponds to the diseased tissue.
In a positive development mode, it is necessary to adopt an injection method of the fluorescent developer that matches the positive development mode, and based on the injection method of the fluorescent developer that matches the positive development mode, it is necessary to ensure that the diseased tissue is developed and the normal tissue is not developed. In a negative development mode, it is necessary to adopt an injection method of the fluorescent developer that matches the negative development mode, and based on the injection method of the fluorescent developer that matches the negative development mode, it is necessary to ensure that the diseased tissue is not developed and the normal tissue is developed.
As illustrated in
The fluorescence image on the lower side of
Since there are a plurality of development modes, e.g., a positive development mode and a negative development mode, it is possible to pre-configure which development mode is used to collect the fluorescence image. For example, it is possible to pre-configure that the fluorescence image is collected in a positive development mode, and the fluorescence image collected in the positive development mode may refer to the fluorescence image illustrated in the upper side of
Regardless of which development mode is used to collect the fluorescence image, when a fused image is generated based on a visible light image and a fluorescence image, the fluorescence image is all superimposed on the visible light image for display, which may result in a part of the visible light image to be covered, causing certain visual obstruction, and subsequently resulting in a poor effect of the fused image and failure to clearly show the normal tissue and the diseased tissue at a designated location inside a target object, which affects the quality and efficiency of a doctor's operation.
For example, as illustrated in
In view of this, embodiments of the present disclosure provide an image processing method, which may determine the to-be-cut boundary corresponding to the diseased tissue (i.e., the to-be-cut boundary of the focus region), and superimpose the to-be-cut boundary of the focus region on the visible light image for display. so as to improve the situation of the visible light image being covered by fluorescent development, avoid or reduce visual interference, and the doctor may know the to-be-cut boundary of the focus region based on the image, and may perform cutting according to the to-be-cut boundary of the focus region (the development region in a positive development mode or the non-development region in a negative development mode), thereby improving the quality and efficiency of the doctor's operation.
The following is a description of the technical solutions of embodiments of the present disclosure in combination with specific embodiments.
Embodiments of the present disclosure provide an image processing method, the method may be applied to a camera system host, referring to
At step 401, a visible light image and a fluorescence image corresponding to a designated location inside a target object are acquired. where an acquisition time of the visible light image and an acquisition time of the fluorescence image may be the same.
For example, the designated location inside the target object may include diseased tissue and normal tissue, the visible light image may include a region corresponding to the diseased tissue and a region corresponding to the normal tissue, and the fluorescence image may include a region corresponding to the diseased tissue and a region corresponding to the normal tissue.
For example, when it is necessary to collect an image of the designated location (i.e., the location to be inspected, that is, the region inside the patient that needs to be inspected) inside the target object (e.g., a patient or other subjects), an endoscope (such as a white light endoscope and a fluorescence endoscope) may be inserted into the designated location inside the target object, the white light endoscope collects a visible light image of the designated location inside the target object, and the fluorescence endoscope collects a fluorescence image of the designated location inside the target object, and the camera system host may obtain the visible light image from the white light endoscope and the fluorescence image from the fluorescence endoscope.
For example, the development mode may be a positive development mode or a negative development mode. If the development mode is a positive development mode, when a fluorescence image corresponding to the designated location inside the target object is acquired, the fluorescence image is a fluorescence image corresponding to the positive development mode, i.e., the development region of the fluorescence image corresponds to the diseased tissue, and the non-development region of the fluorescence image corresponds to the normal tissue. If the development mode is a negative development mode, when a fluorescence image corresponding to the designated location inside the target object is acquired, the fluorescence image is a fluorescence image corresponding to the negative development mode, i.e., the development region of the fluorescence image corresponds to normal tissue, and the non-development region of the fluorescence image corresponds to diseased tissue.
For example, when the visible light image and the fluorescence image corresponding to a designated location inside the target object are acquired, since there may be diseased tissue and normal tissue at the designated location inside the target object, when the visible light image of the designated location inside the target object is acquired by the white light endoscope, the visible light image may include a region corresponding to normal tissue and a region corresponding to diseased tissue When a fluorescence image of the designated location inside the target object is acquired by the fluorescence endoscope, the fluorescence image may include a region corresponding to normal tissue and a region corresponding to diseased tissue.
In summary, the visible light image and the fluorescence image corresponding to the designated location inside the target object may be obtained, the visible light image may include a region corresponding to the diseased tissue and a region corresponding to the normal tissue, and the fluorescence image may include a region corresponding to the diseased tissue and a region corresponding to the normal tissue.
At step 402, a to-be-cut boundary corresponding to the diseased tissue is determined from a to-be-detected image, where the to-be-detected image is the fluorescence image, or the to-be-detected image is a fused image of the visible light image and the fluorescence image.
In a possible implementation, the following steps may be used to determine the to-be-cut boundary as described below.
At step 4021: a target region corresponding to the diseased tissue is determined from the to-be-detected image, where the target region may be a region in the to-be-detected image, and the target region in the to-be-detected image corresponds to the diseased tissue.
For example, the to-be-detected image may be a fluorescence image, in which case the target region corresponding to the diseased tissue is determined from the fluorescence image, and the target region is the region corresponding to the diseased tissue in the fluorescence image. Alternatively, the to-be-detected image may be the fused image of the visible light image and the fluorescence image, in which case the fused image is generated based on the visible light image and the fluorescence image, and the target region corresponding to the diseased tissue is determined from the fused image, and the target region is the region corresponding to the diseased tissue in the fused image.
When generating the fused image based on the visible light image and the fluorescence image, the following methods may be included but not limited to performing contrast enhancement of the fluorescence image to enhance the contrast of the fluorescence image to obtain a contrast-enhanced fluorescence image. The contrast enhancement method may include but is not limited to histogram equalization, local contrast enhancement, etc., and the contrast enhancement method is not limited. Then, color mapping is performed on the contrast-enhanced fluorescence image, i.e., the fluorescence image is dyed to obtain a color-mapped fluorescence image. When the fluorescence image is dyed, different fluorescent brightness in the fluorescence image may correspond to different hues and saturations, and the color mapping method is not limited. Then, the color-mapped fluorescence image and the visible light image are fused to obtain the fused image (i.e., the dyed visible light image). For example, the color-mapped fluorescence image may be superimposed on the visible light image for display, thereby obtaining the fused image, i.e., the visible light image is dyed based on the fluorescence image, and then achieving the effect of displaying the development region of the fluorescence image on the visible light image.
The above method is only an example of generating a fused image, and the generating method is not limited.
In the embodiments, the fluorescence image may be used as the to-be-detected image, and the target region corresponding to the diseased tissue is determined from the to-be-detected image; or, the fused image may be generated based on the visible light image and the fluorescence image, and the fused image may be used as the to-be-detected image, and the target region corresponding to the diseased tissue is determined from the to-be-detected image.
In a possible implementation, after the to-be-detected image is obtained, the following steps may be used to determine the target region corresponding to the diseased tissue from the to-be-detected image. The following steps are only examples, and the determining method is not limited, as long as the target region corresponding to the diseased tissue can be obtained.
At step 40211: based on a pixel value corresponding to each pixel point in the to-be-detected image respectively, target pixel points corresponding to the diseased tissue are selected from all pixel points in the to-be-detected image.
For example, for the to-be-detected image, based on a feature of the to-be-detected image, it may be known that the brightness of the development region of the to-be-detected image (e.g., the development region of the fluorescence image or the fused image) is relatively large, and the brightness of the non-development region of the to-be-detected image is relatively small, therefore, based on a pixel value corresponding to each pixel in the fluorescence image respectively, the target pixels corresponding to the diseased tissue may be selected. For example, in the positive development mode. the diseased tissue corresponds to the development region, and the normal tissue corresponds to the non-development region, i.e., the brightness value corresponding to the diseased tissue is relatively large, and the brightness value corresponding to the normal tissue is relatively small. Therefore, the pixel points with large brightness values may be used as the target pixel points. In the negative development mode, the diseased tissue corresponds to the non-development region, and the normal tissue corresponds to the development region, i.e., the brightness value corresponding to the diseased tissue is relatively small, and the brightness value corresponding to the normal tissue is relatively large. Therefore, the pixel points with small brightness values may be used as the target pixel points. Based on the above principles, in order to select the target pixel points corresponding to the diseased tissue, the following methods may be used in the embodiments as described below.
Method 1, if the fluorescence image is a fluorescence image in a positive development mode, i.e., a fluorescence image acquired in the positive development mode, where in the positive development mode, the development region of the fluorescence image corresponds to the diseased tissue, then, for each pixel point in the to-be-detected image (i.e., the fluorescence image or the fused image), if the pixel value (e.g., brightness value) corresponding to the pixel point is larger than the first threshold, determining that the pixel point is a target pixel point; if the pixel value corresponding to the pixel point is smaller than or equal to the first threshold, determining that the pixel point is not a target pixel point.
After the above processing is performed on all pixels in the to-be-detected image, target pixels may be selected from all pixels in the to-be-detected image, and there may be a plurality of target pixels. In summary, it may be seen that the target pixel points are the pixel points in the to-be-detected image whose pixel value is greater than the first threshold.
For example, a target pixel in the to-be-detected image may be determined by using a binarization method, and binarization is to set the pixel value of the pixel on the image to 0 or 255. For example, the value range of the pixel value is 0-255, and an appropriate binarization threshold may be set. If the pixel value of the pixel is larger than the binarization threshold, the pixel value of the pixel is set to 255. If the pixel value of the pixel is smaller than or equal to the binarization threshold, the pixel value of the pixel is set to 0. Therefore, a binarized image is obtained, and the pixel values of pixels in the binarized image are 0 or 255.
Based on the above principle, a threshold may be pre-configured as the first threshold. The first threshold may be configured based on experiences and is not limited. Based on the first threshold, the boundary between the development region and the non-development region may be represented, and the development region and the non-development region in the to-be-detected image may be distinguished based on the first threshold. Based on this, since the brightness value corresponding to the diseased tissue is relatively large and the brightness value corresponding to the normal tissue is relatively small, the pixel point with a large brightness value may be used as a target pixel point. Therefore, for each pixel point in the to-be-detected image, if the pixel value corresponding to the pixel point is larger than the first threshold, it may be determined that the pixel point is the target pixel point. and if the pixel value corresponding to the pixel point is smaller than or equal to the first threshold, it may be determined that the pixel point is not a target pixel point.
In the above embodiment, for each pixel point in the to-be-detected image, the pixel value corresponding to the pixel point may be the pixel value of the pixel point itself, or the local pixel mean value corresponding to the pixel point. For example, a sub-block centered on the pixel point is obtained, and the sub-block includes M number of pixel points, where M is a positive integer, and an average value of pixel values of the M number of pixel points is used as the local pixel mean value corresponding to the pixel point.
Method 2, if the fluorescence image is a fluorescence image in a negative development mode, i.e., a fluorescence image acquired in the negative development mode, where in the negative development mode, the non-development region of the fluorescence image corresponds to the diseased tissue, then, for each pixel point in the to-be-detected image (i.e., the fluorescence image or the fused image), if the pixel value (e.g., brightness value) corresponding to the pixel point is smaller than the second threshold, determining that the pixel point is a target pixel point; if the pixel value corresponding to the pixel point is larger than or equal to the second threshold, determining that the pixel point is not a target pixel point. After the above processing is performed on all pixels in the to-be-detected image, target pixels may be selected from all pixels in the to-be-detected image, i.e., the pixel points whose pixel value in the to-be-detected image is smaller than the second threshold.
For example, a threshold may be pre-configured as the second threshold. The second threshold may be configured based on experiences and is not limited. Based on the second threshold, the boundary between the development region and the non-development region may be represented, and the development region and the non-development region in the to-be-detected image may be distinguished based on the second threshold. Based on this, since the brightness value corresponding to the diseased tissue is relatively small and the brightness value corresponding to the normal tissue is relatively large, the pixel point with a small brightness value may be used as a target pixel point. Therefore, for each pixel point in the to-be-detected image, if the pixel value corresponding to the pixel point is smaller than the second threshold, it may be determined that the pixel point is the target pixel point, and if the pixel value corresponding to the pixel point is larger than or equal to the second threshold, it may be determined that the pixel point is not a target pixel point.
In the above embodiment, for each pixel point in the to-be-detected image, the pixel value corresponding to the pixel point may be the pixel value of the pixel point itself, or may be the local pixel mean value corresponding to the pixel point.
Method 3, the to-be-detected image is input to a trained target segmentation model, such that the target segmentation model determines, based on a pixel value corresponding to each pixel point in the to-be-detected image respectively, a predicted label value corresponding to each pixel point in the to-be-detected image respectively, where the predicted label value corresponding to each pixel point in the to-be-detected image is the first value or the second value, the first value is configured to indicate that the pixel point is a pixel point corresponding to the diseased tissue, and the second value is configured to indicate that the pixel point is not a pixel point corresponding to the diseased tissue. The pixel point with the predicted label value of the first value is determined as the target pixel point.
For example, in method 3, a machine learning algorithm may be used to train a target segmentation model, and the target segmentation model may be used to implement segmentation of the focus region (i.e., the target region corresponding to the diseased tissue), i.e., to distinguish the target pixel points corresponding to the diseased tissue from all the pixel points in the to-be-detected image.
In a possible implementation, the training process and the testing process may include the following steps as described below.
At step S11, an initial segmentation model is obtained, which may be a machine learning model, such as a deep learning model or a neural network model, etc., and the type of the initial segmentation model is not limited. The initial segmentation model may be a classification model, which is configured to output a predicted label value corresponding to each pixel in the image respectively, where the predicted label value is the first value or the second value, i.e., the output of the initial segmentation model corresponds to two categories, the first value is configured to indicate that the pixel point corresponds to the diseased tissue, and the second value is configured to indicate that the pixel point does not correspond to the diseased tissue, and the structure of the initial segmentation model is not limited, as long as the initial segmentation model may implement the above functions.
For example, the network structure of the initial segmentation model may refer to
At step S12, a sample image and calibration information corresponding to the sample image are obtained, the calibration information includes a calibration label value corresponding to each pixel in the sample image respectively. If the pixel point is a pixel point corresponding to diseased tissue, a calibration label value corresponding to the pixel point is the first value (e.g., 1, etc.) to indicate that the pixel point corresponds to the diseased tissue. If the pixel point is not a pixel point corresponding to diseased tissue, a calibration label value corresponding to the pixel point is the second value (e.g., 0, etc.) to indicate that the pixel point does not correspond to the diseased tissue.
A large number of sample images may be acquired, each sample image is a fluorescence image or a fusion image inside the target object, and the sample image may also be referred to as a training image. For each sample image, the user may manually calibrate the calibration information corresponding to the sample image. For example, the focus region (i.e., the region corresponding to the diseased tissue) needs to be marked (i.e., the contour of the focus region is marked), a calibration mask diagram may be output after marking, and the calibration mask diagram is the calibration information corresponding to the sample image, in the calibration mask diagram, the values of the outline of the focus region are all the first value, and the values inside the contour are all the first value, indicating that the contour and the inside of the contour are all pixel points corresponding to the diseased tissue. The values outside the contour are all the second value, indicating that the outside of the contour is not the pixel points corresponding to the diseased tissue.
Referring to
In summary, for each sample image, the sample image and the calibration information corresponding to the sample image may be obtained, and the calibration information corresponding to the sample image may be a calibration mask diagram.
At step S13, the sample image is input to an initial segmentation model, such that the initial segmentation model determines, based on a pixel value corresponding to each pixel point in the sample image respectively, a predicted label value and a predicted probability corresponding to the predicted label value that correspond to each pixel point respectively, where the predicted label value corresponding to the pixel point in the sample image may be the first value or the second value.
For example, since the brightness of the development region of the sample image (e.g., the fluorescence image) is relatively large, and the brightness of the non-development region is relatively small, the working principle of the initial segmentation model is to determine the segmentation result of the sample image based on a pixel value corresponding to each pixel point in the sample image respectively, and the working process of the initial segmentation model is not limited, as long as the initial segmentation model can output the segmentation result. In summary, after the sample image is input to the initial segmentation model, the initial segmentation model may determine, based on a pixel value corresponding to each pixel point in the sample image respectively, a predicted label value and a predicted probability corresponding to the predicted label value that correspond to each pixel point respectively, e.g., the first pixel point corresponds to the first value and the predicted probability of the pixel point corresponding to the first value is 0.8, the second pixel point corresponds to the first value and the predicted probability of the pixel point corresponding to the first value is 0.6, the third pixel point corresponds to the second value and the predicted probability of the pixel point corresponding to the second value is 0.8, and so on.
At step S14, a target loss value is determined based on the calibration label value and the predicted label value corresponding to each pixel point. For example, for each pixel point, the target loss value is determined based on the calibration label value corresponding to the pixel point, the predicted label value corresponding to the pixel point and the predicted probability corresponding to the predicted label value.
For example, assuming that the loss function is a cross entropy loss function, the loss value may be determined by using the following formula. The cross entropy loss function is only an example, and the type of the loss function is not limited.
In the above formula, for each pixel point. loss represents the loss value corresponding to the pixel point, M represents the number of categories, which is 2 in this embodiment, i.e., there are two categories in total, category 1 corresponds to the first value, and category 2 corresponds to the second value. yc is the value corresponding to the calibration label value, which may be equal to 0 or 1. If the calibration label value corresponding to the pixel point is the first value, y1 is 1, and y2 is 0. If the calibration label value corresponding to the pixel point is the second value, y1 is 0, and y2 is 1. pc is the predicted probability corresponding to the predicted label value. If the probability of the first value corresponding to the pixel point is p_1, the probability of the second value corresponding to the pixel point is 1-p_1.
In summary, for each pixel in the sample image, the above formula may be used to calculate the loss value corresponding to the pixel point, and then the target loss value may be determined based on the loss values corresponding to all pixel points in the sample image. For example, the target loss value may be determined based on the arithmetic average of the sum of the loss values corresponding to all pixel points.
At step S15, the initial segmentation model is trained based on the target loss value to obtain a target segmentation model.
For example, the network parameters of the initial segmentation model are adjusted based on the target loss value to obtain an adjusted segmentation model, and the goal of adjusting the network parameters is to make the target loss value smaller and smaller. After obtaining the adjusted segmentation model, the adjusted segmentation model is used as the initial segmentation model, and steps S13-S14 are re-executed until the target loss value meets an optimization goal, and the adjusted segmentation model is used as the target segmentation model.
At this point, the target segmentation model is obtained through training, and the testing process is performed based on the target segmentation model.
At step S16, during the testing process, after obtaining the to-be-detected image, the to-be-detected image may be input to the target segmentation model (the working principle of the target segmentation model refers to the working principle of the above initial segmentation model), such that the target segmentation model determines a predicted label value corresponding to each pixel point respectively, based on a pixel value corresponding to each pixel point in the to-be-detected image respectively. If the predicted label value corresponding to the pixel point is the first value, it is determined that the pixel point is the pixel point corresponding to the diseased tissue, i.e., the pixel point is a target pixel point. If the predicted label value corresponding to the pixel point is the second value, it is determined that the pixel point is not the pixel point corresponding to the diseased tissue, i.e., the pixel point is not a target pixel point. In summary, the pixel point whose predicted label value is the first value may be determined as the target pixel point, so as to find all target pixel points corresponding to the diseased tissue from the to-be-detected image.
In summary, it may be seen that based on method 1, method 2, or method 3, the target pixel points corresponding to the diseased tissue may be selected from all the pixel points of the to-be-detected image, and the selection method is not limited.
At step 40212, at least one connected component composed of all target pixel points in the to-be-detected image is acquired.
For example, after the target pixel points are determined from the to-be-detected image, a connected region detection algorithm may be used to determine the connected component composed of these target pixel points (i.e., the connected region of the target pixel points). There may be at least one connected component, and the process of determining the connected components is not limited.
The region composed of adjacent target pixel points is referred to as a connected component. The connected region detection algorithm refers to finding and marking each connected component in the to-be-detected image, that is to say, based on the connected region detection algorithm, each connected component in the to-be-detected image may be found, and the process is not limited.
At step 40213, a target region corresponding to the diseased tissue is determined from the to-be-detected image based on a connected component.
For example, there may be at least one connected component. If the number of connected domains is one, the connected component is used as the target region corresponding to the diseased tissue in the to-be-detected image. If the number of connected components is at least two, the connected component with the largest region is used as the target region corresponding to the diseased tissue in the to-be-detected image, other connected components may also be used as the target region, a plurality of adjacent connected components may also be used as the target regions corresponding to the diseased tissue in the to-be-detected image, or all the connected components may be used as the target regions corresponding to the diseased tissue in the to-be-detected image, which is not limited.
In a possible implementation, after obtaining the connected component composed of target pixel points, the connected component may also be subjected to morphological processing (e.g., using a 9*9 filter kernel to smooth the connected component to obtain a connected component with smooth edges) and interference removal processing (e.g., removing isolated points), to obtain a connected component after morphological processing and interference removal processing, such that the target region corresponding to the diseased tissue is determined from the to-be-detected image, based on the connected component after morphological processing and interference removal processing.
In summary, based on step 40211 to step 40213, the target region corresponding to the diseased tissue may be obtained, and at this point, step 4021 is completed, and the subsequent step 4022 is performed based on the target region.
At step 4022, a region contour corresponding to the diseased tissue is determined based on the target region corresponding to the diseased tissue. For example, all boundary pixel points in the target region are determined, and the region contour corresponding to the diseased tissue is determined based on all boundary pixel points in the target region, i.e., all boundary pixel points constitute the region contour.
At step 4023, the to-be-cut boundary corresponding to the diseased tissue is determined based on the region contour.
In a possible implementation, after the region contour corresponding to the diseased tissue is obtained, the region contour may be directly determined as the to-be-cut boundary, indicating that cutting needs to be performed along the region contour. The region contour may be an irregular shape or any shape, and the region contour is not limited.
In another possible implementation, after the region contour corresponding to the diseased tissue is obtained, it may also be determined whether there is a coincident boundary between the region contour and the organ contour of the organ corresponding to the diseased tissue. If there is no coincident boundary between the region contour and the organ contour, the region contour is determined as the to-be-cut boundary, indicating that cutting needs to be performed along the region contour. If there is a coincident boundary between the region contour and the organ contour, a non-coincident boundary (i.e., the remaining boundary in the region contour except the coincident boundary) between the region contour and the organ contour is determined as the to-be-cut boundary.
In this embodiment, the organ contour of the organ corresponding to the diseased tissue may be acquired first, and the acquisition method is not limited in the embodiment, as long as the organ contour of the organ corresponding to the diseased tissue can be obtained. For example, a deep learning model for recognizing an organ contour may be pre-trained, and the training process of the deep learning model is not limited. On this basis, the above visible light image may be input to the deep learning model, and the deep learning model outputs the organ contour of the organ corresponding to the diseased tissue, and the working principle of this deep learning model is not limited, as long as the deep learning model can output the organ contour.
After the region contour corresponding to the diseased tissue and the organ contour corresponding to the diseased tissue are obtained, it may be determined whether there is a coincident boundary between the region contour and the organ contour, i.e., whether there is a coincident line segment.
If there is no coincident boundary, it means that cutting needs to be performed along the region contour in order to completely cut the diseased tissue. Therefore, the region contour may be determined as the to-be-cut boundary.
If there is a coincident boundary, it means that the edge of the organ is already diseased tissue For example, the entire left side (or right side, or upper side, or lower side, etc.) of the organ corresponding to the diseased tissue is already diseased tissue, and the region contour will coincide with the left edge of the organ corresponding to the diseased tissue. In this case, the diseased tissue may be completely cut by cutting only along the non-coincident boundary. Therefore, the non-coincident boundary between the region contour and the organ contour is determined as the to-be-cut boundary. The coincident boundary between the region contour and the organ contour is the edge of the organ corresponding to the diseased tissue. When cutting along the non-coincident boundary, the region corresponding to the coincident boundary is also cut.
In summary, the to-be-cut boundary may be determined. The above are just two examples, and the determining method is not limited, as long as the diseased tissue can be completely cut based on the to-be-cut boundary.
At this point, step 402 is completed, the to-be-cut boundary is obtained, and the subsequent step 403 is performed based on the to-be-cut boundary.
At step 403, a target image is generated, where the target image includes the visible light image and the to-be-cut boundary.
In a possible implementation, after the to-be-cut boundary corresponding to the diseased tissue is determined from the to-be-detected image, the to-be-cut boundary may be superimposed on the visible light image to obtain the target image, the target image includes the visible light image and the to-be-cut boundary, and the location of the to-be-cut boundary on the visible light image is the same as the location of the to-be-cut boundary on the to-be-detected image.
After the target image is obtained, the target image may be displayed on the screen for medical staff to view. Since the to-be-cut boundary is the boundary of the focus region, medical staff may clearly see the boundary of the focus region from the target image, and the boundary of the focus region may provide a reference for cutting.
In another possible implementation, in order to more prominently display the to-be-cut boundary in the target image, generating the target image based on the visible light image and the to-be-cut boundary may include the following steps as described below.
At step 4031, a target boundary feature corresponding to the to-be-cut boundary is determined.
For example, the target boundary feature may include but is not limited to a target color and/or a target line type, and the target boundary feature is not limited, which may be any feature used to display the to-be-cut boundary.
The target color may be any color, such as blue, red, green, etc., and the target color is not limited, as long as the target color is different from the color of the target image itself and can highlight the to-be-cut boundary.
The target line type may be any line type, such as a solid line type and a dashed line type, etc. and the target line type is not limited, as long as the target line type may highlight the to-be-cut boundary.
At step 4032, a target cut boundary is generated based on the to-be-cut boundary and the target boundary feature.
For example, if the target boundary feature is a target color, performing color adjustment on the to-be-cut boundary, to obtain the target cut boundary, and the color of the target cut boundary is the target color. For example, if the target color is blue, the color of the to-be-cut boundary is adjusted to blue, to obtain a blue target cut boundary.
If the target boundary feature is a target line type, performing line type adjustment on the to-be-cut boundary, to obtain the target cut boundary, and the line type of the target cut boundary is the target line type. For example, if the target line type is a dashed line type, the line type of the to-be-cut boundary is adjusted to a dashed line type, to obtain a dashed target cut boundary.
If the target boundary feature is a target color and a target line type, performing color adjustment and line type adjustment on the to-be-cut boundary, to obtain the target cut boundary, and the color of the target cut boundary is the target color, and the line type of the target cut boundary is the target line type. For example, if the target color is blue and the target line type is a dashed line type, the color of the to-be-cut boundary is adjusted to blue and the line type of the to-be-cut boundary is adjusted to the dashed line type, to obtain a target cut boundary with the blue color and the dashed line type.
The above are just a few examples of generating target cut boundaries, which is not limited.
At step 4033, the to-be-cut boundary is superimposed on the visible light image to obtain the target image.
For example, the target cut boundary may be superimposed on the visible light image to obtain the target image, which includes the visible light image and the target cut boundary. After the target image is obtained, the target image may be displayed on the screen for the medical staff to view. Since the target cut boundary is the boundary of the focus region, and the target cut boundary is a highlighted boundary, e.g., the target cut boundary is highlighted as a boundary with a target color and/or a target line type, the medical staff may clearly see the boundary of the focus region from the target image.
In a possible implementation, in order to more prominently display the to-be-cut boundary and the information around the to-be-cut boundary in the target image, after the target image is generated based on the visible light image and the to-be-cut boundary, if the fluorescence image is a fluorescence image in a positive development mode, and the fluorescence image includes a development region outside the to-be-cut boundary, the development region outside the to-be-cut boundary is superimposed on the target image, i.e., the low-intensity fluorescent development region outside the to-be-cut boundary is retained on the target image. If the fluorescence image is a fluorescence image in a negative development mode, and the fluorescence image includes a development region inside the to-be-cut boundary, the development region inside the to-be-cut boundary is superimposed on the target image, i.e., the low-intensity fluorescent development region inside the to-be-cut boundary is retained on the target image.
For example, if the fluorescence image is a fluorescence image in a positive development mode, the development region of the fluorescence image corresponds to the diseased tissue, and the to-be-cut boundary is the boundary of the target region corresponding to the diseased tissue. The inside of the to-be-cut boundary is the development region. After the to-be-cut boundary is superimposed on the visible light image to obtain the target image, the development region inside the to-be-cut boundary is not superimposed on the target image. For the fluorescence image, there may also be a development region outside the to-be-cut boundary, but an intensity of the development region outside the to-be-cut boundary is lower than an intensity of the development region inside the to-be-cut boundary. After the to-be-cut boundary is superimposed on the visible light image to obtain the target image, the development region outside the to-be-cut boundary may not be superimposed on the target image, or the development region outside the to-be-cut boundary may be superimposed on the target image, i.e., the development region is expanded from the to-be-cut boundary to the outer periphery, and the low-intensity fluorescence development region outside the to-be-cut boundary in the fluorescence image is retained in the target image.
If the fluorescence image is a fluorescence image in a negative development mode, the development region of the fluorescence image corresponds to normal tissue, the non-development region corresponds to the diseased tissue, and the to-be-cut boundary is the boundary of the diseased tissue corresponding to the target region. The outside of the to-be-cut boundary is the development region. After the to-be-cut boundary is superimposed on the visible light image to obtain the target image, the development region outside the to-be-cut boundary is not superimposed on the target image. For the fluorescence image, there may also be a development region inside the to-be-cut boundary, but the intensity of the development region inside the to-be-cut boundary is lower than the intensity of the development region outside the to-be-cut boundary. After the to-be-cut boundary is superimposed on the visible light image to obtain the target image, the development region inside the to-be-cut boundary may not be superimposed on the target image, or the development region inside the to-be-cut boundary may be superimposed on the target image, i.e., the development region is expanded from the to-be-cut boundary to the inner periphery, and the low-intensity fluorescence development region inside the to-be-cut boundary in the fluorescence image is retained in the target image.
The above process is described below in combination with
For the five images on the lower side of
The target image display methods in
In a possible implementation, regarding the triggering timing of step 401 to step 403, if a display switching command for the fluorescence image is received, and the display switching command is configured to indicate the display of the to-be-cut boundary, then step 401 to step 403 may be executed, that is to say, the to-be-cut boundary corresponding to the diseased tissue is determined from the to-be-detected image, and the to-be-cut boundary is superimposed on the visible light image to obtain the target image, and the target image is displayed. If the display switching command for the fluorescence image is not received, a fused image may be generated based on the visible light image and the fluorescence image, and the fused image may be displayed, and the embodiments will not repeat the process.
When the operation on the target object enters the resection stage, the medical staff may issue a display switching command for the fluorescence image, such that the display switching command for the fluorescence image may be received, and then the image processing method of the embodiments is adopted to superimpose the to-be-cut boundary on the visible light image.
It can be seen from the above technical solution that in embodiments of the present disclosure, the to-be-cut boundary corresponding to the diseased tissue (i.e., the boundary of the focus region) may be determined, and a target image is generated based on the visible light image and the to-be-cut boundary, i.e., the to-be-cut boundary is superimposed on the visible light image for display. Since the to-be-cut boundary is superimposed on the visible light image for display instead of superimposing the fluorescent image on the visible light image for display, the problem of most parts of the visible light images being covered is avoided, the situation of visible light images being covered by fluorescent development is improved, and visual obstructions are avoided or alleviated. The effect of the target image is better, which can clearly display the normal tissue and diseased tissue at the designated location inside the target object, improving the quality and efficiency of the doctor's operation and providing a better user experience. Since the to-be-cut boundary is displayed on the target image, the doctor may know the to-be-cut boundary corresponding to the diseased tissue, and may cut according to the to-be-cut boundary, improving the quality and efficiency of the doctor's operation.
The image processing method of the embodiments of the present disclosure is described below in combination with specific application scenarios.
As illustrated in
In a feasible implementation, as illustrated in
In
For the implementation process of contour detection by using a region segmentation method, step 402 can be referred, and method 1 and method 2 in step 402 are region segmentation methods, which will not be repeated here.
In another feasible implementation, contour detection may be performed by using a machine learning method. For example, a target segmentation model is first obtained through training. For endoscopic images (such as fluorescence images or fused images), the endoscopic images may be input to the target segmentation model, and the target segmentation model outputs the region contour of the focus region. The implementation process of contour detection by using a machine learning method may refer to step 402, and method 3 in step 402 is a machine learning method, which will not be repeated here.
Based on the same application concept as the above method, embodiments of the present disclosure provide an image processing apparatus. Referring to
For example, when determining the to-be-cut boundary corresponding to the diseased tissue from the to-be-detected image, the determination module 72 is configured to: determine the target region corresponding to the diseased tissue from the to-be-detected image, determine the region contour corresponding to the diseased tissue based on the target region, and determine the to-be-cut boundary corresponding to the diseased tissue based on the region contour.
For example, when determining the target region corresponding to the diseased tissue from the to-be-detected image, the determination module 72 is configured to: select the target pixel points corresponding to the diseased tissue from all the pixel points of the to-be-detected image based on a pixel value corresponding to each pixel point in the to-be-detected image respectively, acquire at least one connected component composed of the target pixel points in the to-be-detected image, and determine the target region from the to-be-detected image based on the at least one connected component.
For example, when selecting the target pixel points corresponding to the diseased tissue from all the pixel points of the to-be-detected image based on the pixel value corresponding to each pixel point in the to-be-detected image respectively, the determination module 72 is configured to: if the fluorescence image is a fluorescence image under a positive development mode, when a pixel value corresponding to a pixel point in the to-be-detected image is larger than a first threshold, determine that the pixel point is a target pixel point; where, in the positive development mode, the development region of the fluorescence image corresponds to the diseased tissue; or, if the fluorescence image is a fluorescence image under a negative development mode, when the pixel value corresponding to the pixel point in the to-be-detected image is smaller than a second threshold, determine that the pixel is a target pixel point; where, in the negative development mode, the non-development region of the fluorescence image corresponds to the diseased tissue.
For example, when selecting the target pixel points corresponding to the diseased tissue from all the pixel points of the to-be-detected image based on the pixel value corresponding to each pixel point in the to-be-detected image respectively, the determination module 72 is configured to: input the to-be-detected image to a trained target segmentation model, such that the target segmentation model determines, based on the pixel value corresponding to each pixel point in the to-be-detected image respectively, a predicted label value corresponding to each pixel point in the to-be-detected image respectively, where the predicted label value corresponding to each pixel point in the to-be-detected image is the first value or the second value; and the pixel point in the to-be-detected image whose predicted label value is the first value is determined as a target pixel.
For example, when training the target segmentation model, the determination module 72 is configured to: acquire a sample image and calibration information corresponding to the sample image, the calibration information including a calibration label value corresponding to each pixel point in the sample image respectively; where, if a first pixel point in the to-be-detected image is a pixel point corresponding to a diseased tissue, the calibration label value corresponding to the first pixel point is the first value, and if a second pixel point in the to-be-detected image is not a pixel point corresponding to a diseased tissue, the calibration label value corresponding to the second pixel point is the second value; input the sample image to the initial segmentation model, such that the initial segmentation model determines a predicted label value corresponding to each pixel point in the to-be-detected image respectively based on the pixel value corresponding to each pixel point in the sample image respectively; where the predicted label value corresponding to the pixel point in the to-be-detected image is the first value or the second value; determine a target loss value based on the calibration label value and the predicted label value corresponding to each pixel point in the to-be-detected image, and train the initial segmentation model based on the target loss value to obtain the target segmentation model.
For example, when determining the region contour corresponding to the diseased tissue based on the target region, the determination module 72 is configured to: determine boundary pixel points in the target region, and determine the region contour corresponding to the diseased tissue based on the boundary pixel points.
For example, when determining the to-be-cut boundary corresponding to the diseased tissue based on the region contour, the determination module 72 is configured to: determine the region contour as the to-be-cut boundary; or, if there is a coincident boundary between the region contour and an organ contour of an organ corresponding to the diseased tissue, determine the non-coincident boundary between the region contour and the organ contour as the to-be-cut boundary.
For example, when generating the target image. the generation module 73 is configured to: superimpose the to-be-cut boundary on the visible light image to obtain the target image; or, determine a target boundary feature, generate a target cut boundary based on the to-be-cut boundary and the target boundary feature, and superimpose the target cut boundary on the visible light image to obtain the target image.
For example, when generating the target cut boundary based on the to-be-cut boundary and the target boundary feature, the generation module 73 is configured to: if the target boundary feature is a target color, perform color adjustment on the to-be-cut boundary, to obtain the target cut boundary. where the color of the target cut boundary is the target color; if the target boundary feature is a target line type, perform line type adjustment on the to-be-cut boundary, to obtain the target cut boundary, where the line type of the target cut boundary is the target line type; if the target boundary feature is a target color and a target line type, perform color adjustment and line type adjustment on the to-be-cut boundary, to obtain the target cut boundary, where the color of the target cut boundary is the target color, and the line type of the target cut boundary is the target line type.
For example, after generating the target image, the generation module 73 is further configured to: if the fluorescence image is a fluorescence image in a positive development mode, and the fluorescence image includes a development region outside the to-be-cut boundary, superimpose the development region outside the to-be-cut boundary on the target image; if the fluorescence image is a fluorescence image in a negative development mode, and the fluorescence image includes a development region inside the to-be-cut boundary, superimpose the development region inside the to-be-cut boundary on the target image.
For example, when determining the to-be-cut boundary corresponding to the diseased tissue from the to-be-detected image, the determination module 72 is configured to: if a display switching command for the fluorescence image is received, the display switching command is configured to indicate displaying the to-be-cut boundary, determine the to-be-cut boundary corresponding to the diseased tissue from the to-be-detected image.
Based on the same application concept as the above method, embodiments of the present disclosure provide an image processing device (i.e., the camera system host of the above embodiment), the image processing device may include a processor and a machine-readable storage medium, where the machine-readable storage medium stores machine-executable instructions that can be executed by the processor, and the processor is configured to execute the machine-executable instructions to implement the image processing method disclosed in the examples of the present disclosure.
Based on the same application concept as the above method, embodiments of the present disclosure provide a machine-readable storage medium storing a plurality of computer instructions, and when the plurality of computer instructions are executed by one or more processors, the above image processing method disclosed in the examples of the present disclosure can be implemented.
The machine-readable storage medium may be any electronic, magnetic, optical, or other physical storage device, and may contain or store information, such as executable instructions, data, and so on. For example, the machine readable storage medium may be a Radom Access Memory (RAM), a volatile memory, a non-volatile memory, a flash memory, a storage drive (e.g. hard disk drive), a solid state harddisk, any type of storage disk (e.g., optical disks, DVDs, etc.), or a similar storage medium, or a combination thereof.
The system, apparatus, module, or unit described in the previous embodiments may be implemented by a computer chip or entity, or may be implemented by using a product with a certain function. An example of a device is a computer, and the form of the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email sending and receiving device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.
For convenience of description, when describing the above apparatus, the functions are divided into various units and described separately. Of course, when implementing the present disclosure, the functions of the units may be implemented in the same software or multiple software and/or hardware.
A person of ordinary skill in the art should understand that the implementations of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may take the form of an entirely hardware implementation, an entirely software implementation, or an implementation combining both software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk memory, CD-ROM, optical memory, etc.) containing computer-usable program code.
The embodiments of this application are described with reference to flowcharts and/or block diagrams of the method, the terminal device (the system), and the computer program product in the embodiments of this application. It should be understood that, each process and/or block in the flowcharts and/or block diagrams as well as combinations of processes and/or blocks in the flowcharts and/or block diagrams may be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special purpose computer, an embedded processor or other programmable data processing device to generate a machine, such that the instructions executed by the processor of a computer or other programmable data processing device produce a device for achieving the functions specified in a flow chart or processes or processes and/or block diagram of one box or more boxes.
These computer program instructions may also be stored in computer-readable memory capable of booting a computer or other programmable data processing device to work in a particular manner, such that the instructions stored in the computer-readable memory produce manufactured products including an instruction device, the instruction device implements the function specified in a flow chart one or more processes and/or block diagram one box or more boxes.
These computer program instructions may also be stored in computer-readable memory capable of booting a computer or other programmable data processing device to work in a particular manner, such that the instructions stored in the computer-readable memory produce manufactured products including an instruction device, the instruction device implements the function specified in a flow chart one or more processes and/or block diagram one box or more boxes.
The above are implementations of the present disclosure and are not intended to limit the present disclosure. For those skilled in the art, this disclosure may have various modifications and changes. Any modification, equivalent replacement, and improvement made within the spirit and principle of this disclosure shall be included in the scope of claims of this disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202111501127.8 | Dec 2021 | CN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/CN2022/115375 | 8/29/2022 | WO |
