The present disclosure relates to medical analysis using machine learning models, and more particularly, to analyzing oocytes using sensor information and machine learning models.
In recent years, there have been advancements in treatments for infertility. An example treatment includes in-vitro-fertilization (“IVF”). An IVF starts with an ovarian stimulation phase which stimulates egg production. Eggs (oocytes) may be retrieved from the patient and fertilized in-vitro to form embryos. Multiple tests and analysis may be performed on the embryos in an effort to select the most viable and/or advantageous embryo for implantation. However, such tests and analyses suffer from technological problems and thus an accurate scheme to effectuate this selection presents technological challenges.
As an example, different grading systems have been developed to assist in determining the viability of each embryo. These grading systems generally involve manual annotation of embryo image or time-lapse video. As may be appreciated, the selection process proves to be error-prone due to, for example, the subjective judgments of embryologists. Additionally, there is currently no standard grading system that is universally adopted to select quality oocytes. Current automated techniques which leverage analyses of embryos are inaccurate and fail to utilize disparate sensor information.
This specification describes techniques for selecting good quality eggs (e.g., oocytes) which have, for example, an enhanced probability of blastocyst formation. As will be described, a system may leverage disparate sensor information to analyze oocytes using machine learning techniques. The sensor information may include, for example, images of the oocytes (unfertilized eggs) undergoing deformation due to application of pressure along with pressure measurements indicative of the pressure applied to the oocytes. This sensor information may enable an understanding of the morphological and mechanical characteristics of the oocytes as the oocytes undergo deformation, which allows for a more accurate assessment of the quality of each oocyte. In contrast, prior techniques relied upon hand-tuned models or grading systems which used a single type of information (e.g., images) and which were prone to error. As will be described, the disclosed technology leverages the disparate sensor information, along with specifically trained models, to more accurately and efficiently assess qualities or viability of eggs at an earlier stage.
To evaluate the viability of eggs, some prior techniques rely upon an embryologist to visually assess embryos (e.g., fertilized eggs). Some clinics record images of the embryos, and an embryologist may score an embryo based on various grading systems and their visual assessment. One major challenge in embryo selection is the high level of labor, subjectivity and variability that exists between embryologists of different skill levels and grading systems of different performances. Specifically, embryologists often disagree with each other or even with themselves on which embryo has the best viability for transfer after assessing embryos visually for considerable amount of time. Further, it remains unclear which embryonic features associated with a particular grading system are ultimately predictive on the success rate of each embryo.
Other prior techniques include automated techniques for selecting quality eggs. These techniques may rely on particular characteristics of fertilized eggs. Typically, the particular characteristics of fertilized eggs are obtained by analyzing (e.g., using microscopes and computer vision technology) video capturing changes in eggs during growth and development. However, selecting eggs based on these characteristics (e.g., characteristics derived based on observing the growth and development of eggs) may not yield satisfactory results.
In contrast, the disclosed technology allows for analyzing the viability of eggs at an earlier stage (e.g., when eggs are still unfertilized rather than when eggs are fertilized). Thus, the disclosed technology can eliminate the extra complexity associated with fertilizing eggs which may not be implanted later. By utilizing machine learning models for selecting oocytes based on extracted features of the oocytes, the disclosed technologies can obtain more objective and quantitative analysis on the viability of the oocytes.
Furthermore, the disclosed technology leverages machine learning techniques to analyze both morphological and mechanical features of oocytes. The morphological features, as will be described, can include geometrical information associated with an oocyte such as the size or length of zona pellucida, cytoplasm, polar body, perivitelline space, extents to which the oocyte is aspirated into a pressure-applying tool (e.g., aspiration depth), and so on. The mechanical features, as will be described, can include parameters determined or derived based on deformation characteristics of an oocyte. For example, mechanical features can include at least morpho-kinetic parameters described in a particular model (e.g., Zener model) or the like.
These machine learning techniques may utilize the features to indicate one or more metrics indicative of quality of an oocyte. An example metric may include a value indicative of a likelihood or probability of blastocyst formation. Additional example metrics may include an indication of the oocyte being “good”, the likelihood or probability of aneuploidy and implantation rate, and so on.
More specifically, the system may obtain an image sequence which depicts an oocyte being deformed due to a mechanical stimulus. An example stimulus may include the oocyte being aspirated into a portion of a pressure tool (e.g., a pipette) which applies pressure to the oocyte. The system may use example computer vision techniques to process the image sequence to derive the above-described morphological features and mechanical features. In some examples, to deal with different resolutions of hardware (e.g., microscope cameras) which are used to capture images, a normalization technique may be utilized. The normalization technique may be integrated as a part of image processing techniques performed on captured image sequences associated with oocytes. The normalization technique will be described in greater detail below.
The extracted morphological features and mechanical features of the oocyte may then be used to train, or run inferences using, a machine learning model. The training process may include using a subset of the morphological and mechanical features to train the machine learning model. The trained machine learning model may then be employed to generate one or more metrics indicative of the quality of the oocyte, where a particular metric may be indicative of blastocyst formation from the oocyte.
Additionally, and optionally, the metrics may be presented to a user or a professional (e.g., an embryologist) through an interactive user interface. As such, further analysis or evaluation on the viability of the oocyte may be more efficiently conducted. Thus, more objective, automated and time-efficient evaluation of the viability of oocytes can be achieved based on embodiments of the present disclosure.
As an example, accuracy associated with the disclosed technology, a total of 185 oocytes were evaluated to assess their viability of blastocyst formation. The 185 samples were split by 80% for training a machine learning model and 20% for testing the machine learning model. The machine learning utilized for viability evaluation was a Support Vector Machine (SVM). The machine learning model was trained to predict whether there will be blastocyst or no blastocyst for a particular oocyte in the samples. The predictive outcomes were statistically analyzed and showed an accuracy of 73%, a sensitivity of 85%, a specificity of 59%, a positive predictive value (PPV) of 77%, and a negative predictive value (NPV) of 71%. Compared with statistics compiled based on prediction made by an embryologist that show an accuracy of 45%, a sensitivity of 59%, a specificity of 33%, a PPV of 43%, and a NPV of 49%, the systems and methods according to the present disclosure achieve a better performance statistically.
The foregoing aspects and many of the attendant advantages of this disclosure will become more readily appreciated as the same become better understood by reference to the following description, when taken in conjunction with the accompanying drawings.
Example Block Diagrams
The image sequence 102 may include multiple images (e.g., images 104A-104N), for which each image depicts an oocyte 110 along with a pressure tool 112. More specifically, images 104A-104N may form a video depicting a process of aspirating the oocyte 110 into the pressure tool 112. Image 104A can represent, for example, the first frame of the video and image 104N can represent, for example, the last frame of the video. Image 104A depicts the oocyte 110 as not yet being aspirated into the pressure tool 112 while image 104N depicts at least part of the oocyte 110 being aspirated into the pressure tool 112. In some examples, the image sequence 102 may have a frame rate of 10 Hz, 20 Hz, 70 Hz, 3000 Hz, with a total video length of 1 second, 2 seconds, 10 seconds, and so on.
Additionally, in some embodiments, the pressure tool 112 is a pipette that has a diameter between particular thresholds (e.g., between 10 micrometer (μm), 20 μm, 40 μm, and 60 μm, 70 μm, 100 μm, and so on). The pipette may apply a negative pressure (e.g., a lower pressure inside the pipette relative to the pressure outside the pipette) on the oocyte 110 for aspirating the oocyte 110 into the pipette without damaging the oocyte 110. An example pressure may include between −0.01 psi to −0.5 psi. For example, the pipette may abut or otherwise be in contact with the oocyte. Although the image sequence 102 illustrates the oocyte 110 being aspirated into the pressure tool 112, in some embodiments the pressure tool 112 may apply other forms of mechanical stimulus (e.g., positive pressure). In this way, different morphological responses of the oocyte 110 may be obtained. The morphological responses can be used by the feature pre-processing engine 120 to analyze and/or extract different morphological features.
In
With respect to pressure values 106, the pressure values 106 can include a plurality of numerical values that indicate how much force is applied on the oocyte 110 during the timeframe or time period of the images 104A-104N. For example, the pressure values 106 may indicate that a first pressure (e.g., −0.3 psi) is applied on the oocyte 110 at the moment (e.g. time or timestamp) image 104A was captured and a Nth pressure is applied on the oocyte 110 at the moment the image 104N was captured. The pressure values applied on the oocyte 110 at different images in the image sequence 102 may be the same or may not be the same.
In some examples, the pressures applied on the oocyte 110 may be increasing as time progresses while in other examples the pressures applied may be decreasing as time progresses. Additionally, the pressure values 106 may include forces applied on the oocyte 110, where the forces can be calculated based on pressures generated by the pressure tool 112 and applied on the oocyte 110. The forces applied on the oocyte 110 can be used to derive mechanical features of the oocyte 110, which will be described in greater detail later with respect to
The clinical information 108 may include age and body mass index (BMI) of the patient from whom the oocyte 110 originates. Additionally, the clinical information 108 may include information indicating whether the oocyte 110 has gone through cryopreservation (CP). The clinical information 108 may also indicate available number of mature oocyte (MII) associated with the patient.
Based on at least some of the image sequence 102, pressure values 106 and clinical information 108, the feature pre-processing engine 120 may extract the features 122 associated with the oocyte 110. The features 122 can include morphological features (sizes or lengths of zona pellucida, cytoplasm, polar body, or perivitelline space) and mechanical features (e.g., elasticity and/or viscosity) of the oocyte 110. Certain features may be generated per image in the image sequence 102, while other features may be determined based on all, or a subset of, the images. These features 122 will be described in more detail below, with respect to
Based on the features 122, the machine learning model 130 may generate oocyte quality information 132 for the oocyte 110. Example information 132 may include the likelihood of blastocyst formation. In some embodiments, the machine learning model 130 may be a support vector machine (SVM) which is trained to output the information 132. In other examples, the machine learning model 130 may be a deep learning model. For example, the deep learning model may include a recurrent neural network (RNN) which is trained to output the information 132. In this example, the RNN may be input the features 122 as a sequence and may output the information 132 for the sequence. The model may also be a convolutional neural network or fully-connected network. The machine learning model 130 may utilize all or a subset of the features 122 to determine the oocyte quality information 132. For example, the SVM may be trained to utilize a subset of the features 122.
In some embodiments, the oocyte quality information 132 may further indicate the likelihood of whether a “usable” blastocyst will be formed by the oocyte 110, where “usable” means the blastocyst formed by the oocyte 110 can be suitable for transferring or for implantation. Additionally, or alternatively, the oocyte quality information 132 may indicate the likelihood of whether the blastocyst to be formed by the oocyte 110 will be “unusable” blastocyst, where “unusable” means the blastocyst formed by the oocyte 110 will be in such poor condition that the blastocyst is not suitable for furthering an IVF treatment.
Block Diagram—Morphological Feature Generation
The determination of some of the example morphological features 210 of the oocyte 110 can be illustrated by the oocyte measurements 202N, which depicts example measurements (e.g., bounding box A, distance B, distance C, distance D, distance E, bounding box F, and distance G). The oocyte measurements 202N corresponds to the measurement of the image 104N, and the oocyte measurements 202A corresponds to the measurement of the image 104A.
With respect to the illustrated oocyte measurements 202N, bounding box A may represent a region of interest (ROI) which is cropped for further image processing performed on the image sequence 102. Distance B may indicate a length and/or size of an upper side of the zona pellucida of the oocyte 110. Distance C may indicate a length and/or size of a right side of the zona pellucida of the oocyte 110. Distance D may indicate a length and/or size of a lower side of the zona pellucida of the oocyte 110. Distance E may indicate the inner zona pellucida of the oocyte 110. Distance G may indicate a length of the inner diameter of the pressure tool 112. Bounding box F may represent a region of interest (ROI) which may be cropped for calculating the aspiration depth of the oocyte 110 into the pressure tool 112. Calculating aspiration depth is described in more detail below.
Different image processing and object identification techniques may be utilized to obtain the example morphological features 210. In some examples, computer vision techniques can be used to derive the example morphological features 210. For example, edge detection techniques may be used to identify boundaries associated with the different portions of the oocyte. In other examples, image segmentation models based on deep learning algorithms can be utilized to derive the example morphological features 210. For example, segmentation masks may be generated for the features 210. As may be appreciated, a segmentation mask may assign a color or pixel value indicative of whether a pixel forms part of a classified feature (e.g., zona pellucida and so on). The derivation of each of the example morphological features are discussed with greater detail below.
Bounding box A can define the ROI that will be cropped for any image (e.g., image 104A or image 104N) of the image sequence 102, where some of the morphological features 210 such as the inner boundary of the zona pellucida, the size of the pressure tool 112 and the aspiration depth of the oocyte 110 can be calculated within this ROI. In some examples, bounding box A can be identified according to the following example technique. For example, a tip of the pressure tool 112 (e.g., a pipette) which contacts the oocyte 110 can be identified and/or positioned based on a pipette tip image that may be obtained in advance. Based on the position of the tip of pipette, the bounding box A can be drawn to make out the ROI around the tip of pipette. Although bounding box A is shown as a rectangular, other shapes can be used to define the ROI.
Distance B, distance C and distance D define the length or thickness of the zona pellucida on the upper side, right side and lower side, respectively. By averaging the lengths of these distances, a morphological feature which is indicative of an average size of the zona pellucida of the oocyte 110 may be obtained. Additionally, distance E may be defined based on the position of the tip of pipette that is obtained above when demarcating bounding box A and the inner boundary of the zona pellucida of the oocyte 110.
As an example, distance E can be useful for deriving a horizontal (e.g., along the direction of arrow C and arrow E) length of the cytoplasm of the oocyte 110. More specifically, the horizontal length of the cytoplasm of the oocyte 110 can be derived by subtracting the combined length of arrow E and arrow C from the horizontal length (e.g., defined by the right tip of arrow C and left tip of arrow E) of the oocyte 110. The horizontal length of the cytoplasm of the oocyte 110 can also be utilized as a part of the example morphological features 210. Similarly, the vertical (e.g., along the direction of arrow B and arrow D) length of the cytoplasm of the oocyte 110 can be derived by subtracting the combined length of arrow B and arrow D from the vertical length (e.g., defined by the top end of arrow B and bottom end of arrow D) of the oocyte 110. The vertical length of the cytoplasm of the oocyte 110 can also be used as a part of the features 122.
The bounding box F can be utilized as a ROI for calculating the aspiration depth (e.g., how much length the oocyte 110 is aspirated into the pressure tool 112 compared with the position of the oocyte 110 right before being aspirated) of the oocyte 110. The aspiration depth may be useful for deriving mechanical features associated with the oocyte 110, which is discussed in more detail below with respect to
The ROI (e.g., bounding box F) may be demarcated (e.g., identified) based on the position of the tip of the pressure tool 112 which is in contact with the oocyte 110. As illustrated in the oocyte measurements 202N, the bounding box F has a rectangular shape; however, in other examples, the bounding box F can be of different shapes (e.g., square or irregular shapes). In some embodiments, the bounding box F extends to the left-most part of an image at a particular threshold distance. That is, the distance may be set to ensure that it encompasses an end or extremity of the oocyte being aspirated into the tool 112.
Pixels which are aligned with each other vertically within the bounding box F may be summed to derive a pixel curve. More specifically, assuming the bounding box F spans across N pixels horizontally and M pixels vertically, the pixel intensities of the pixels having the same horizontal position within the bounding box F can be summed. For example, M pixels may be summed at a particular horizontal position. Thus, N may be a positive integer which depends on the resolution of the image sequence 102 and the horizontal length of the bounding box F. For example, if there are 50 horizontal positions then for each position the pixels which extend vertically within the box F may be summed to arrive at a value.
These values may then be used to generate a pixel curve which has the summed values on a first axis (e.g., the y axis) and the number of horizontal positions on a second axis (e.g., the x axis). The pixel curve may be smoothed using signal processing techniques such as sliding window moving average method.
Subsequently, the first order derivative may be determined on the smoothed pixel curve. This derivative may represent vertically summed intensities. The minimum value of the derivative of the pixel curve may then be found, which indicates the end of aspiration depth of the oocyte 110 for a particular image (e.g., image 104A).
The above-described example technique may be performed on other images in the image sequence 102 such that the end of aspiration depth can be acquired for each of the images 104A-104N. In some examples, the aspiration depth can be calculated based on the end of the aspiration depth and an inner boundary of zona pellucida of the oocyte 110. By calculating the aspiration depth for each of the images 104A-104N in the image sequence 102, the aspiration depth of the oocyte 110 over time can be plotted as a chart where a first axis shows the time, and a second axis shows the aspiration depth of the oocyte 110. In some examples, the aspiration depth can be further normalized based on the size of zona pellucida of the oocyte 110. For example, the aspiration depth can be normalized by subtracting the average of distance B, distance C and distance D from the calculated aspiration depth.
While the above describes an example of determining aspiration depth, as may be appreciated other techniques may be used and fall within the disclosure herein. For example, deep learning techniques may be used to determine a boundary or extremity of the oocyte into the tool. As another example, edge detection techniques (e.g., edge detection kernels) may be used to identify the edge of the oocyte in the tool.
Additionally, and optionally, the feature pre-processing engine 120 may not need to analyze all the images in the image sequence 102. In some examples, only a portion of the image sequence 102 is analyzed by selecting a start frame and performing analysis on the start frame and the frames after the start frame. The start frame may be the frame that captures the image of the oocyte 110 right before the oocyte 110 was to be aspirated into the pressure tool 112. For example, the system may identify the frame associated with a time stamp prior to a time or time stamp associated with application of negative pressure. As another example, the system may analyze the image frames and identify a frame prior to one which has an aspiration depth greater than a threshold (e.g., zero, a small value, and so on). Advantageously, the analysis can be more time-efficient and consuming less amount of computing power.
Additionally, and optionally, the pre-processing engine 120 can reduce (e.g., shrink) the number of frames to be analyzed by removing consecutive frames during which the oocyte 110 remains still or unmoved. For example, the pre-processing engine 120 may determine that only 0.5 seconds of the video depicted by the image sequence 102 show movement of the oocyte 110 while the remaining 1.5 seconds of the video of the image sequence 102 show the oocyte 110 remains unmoved or stays relatively static. The pre-processing engine 120 may then extract the 0.5 seconds of the video for further analysis. Advantageously, the analysis time can be reduced by shrinking the video length for analysis.
The feature pre-processing engine 120 can additionally determine the inner diameter of the pressure tool 112, which is illustrated by distance G in the oocyte measurements 202N. In some embodiments, the length of the inner diameter of the pressure tool 112 can be acquired by calculating the number of pixels that are aligned vertically along the inner diameter of the pressure tool 112. The inner diameter of the pressure tool 112 can be used as a normalization factor for increasing the interoperability of the oocyte analysis system 100 across different video capturing platforms that might have different hardware specifications (e.g., resolutions of captured images).
Normalization may additionally relate to the oocyte's z-height in an object housing the oocyte while images are being taken. That is, the oocyte may be placed in water or another liquid and vary in height (e.g., closer to the bottom of the water or the top). This adjustment in height may therefore change dimensions of the oocyte as compared to another oocyte or as compared to the same oocyte in different images in the sequence 102. For example, the oocyte may have slight changes in height during the image sequence 102.
More specifically, the aspiration depth of an oocyte over time may be normalized by a ratio between an inner diameter of the pressure tool 112 and a pixel length of a particular image sequence. Using computer vision techniques, an inner diameter of the pressure tool 112 can be identified and the length of the inner diameter can then be calculated by counting how many pixels along an-axis of the pressure tool 112 the inner diameter covers. For example, the inner diameter of the pressure tool 112 may span across 1,000 pixels and the aspiration depth can then be divided by 1,000 for normalization. Normalizing the aspiration depth of oocytes by the inner diameter of the pressure tool 112 can put the different frames of oocytes acquired under different image capturing settings on the same footing for the purpose of calculating morphological features associated with oocytes. As such, the inter-operability and reproducibility of the presently disclosed systems and methods can be enhanced.
Additional morphological features associated with the oocyte 110 can also be acquired by the feature pre-processing engine 120. Example features may include sizes and/or lengths of the polar body, cytoplasm and/or perivitelline space (e.g., PVS, the space between zona pellucida and cytoplasm) of the oocyte 110. Both image segmentation models based on deep learning algorithms and computer vision technology can be adopted to determine these additional morphological features. Advantageously, the additionally morphological features available can be potentially useful for training and/or testing the machine learning model 130 to improve the accuracy, sensitivity, specificity, NPV, and PPV of the results generated by the machine learning model 130.
Block Diagram—Mechanical Feature Generation
The pressure values 106 may include forces applied on the oocyte 110 (e.g., via the above-described pressure tool 112). Specifically, the forces applied on the oocyte 110 may be calculated based on the equation (A) below. The forces may therefore be based on the pressure values and geometry information associated with the pipette.
The feature pre-processing engine 120 may determine the features 254 using the above-described forces optionally in combination with the aspiration depths described in
In some embodiments, a linear elastic model, a modified linear elastic model, standard linear solid model can be used to obtain features which are indicative of the viscoelastic behavior of the oocyte 110. Example models may include a Zener model, a modified Zener model, and so on. For example, the following equation (B) may be used to fit the aspiration depths of the oocyte 110 over time to obtain mechanical features of the oocytes 110. For example, and as illustrated in
The parameters k0 and k1, can describe the “instant elongation” experienced by the oocyte 110 when the forces are applied on the oocyte 110. This instant elongation corresponds to, or is proportional to, 1/(k0+k1) and can be viewed as a measure of the “slack” in the elastic elements of the oocyte 110, or the amount of force that can be applied on the oocyte 110 before a marked resistance is exhibited. The parameter k1 can be viewed as a general measure of stiffness and may represent how tightly the proteins in the cytoplasm or zona pellucida are connected. The parameter η1 can be viewed as a measure of how much the zona pellucida continues to deform in response to the applied forces calculated in accordance with equation (A). Like in the linear elastic solid model, after the spring elements have fully extended, η1 is responsible for shape changes at the molecular level that keeps the oocyte 110 elongating. The parameter τ represents how fast (e.g., speed) the oocyte 110 deforms (e.g., enters into the pressure tool 112) after the initial instant elongation. η0 can be viewed as a measure of the viscosity of the cytoplasm or the fluid in the space (e.g., PVS) between the zona pellucida and the inner cell of the oocyte 110.
In addition to use of negative pressure, other non-limiting examples of causing movement or deformation on the oocyte 110 may be used and fall within the scope of the disclosure herein. For example, positive pressure on the oocyte 110 may be used to eject the oocyte 110 or otherwise deform, hold or perturbs different portions of the oocyte 110. Example portions may include the zona pellucida, cytoplasm, or portions of the oocyte 110 near its surfaces. Other forms of forces (e.g., optical pressure) can also be applied upon the oocyte 110.
In some examples, the pressures or forces applied on the oocyte 110 are suitably tuned through the pressure tool 112 to avoid unwanted effects on the oocyte 110. For example, a pressure applied on the oocyte 110 which is too high may damage the structure of the oocyte 110 and reduce the viability of the oocyte 110. In some embodiments, the pressure applied on the oocyte 110 is between −0.01 psi to −0.5 psi (or 0.01 psi to 0.5 psi if positive pressure is applied). In some aspects, the pressure applied on the oocyte 110 through the pressure tool 112 is adjusted based on how many days (e.g., 1, 2 or 3 days) has elapsed following fertilization. In some embodiments, the inner diameter of the pressure tool 112 is between 40 μm to 70 μm and the pressure applied may be adjusted based on the inner diameter of the pressure tool 112 to produce an appropriate level of force to be applied on the oocyte 110.
Example User Interfaces
With reference to
As illustrated, the user interface 300A receives oocyte quality information 132 from the system 100 described herein. The information 132 may reflect respective metrics of quality for a multitude of oocytes extracted from a patient. In the illustrated example, the user interface 300A is presenting summary information based on an analysis of 5 oocytes. The user interface 300A indicates that “oocyte 3” has the highest likelihood of blastocyst formation. This indication may be based on likelihoods of blastocyst formation as determined by a machine learning model (e.g., model 130).
The user interface 300A may respond to user input associated with viewing a detailed analysis. In some embodiments, the detailed analysis may include all, or a subset, of the features described above. The detailed analysis may include graphical representations, such as images from an image sequence, associated with oocyte 3. In some embodiments, the image sequence may be presented as a movie or animation in the user interface 300A.
In some examples, the oocyte quality information 132 can be connected with or compared to evaluation results obtained from Preimplantation Genetic Testing (PGT) or implantation conducted on the same oocyte samples. For instance, for oocytes that are indicated by the oocyte quality information 132 to be capable of blastocyst formation (e.g., “good” oocytes), PGT for aneuploidy (PGT-A) can be conducted to derive the euploidy rate of the oocytes selected by the oocyte analysis system 100. As another example, for oocytes that are indicated by the oocyte quality information 132 to be not capable of blastocyst formation (e.g., “bad” oocytes), PGT for aneuploidy (PGT-A) can also be conducted to derive the aneuploidy rate of the oocytes selected by the oocyte analysis system 100.
As another example, oocytes which are indicated by the oocyte quality information 132 to be capable of blastocyst formation (e.g., “good” oocytes, such as with likelihoods greater than a threshold) can be further evaluated after implantation. For example, this may be used to gauge the prediction capability on oocyte viability of the oocyte analysis system 100. The probability of embryo implantation can be obtained based on the “good” oocytes indicated by the oocyte analysis system 100 to evaluate the predictability of the oocyte analysis system 100. Advantageously, by connecting the different stages of evaluation on the quality of embryo, the oocyte analysis system 100 can be improved based on the evaluation results from PGT-A and implantation. For instance, a lower probability of embryo implantation may suggest that the parameters of machine learning model 130 need to be adjusted by using different subsets in the features 122 to train the machine learning model 130.
Although not illustrated in
The user interface 300B may also receive user inputs that cause the example oocyte analysis system 100 to analyze a particular oocyte to generate oocyte quality information 132 about the particular oocyte. Additionally, the user interface 300B may alert a user to check or re-calibrate position of a camera that is used to capture the image sequence 102 for analyzing quality of one or more oocytes. In some embodiments, the oocyte quality information 132 generated by the example oocyte analysis system 100 may be presented to users as described below.
Shown in the left is the analysis result of an oocyte that has a good score (e.g., 95), which may mean that an “usable” blastocyst is very likely to be formed by the oocyte. In contrast, shown in the right is the analysis result of an oocyte that has a poor score (e.g., 14), which may mean that no “usable” blastocyst is very likely to be formed by the oocyte. In the middle, the user interface 300C shows an analysis result of an oocyte that has a “normal” score (e.g., 60), which may mean that the oocyte is more likely to form a usable blastocyst than not.
After receiving an analysis result of a particular oocyte, the user interface 300C can allow users to view analysis result of other oocytes or prompt the example oocyte analysis system 100 to analyze quality of oocytes that have not been analyzed. Specifically, the user may view the analysis result of another oocyte by pressing the button “Next oocyte” or view analysis result of a previously analyzed oocyte by pressing the button “Back.”
Example Flowcharts
At block 402, the system obtains images which form an image sequence of an oocyte. As discussed above, the images may be captured by microscopic cameras and depict a sequence of events showing deformation or movements of an oocyte resulted from application of forces on the oocyte by a tool (e.g., pipette).
At block 404, the system obtains pressure values associated with the oocyte being aspirated into the tool. For example, the pressure values may include pressures or forces applied to the oocyte during the process of the oocyte 110 being aspirated into the pressure tool. In some embodiments, the pressure values may be maintained as the same throughout the aspiration process. In some embodiments, the pressure values may vary in a certain manner (e.g., lower pressure followed by higher pressure applied).
At block 406, the system determines morphological features associated with the oocyte based on the obtained images that form an image sequence. Illustratively, the morphological features associated with the oocyte include aspiration depth of the oocyte, size and/or length of the cytoplasm, size and/or length of the zona pellucida and length of diameter of the tool that is used to aspirate the oocyte.
As discussed above in
At block 408, the system determines mechanical features associated with the oocyte. For example, the system determines parameters indicative of deformation or movement of the oocyte 110 during the image sequence. In this example, the parameters may relate to an elastic model as described in
At block 410, the system determines metrics indicative of oocyte quality using a machine learning model. The system provides the features, for example concatenated features determined for the images or a sequence of features determined for respective images, as input to the machine learning model. In some embodiments, the machine learning model 130 may be a support vector machine. In some embodiments, the model may be a deep learning model (e.g., a neural network). In some embodiments, a subset of the features may be provided. For example, one feature, two features, three features, 10 features, and so on may be provided.
The metrics may include at least information that is indicative of blastocyst formation of the oocyte. Additionally, one or more of the metrics may be indicative of aneuploidy and/or implantation associated with the oocyte.
In some embodiments, one or more of the metrics may indicate whether the oocyte 110 will form a “good” blastocyst, where “good” blastocyst may mean an associated Gardner Embryo/Blastocyst Grading is greater than 3 CC. Additionally and optionally, besides using the example morphological features and the example mechanical features to determine the metrics indicative of quality of the oocyte, the machine learning model may further use clinical information of the patient to determine the oocyte quality information. As mentioned in discussion with respect to
After determining the metrics indicative quality of the oocyte, process 400 may return to block 402 to determine quality information for another oocyte.
At block 502, the system obtains images sequences and pressure values associated with a multitude of oocytes. As described herein, an image sequence may depict an oocyte being deformed due to application or pressure via a pressure tool.
At block 504, the system obtains metrics indicative of quality of individual oocyte in the multiple of oocytes. Determining metrics is described in more detail above and may indicate a likelihood of blastocyst formation for each of the oocytes.
At block 506, the system selects a subset of oocytes from the multitude of oocytes. For example, the system may identify a top threshold number of oocytes based on their respective likelihoods of blastocyst formation. As another example, the system may aggregate or otherwise combine the metrics for each oocyte. For this example, the system may select a top threshold number of oocytes based on the aggregated or combined metrics. As described above, the metrics may indicate blastocyst formation along with successful outcomes of later stages or metrics associated with chromosomal abnormalities (e.g., pgt-a, euploidy), and so on.
At block 508, the system outputs and/or presents information associated with the selected subset of oocytes. The information may be presented through a graphical user interface (GUI), such as the user interface 300A illustrated in
Example System
As illustrated, the oocyte analysis system 100 includes a processor 602, pressure tool 604 (e.g., the pressure tool 112 of
The processor 602 may also communicate to and from the memory 614. The memory 614 may contain computer program instructions (grouped as modules or components in some embodiments) that the processor 602 may execute in order to implement one or more embodiments. The memory 614 generally includes RAM, ROM, and/or other persistent, auxiliary, or non-transitory computer-readable media. The memory 614 may store an operating system 616 that provides computer program instructions for use by the processor 602 in the general administration and operation of the oocyte analysis system 100. The memory 614 may further store specific computer-executable instructions and other information (which may be referred to herein as “modules” or “engines”) for implementing aspects of the present disclosure. For example, the memory 614 may include the feature pre-processing engine 632 and the machine learning model 634, which may implement aspects of the present disclosure as described above. The memory 614 may further store, for example, user interface module 618 that may enable presentation of information to the user interface 300A of
It will be recognized that many of the components described in
All of the methods and tasks described herein may be performed and fully automated by a computer system. The computer system may, in some cases, include multiple distinct computers or computing devices (e.g., physical servers, workstations, storage arrays, cloud computing resources, etc.) that communicate and interoperate over a network to perform the described functions. Each such computing device typically includes a processor (or multiple processors) that executes program instructions or modules stored in a memory or other non-transitory computer-readable storage medium or media or device (e.g., solid state storage devices, disk drives, etc.). The various functions disclosed herein may be embodied in such program instructions, or may be implemented in application-specific circuitry (e.g., ASICs or FPGAs) of the computer system. Where the computer system includes multiple computing devices, these devices may, but need not, be co-located. The results of the disclosed methods and tasks may be persistently stored by transforming physical storage devices, such as solid-state memory chips or magnetic disks, into a different state. In some embodiments, the computer system may be a cloud-based computing system whose processing resources are shared by multiple distinct business entities or other users.
The processes described herein or illustrated in the figures of the present disclosure may begin in response to an event, such as on a predetermined or dynamically determined schedule, on demand when initiated by a user or system administrator, or in response to some other event. When such processes are initiated, a set of executable program instructions stored on one or more non-transitory computer-readable media (e.g., hard drive, flash memory, removable media, etc.) may be loaded into memory (e.g., RAM) of a server or other computing device. The executable instructions may then be executed by a hardware-based computer processor of the computing device. In some embodiments, such processes or portions thereof may be implemented on multiple computing devices and/or multiple processors, serially or in parallel.
Depending on the embodiment, certain acts, events, or functions of any of the processes or algorithms described herein can be performed in a different sequence, can be added, merged, or left out altogether (e.g., not all described operations or events are necessary for the practice of the algorithm). Moreover, in certain embodiments, operations or events can be performed concurrently, e.g., through multi-threaded processing, interrupt processing, or multiple processors or processor cores or on other parallel architectures, rather than sequentially.
The various illustrative logical blocks, modules, routines, and algorithm steps described in connection with the embodiments disclosed herein can be implemented as electronic hardware (e.g., ASICs or FPGA devices), computer software that runs on computer hardware, or combinations of both. Moreover, the various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a machine, such as a processor device, a digital signal processor (“DSP”), an application specific integrated circuit (“ASIC”), a field programmable gate array (“FPGA”) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A processor device can be a microprocessor, but in the alternative, the processor device can be a controller, microcontroller, or state machine, combinations of the same, or the like. A processor device can include electrical circuitry configured to process computer-executable instructions. In another embodiment, a processor device includes an FPGA or other programmable device that performs logic operations without processing computer-executable instructions. A processor device can also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Although described herein primarily with respect to digital technology, a processor device may also include primarily analog components. For example, some or all of the rendering techniques described herein may be implemented in analog circuitry or mixed analog and digital circuitry. A computing environment can include any type of computer system, including, but not limited to, a computer system based on a microprocessor, a mainframe computer, a digital signal processor, a portable computing device, a device controller, or a computational engine within an appliance, to name a few.
The elements of a method, process, routine, or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor device, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of a non-transitory computer-readable storage medium. An exemplary storage medium can be coupled to the processor device such that the processor device can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor device. The processor device and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor device and the storage medium can reside as discrete components in a user terminal.
Conditional language used herein, such as, among others, “can,” “could,” “might,” “may,” “e.g.,” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments include, while other embodiments do not include, certain features, elements or steps. Thus, such conditional language is not generally intended to imply that features, elements or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without other input or prompting, whether these features, elements or steps are included or are to be performed in any particular embodiment. The terms “comprising,” “including,” “having,” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list.
Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to present that an item, term, etc., may be either X, Y, or Z, or any combination thereof (e.g., X, Y, or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain embodiments require at least one of X, at least one of Y, and at least one of Z to each be present.
While the above detailed description has shown, described, and pointed out novel features as applied to various embodiments, it can be understood that various omissions, substitutions, and changes in the form and details of the devices or algorithms illustrated can be made without departing from the spirit of the disclosure. As can be recognized, certain embodiments described herein can be embodied within a form that does not provide all of the features and benefits set forth herein, as some features can be used or practiced separately from others. All changes which come within the meaning and range of equivalency of the claims are to be embraced within their scope.
Number | Name | Date | Kind |
---|---|---|---|
20130184518 | Zarnescu et al. | Jul 2013 | A1 |
20190042958 | Letterie et al. | Feb 2019 | A1 |
Number | Date | Country |
---|---|---|
110969616 | Apr 2020 | CN |
Entry |
---|
Isa et al. (Appl. Sci. (2023) vol. 13, 22 pages; published Jan. 16, 2023). |
Ma et al. (Theranostics (2021) vol. 11, Issue 15:7391-7424). |
Liu et al. Micromachines (2019) vol. 10: 16 pages). |
Lamont et al. (Journal of the Mechanical Behavior of Biomedical Materials (2023) vol. 138:12 pages). |
McQuin et al. (PLoS Biology (2018) vol. 16(7):17 pages). |
Gröger et al. at pp. 65-75 (Medical Image Learning with Limited and Noisy Data (2022), LNCS 13559; article entitled: BoxShrink: From Bounding Boxes to Segmentation Masks by Gröger et al. at pp. 65-75). |
Moen et al. (Nature Methods (2019) vol. 16:1233-1246). |
Khalilian et al., “Estimating Young's modulus of zona pellucida by micropipette aspiration in combination with theoretical models of ovum,” Journal of Royal Society Interface, 2010, 7, p. 687-694, doi:10.1098/rsif.2009.0380. |
Yanez et al., “Human oocyte developmental potential is predicted by mechanical properties withinhours after fertilization,” Nature Communications, 2017,7:10809, pp. 1-12, DOI: 10.1038/ncomms10809. |
International Search Report and Written Opinion mailed Dec. 6, 2023 in International Application PCT/CN2023/108002. |