The present invention relates generally to the field of detection, processing, and manipulation of biological cells, and in particular to systems and methods for vitrification of mammalian oocytes or embryos.
Cryopreservation of mammalian reproductive cells is an essential technique in IVF (in vitro fertilization) clinics. Oocytes and embryos are routinely frozen and preserved for use at a later time. Patients who undergo therapeutic procedures that can place their fertility at risk, such as chemotherapy, have the option of preserving their oocytes for future use through IVF techniques. Additionally, fertilized embryos are often more needed for one cycle of IVF treatment. The rest of fertilized embryos are usually cryopreserved for future use.
Techniques of oocyte/embryo cryopreservation are classified into two categories, slow freezing and fast freezing (i.e., vitrification). Slow freezing is a well-established technique developed during the early 1970s, which makes use of programmable sequences, or controlled cooling rates. Vitrification or fast freezing is a more effective cryopreservation method, first reported in 1985 (see W. F. Rall and G. M. Fahy, “Ice-free cryopreservation of mouse embryos at −196 degrees C. by vitrification,” Nature, vol. 313, no. 6003, pp. 573-5, 1985). Vitrification is considered superior to slow freezing, because it vitrifies the oocyte/embryo with no crystal formation during freezing. The addition of cryoprotectants in vitrification increases the embryo viscosity, and makes the vitrified embryos syrupy. When directly freezing oocytes/embryos in liquid nitrogen, the syrupy content inside the cell forms amorphous ice instead of ice crystals, which minimize the vital damage to the cell during freezing.
Oocyte/embryo vitrification is done manually in IVF clinics. An operator looks though the microscope eyepieces and manipulates oocytes/embryos using a micropipette. An oocyte/embryo is first taken out from the culture dish and washed with the equilibrium solution (ES) and a series of vitrification solution (VS). Within each step, the control of timing has been proven critical. After the washing steps, the processed oocyte/embryo is placed onto a vitrification straw or into a cryo-pipette. The volume of solution remained around the oocyte/embryo on the straw must be minimal in order to ensure a high cooling rate. The vitrification straw is then plunged into liquid nitrogen for freezing and long-term cryopreservation. A number of different commercial vitrification solutions and protocols exist; however, the core steps are largely the same. All the protocols involve multiple washing steps with ES and VS, placing the vitrified oocytes/embryos on vitrification straws, and freezing the vitrification straws in liquid nitrogen.
Manual operation for oocyte/embryo vitrification is a demanding and tedious task, for the following reasons: (1) washing oocytes/embryos with the highly viscous VS causes osmotic shock to the cells, and osmotic shock can be a major cause for cell damage; (2) most antifreezing solutes (e.g., DMSO) are toxic to ooctytes/embryos. Therefore, the washing time in toxic VS is critical but can be difficult to strictly control; (3) because of their small size (about 150 μm), oocytes/embryos can be difficult to detect and manipulate, especially when the medium surrounding the cells is dynamically changing during micropipette aspiration and dispensing; (4) the manual process has stringent skill requirements, and success rate and cell survival rate vary across different operators.
Heo et al. developed a microfluidic platform to control the loading of cryoprotectants for oocyte cryopreservation (see Y. Heo, H. Lee, B. Hassell, et al., “Controlled loading of cryoprotectants (CPAs) to oocyte with linear and complex CPA profiles on a microfluidic platform,” Lab on a Chip, vol. 11, no. 20, pp. 3530-7, 2011). When an oocyte/embryo is ‘parked’ at a position of the microfluidic platform, one side of the oocyte/embryo directly faces the vitrification solutions while the other side does not. Accordingly, the oocyte/embryo is not exposed to VS as uniformly as in the standard manual protocols. In practice, parking embryos in a container or preset location can bring about significant difficulties for retrieving embryos after washing in VS. The timing of imposing oocytes/embryos to VS is critical because the cryoprotectants in VS can impose toxic effects on embryos if they cannot be retrieved in time.
Genea Biomedx Inc. developed an automated instrument for oocyte/embryo vitrification (U.S. Pat. Appl. Publ. No. 2013/0137080). The Genes system requires a user to manually transfer oocytes/embryos into an array of wells termed ‘pods’ with one oocyte/embryo in each well. In present vitrification protocols used in IVF clinics, an oocyte/embryo is moved in and out of different vitrification solutions. Differently, oocytes/embryos in the Genea system stay in the wells while vitrification solutions are dispensed into and aspirated out of the wells. Since the oocyte/embryo always sits on the bottom of the well, the cell surface in contact with the well bottom cannot be exposed to vitrification solutions as uniformly as the rest of the cell surface. Furthermore, the Genea system dispenses and aspirates fluids based on volume control without monitoring cell position; therefore, the technology cannot meet the minimal volume requirement to achieve a high cooling rate.
Similar to Genea Biomedx's automated vitrification system, patent application publication numbers WO2013020032 by Samuel S. Kim et al, US2011/0207112 by Fred Burbank et al, and WO2013098825 by Amir Arav also disclosed methods of parking oocytes/embryos in preset locations and changing VS. WO2013020032 by Samuel S. Kim et al. describes an automated device comprising a cryoprotectant holder, a cryoprotectant dispenser, and a sample holder oriented to allow a sample to be in contact with cryoprotectant from said cryoprotectant dispenser. In this device, oocytes or embryos are kept in a sample holder (e.g., electron microscopy grid) throughout the entire procedure, while the VS are dispensed in and drained out for washing the sample.
In U.S. Pat. Appl. Publ. No. 2011/0207112 by Fred Burbank et al., one or more oocytes or embryos are positioned in a processing container, the processing container being configured to allow fluid to flow into and out of the processing container, where two or more fluids flow into and out of the container.
WO20130988205 by Amir Arav disclosed a device comprising a draining zone and a capillary draining element. The draining zone is configured to hold a reproductive biological sample. The capillary draining element, whose opening is within the draining zone, is configured to drain liquid away from the draining zone while a reproductive cellular constituent of the reproductive cellular portion remains within the draining zone.
Different from the presently used vitrification protocols by placing vitrified oocyte/embryos to a straw-like carrier, US Pat. Publ. Nos. 2013/0157362 by Fuliang Du et al., 2002/0009704 by Xiangzhong Yang et al., and 2012/0251999 by Utkan Demirci et al. disclosed methods for generating micro/nano droplets of VSs comprising the biological sample, and freezing the droplets directly in liquid nitrogen.
Milton Chin disclosed in US Pat. Appl. Publ. Nos. 2009/0186405 and 2009/0123996 device designs for improving the chilling rate in liquid nitrogen and achieving the self-sealing function for storage of vitrified specimens in liquid nitrogen.
Ru et al. in Chinese Pat. Appl. Publ. No. 202918907U disclosed a semi-automated system for embryo vitrification. The system requires several key steps of manual input to conduct embryo pick-and-place. In paragraph 0056, the system requires the operator to obtain the micropipette tip position by observing from the microscope and input the position information into computer. Similar human inputs are required (see paragraphs 0062, 0070, and in 0072). The automation part is only about using robotic manipulator to pick up an embryo, move to a preset location, and place the embryo into vitrification solutions. There is no computer vision algorithm to automatically detect embryos and micropipette tip. Moreover, the technique of single cell pick-and-place with a robotic system is open knowledge, such as disclosed by Z. Lu, C. Moraes, G. Ye, C. A. Simmons, and Y. Sun in “Single cell deposition and patterning with a robotic system,” PLoS ONE, Vol., 5, e13542, 2010. Additionally, the system disclosed in CN202918907U can only work with a specific Cryotip® method (see paragraph 0086), which is limited for general applications. Without washing embryos in vitrification solutions, the system cannot be used for other protocols (e.g., Kitazato's Cryotop method, and Irvine vitrification protocol) that require embryos to be washed for at least one time.
What is needed is an automated vitrification system designed to automatically process oocytes/embryos and other cell types with vitrification processing solutions, automatically place vitrified oocytes/embryos to vitrification devices (e.g., Cryotop, Cryotip®, Cryoloop, etc.), remove excessive medium from the vitrified oocytes/embryos on the devices, automatically seal the vitrification devices with caps on a sealing machine, and freeze in an automated liquid nitrogen storage tank/system.
According is one embedment of the invention, there is provided a system for automatically cryopreserve and thaw biological materials, such as cells, oocytes and embryos. The alternated system, in one embodiment, includes (a) a carrier plate having (i) an area for holding or loading the biological material, (ii) a multi-well area for holding processing solutions, and (iii) a straw area for holding one or more verification straws; (b) a first robot or robotic manipulator collected to a manipulator to manipulate the biological material and the processing solutions; (c) a second robot or robotic manipulator to manipulate the vitrification straws; (d) a microscope and an image capturing device operatively linked to the microscope to capture images of the biological material, the manipulator and the straws; (e) a computer readable medium having executable instructions; and (f) a processor for executing the executable instructions of the computer readable medium; said executable instructions including instructions for automatically: (i) processing the captured images, and (ii) operatively controlling the microscope, the image capturing device, the first robot, and the second robot.
According to one aspect of this embodiment of the present invention, the system further includes a straw-sealing machine and an automated cryopreservation storage device. and the executable instruction include instructions for automatically controlling the straw-sealing machine and the automated cryopreservation storage device.
According to another aspect of the system of the present invention, the executable instructions further include instructions for: (i) when the biological material is a cell, embryo or oocyte, detecting the cell, embryo or oocyte at different development stages from the captured images; (ii) tracking the biological material in three dimensional space; (iii) monitoring the biological material's volume; (iv) generating a concentration gradient in the treatment solutions; (v) placing the biological material on the straws; and (vi) removing excessive treatment solution from biological material placed on the straw.
According to another aspect of this embodiment the microscope is in communication with a controller including a drive for changing magnifications and focus.
According to another aspect of this embodiment the microscope include an X-Y stage for receiving the carrier plate, the X-Y stage inducing an X-axis linear motion system and a Y-axis linear motion system. In one aspect of the inventor, the executable instructions include instructions to operatively controlling the X-Y stage.
According to another aspect of this embodiment, the microscope is in communication with a controller including a driver for each of the X and Y-axis linger motion systems.
According to another aspect of this embodiment, the X-axis linear motion system and the Y-axis liner motion system are independently controllable.
According to another aspect of this embodiment the carrier plate is integrated with a heating plate to maintain a suitable temperature (for example about 37 degrees Celsius) under the biological material holding area of the carrier plate.
According to another aspect of this embodiment, the manipulator is a micropipette with a tip diameter suitable for manipulating the biological material, for example of about 100 to 200 micrometer.
According to another aspect of this embodiment the micropipette is linked to a micropipette holder, and connected to a motorized syringe.
According to another aspect of this embodiment, the micropipette is placed in the X-Z plane (i.e., perpendicular to the Y axis) at a tiling angle with the tip side down.
According to another aspect of this embodiment, the system further includes a motorized syringe in communication with a controller including a driver to control the linear motion, and the executable instruction include instructions for automatically controlling the motorized syringe.
According to another aspect of this embodiment, the micropipette holder is coupled to the first robotic manipulator.
According to another aspect of this embodiment, the first robotic manipulator has at least three degree of freedom along X, Y and Z.
According to another aspect of this embodiment, the first robotic manipulator is in communication with a controller including a driver for each of the X, Y, and Z axis linear motion system.
According to another aspect of this embodiment real-time images are captured from the microscope and image capturing device at a frame rate of 30 frames per second or higher to provide the system with continuous visual information.
According to another aspect of this embodiment, when the biological material is an embryo, the system is capable of automatically defecting embryos at different development stages (e.g., 2-cell, 4-cell, and blastocyst) from the captured images.
According to another aspect of this embodiment, the system further includes a biological material detection algorithm.
According to another aspect of this embodiment, the system is capable of automatically focusing on the objects by maximizing focus measures.
According to another aspect of this embodiment, the biological material detection algorithm further includes an adaptive threshold algorithm to binarize the gray scale image.
According to another aspect of this embodiment, the biological material detection algorithm further detects the contours of all objects in the binarized image.
According to another aspect of this embodiment, the objects having contour area in a specific range are considered as potential biological material targets.
According to another aspect of this embodiment, in order to avoid false detection, the contours of potential biological material targets are further fitted to circles by using a Hough transform algorithm, and the center point of the circle is considered to be the biological material position.
According to another aspect of this embodiments the system is capable of automatically detecting the micropipette tip from the captured images.
According to another aspect of this embodiment, the micropipette tip detection algorithm further includes two Sobel filters along both x and y axis.
According to another aspect of this embodiment, the Sobel filter along y-axis is used to determine the outside walls of the micropipette, and the Sobel along x-axis is used to determine the tip position.
According to another aspect of this embodiment, the outside walls of the micropipette are further detected by using a Hough line transform algorithm.
According to another aspect of this embodiment, the outside walls of the micropipette tip are used to determine a region of interest (ROI) for biological material determine when the biological material is aspirated into the micropipette.
According to another aspect of this embodiment, the relative distance between micropipette tip and biological materials along a Z axis is determined by a contact detection algorithm that is based on detecting a horizontal sliding motion when the tip of the micropipette contacts a substrate that holds the biological materials (for example a culture dish).
According to another aspect of this embodiment the robotic manipulator moves the micropipette tip close to a target biological material via closed-loop visual servo control.
According to another aspect of this embodiment, the motorized syringe is used to generate a negative pressure to aspirate target biological materials into the micropipette tip after the tip is moved close to the biological material.
According to another aspect of this embodiment, the motorized syringe is also used to generate a positive pressure to dispense the target biological material from the micropipette tip into a desired location.
According to another aspect of this embodiment, a computer vision algorithm is used to detect the biological material's position inside the micropipette.
According to another aspect of this embodiment, the biological material is precisely postponed to a desired location insider the micropipette via closed-loop visual servo control by controlling the motorized syringe.
According to another aspect of this embodiment, the biological material is automatically moved from one location to another location (e.g., transfer from culture dish to multi-well plate).
According to another aspect of this embodiment, the biological material is automatically washed in vitrification solutions by repeating the biological material aspiration and dispensing.
Acceding to another aspect of this embodiment, the system is capable of automatically tracking biological materials in 3D space when the biological material is transferred from ES to VS.
According to another aspect of this embodiment, the system is able to automatically generate a concentration gradient in VS by coordinately moving micropipette tip and dispensing ES of low concentration.
According to another aspect of this embodiment, the system is capable of automatically monitoring the biological material's volume change in real time to provide a criterion for optimizing the washing time in different types of solutions.
According to another aspect of this embodiment, the vitrified biological material is automatically taken out from VS via micropipette aspiration when it teaches the minimum volume.
According to another aspect of this embodiment, the biological material is automatically placed onto a vitrification straw after washed in VS.
According to another aspect of this embodiment the verification straw position is automatically detected by a computer vision algorithm.
According to another aspect of this embodiment, vitrification straw detection further includes a Hough line transform algorithm to detect the two horizontal edges of the straw.
According to another aspect of this embodiment, the vertical distance between micropipette tip and vitrification straw is detected by contact detection that is based on the vitrification straw tip's deflection motion caused by the micropipette tip's contact.
According to another aspect of this embodiment, the micropipette tip is automatically moved to the center of the vitrification straw and contacts the straw surface.
According to another aspect of this embodiment, the vitrified biological material is dispensed on the straw by controlling the motorized syringe to apply a positive pressure.
According to another aspect of this embodiment, excessive medium around the biological material is automatically aspirated away by the micropipette at a location with a distance (e.g., 300 micrometer) away from the position where the biological material is dispensed.
According to another aspect of this embodiment, the aspiration of excessive medium is automatically stopped when the volume of the biological material droplet on the straw does not change further.
According to another aspect of this embodiment, the vitrification straw with vitrified biological materials on the tip is picked up from the carrier plate by a robot.
According to another aspect of this embodiment, the robot plunges the vitrification straw with vitrified biological materials on tip into liquid nitrogen in a sub-container for fast freezing.
According to another aspect of this embodiment, the vitrification straw after freezing is placed on the sliding part of a sealing machine.
According to another aspect of this embodiment, the sealing machine, preloaded with vitrification straw caps, caps the straws by moving the siding part.
According to another aspect of this embodiment, the vitrification straw sealed with the cap is inserted in a slot of the sub-container containing liquid nitrogen.
According to another aspect of this embodiment, the sub-container is automatically transferred into an automated liquid nitrogen storage system for long-term cryopreservation.
According to another aspect of this embodiment, the washing solution types and soaking time can be readily specified via the software interface of the system, according to different vitrification kits and protocols.
According to another aspect of this embodiment, the system is capable of processing multiple biological materials with an optimized schedule for saving the total processing time.
Embodiments of the present invention provide also for automated methods for cryopreserving and thawing biological material, particularly oocytes, embryos, and other cellular materials using the systems of the present invention. In one embodiment of the present invention, a method for processing biological materials include, (a) automatically (i.e., not human operator assisted) and uniformly exposing the biological material to a series of vitrification processing solutions, thereby obtaining vitrified biological material, (b) automatically placing the vitrified biological material on the surface of a vitrification straw, (c) automatically dipping or plunging the vitrification straw carrying the vitrified biological material in cryopreservation solution, (d) automatically sealing the vitrification straw carrying the vitrified biological material, (a) automatically storing the sealed vitrification straw in cryopreservation solution thereby freezing the vitrification straw with the vitrified biological material for long term cryopreservation.
In another embodiment of the method of the present invention, the method further comprises automatically retrieving the frozen seated vitrification straw from the cryopreservation solution, and automatically and uniformly exposing the frozen vitrified biological material to a series of processing solutions, thereby thawing the biological material.
The following figures illustrate various aspects and preferred and alternative embodiments of the invention.
Definitions
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Also, unless indicated otherwise, except within the claims, the use of “or” includes “and” and vice versa. Non-limiting terms are not to be construed as limiting unless expressly stated or the context clearly indicates otherwise (for example “including”, “having” and “comprising” typically indicate “including without limitation”). Singular forms including in the claims such as “a”, “an” and “the” include the plural reference unless expressly stated otherwise. All documents cited are incorporated herein by reference in their entirety. In order to aid in the understanding and preparation of the within invention, the following illustrative, non-limiting, examples are provided.
“Automatic/automated” as used in this document means free of human operator control (for example manual and/or joy-stick based control) or where human intervention is limited to entering input data. Software (image processing and motion control) and hardware are integrated in a computer machine to reduce or eliminate the operator's intervention.
“Motorized” as used in this document moans that devices such as positioning devices, syringes and so forth, are equipped with a motor or motors which are controlled via control algorithms and strategies.
Overview
The present invention relates to hardware components and computer vision algorithms to automate visitation process, including full automation. The present invention may be used to implement vitrification kits/protocols. For instance, the Kitazato's kit/protocol (Kitazato Inc., Cryotop Safety Kit Protocol, 2014) and the Origio Vitrification protocols (Origio MediCult Media, Medicult Vitrification Cooling Protocol, 2014) can be followed in the robotic system of the present invention by simply changing the oocyte/embryo washing time in vitrification solutions.
The robotic system of the present invention is designed to be fully automated, including automated biological material loading and retrieval. The system monitors the response of individual cells in vitrification solution (VS) to determine the optimal washing time in VS. After washing cells in VS, the verified embryos are automatically placed onto vitrification straws. The straws are then sealed with a cap on a cap-sealing machine. The sealed straws are picked up and frozen in a cryopreservation sub-container, such as a liquid nitrogen sub-container. The sub-container containing the frozen vitrification straws is finally placed in an automated liquid nitrogen system for long-term cryopreservation. The robotic system performs tasks in an optimized parallel way to process multiple cells or cell lines. The system also allows users to readily specify the operation time of each step for realizing different verification protocols or for developing new protocols.
A. Automated Vitrification System
Embodiments of the invention relate to a system that includes a motorized X-Y stage microscope movable in two axes for moving sample in the X-Y plane, a camera for capturing images, a robot for positioning a manipulator, such as a micropipette, a motorized syringe connected with the micropipette for aspirating and dispensing embryos/oocytes/cells (biological materials), a carrier plate for holding the biological material culture dish, a multi-well plate for holding processing solutions, such as vitrification and thawing solutions, and multiple vitrification straws, a second robot for handling vitrification straws, and an automated liquid nitrogen storage system for long-term cryopreservation of the vitrified samples. This section describes an embodiment of the hardware system, the system work flow, and technical details.
Vitrification straws may be described as physical carriers to hold embryos for freezing and cryopreserving in, for example, liquid nitrogen. Vitrification straws can be classified into three groups: opening strip based straws (e.g., Cryotop), micropipette based straws (e.g., Cryopipette), and CPA (cryoprotective agent) film based straws (e.g., Cryoloop). The opening strip based straws often consist of a polypropylene strip attached to a hard plastic handle. Vitrified embryos are placed directly on top of the strip. After placing embryos on strip, the straws are sealed with plastic caps to avoid contaminations in long-term cryopreservation. The micropipette based straws use micropipettes to directly aspirate vitrified embryos. After the vitrified embryos are aspirated into the micropipette, the opening of the micropipette is sealed by a heat sealer. The CPA film based straw, taking Cryoloop for example, involves a plastic loop that can be integrated into a lid of a cryovial via a stainless steel stick. During vitrification, the CyroloopT is immersed into a cryoprotective solution to create a thin film of the solution inside the loop. The vitrified embryos are then transferred into the film and sealed in the cryovial. In this document, the term “vitrification straw” refers to all three classification groups. Among all the vitrification straws, Cryotop is a well-accepted method because it involves least residual solutions surrounding vitrified embryos for producing the highest cooling rate and highest post-thawing cell survival rate. Accordingly, herein Cryotop is used as an example to explain the present invention.
In one embodiment with reference to
The system 1 shown in
As shown in
The biological material, such as embryos, oocytes or cells, in dish 301 may be manipulated by using a manipulator, such as micropipette 510. The micropipette may have a tip diameter suitable for aspirating or dispensing the biological material. In the case of embryos, the micropipette may have a tip diameter in the range of about 100-200 micrometer. The micropipette may be inserted into a micropipette holder 511 connected to the motorized syringe 600. The micropipette holder 511 is clamped to a robotic manipulator 501, which has a travel range of centimetres and a resolution better than 1 μm along each axis.
After processing biological material on the carrier plate 300 (i.e., washing biological material with a series of VS, then placing biological material on the vitrification straw), a vitrification straw 303 with vitrified biological material on top may be picked up by a second robot 502 that plunges or dips the straw into liquid nitrogen in a sub-container 901 for fast freezing. The frozen vitrification straw may then be placed by the robot to a cap-sealing machine 800 for sealing vitrification straw with a plastic cap. The sealed straw may finally be inserted into the storage slots of the sub-container, and the sub-container is transferred by the robot 502 into an automated liquid nitrogen storage system 902.
Robots 501 and 502 may be multiple degrees of freedom robots. In one embodiment, robots 501 and 502 may have at least three degrees of freedom. One robot may serve to position the micropipette to pick and place cells and solutions, and the other robot may be used to handle vitrification straws.
With reference to
B. Automated Embryo Vitrification Work Flow
The vitrification work flow described below is explained by using the implementation of Kitazato vitrification protocol as an example. The system of the present invention may also be capable of automating the implementation of other protocols, such as that of Origio Vitrification protocols.
Step 0. System Setup and Preparation
The set up step is the only step which may include sub-steps that may involve an operator. All other steps in the method are fully automated.
Step 1. Automated Pick-Up of Embryos from Culture Dish
Step 2. Alternated Washing of Embryos in ES
Step 3. Automated Washing of Embryos in VS
Step 4. Automated Placing of Embryo on Vitrification Straw
Step 5. Automated Freeze, Seal and Cryopreserve the Vitrification Straw
Typically, more than one oocyte/embryo/cell needs to be processed within a batch. In order to shorten system idle time, task scheduling may be made/optimized.
C. Thawing Process
The set up step is the only step which may include sub-steps that may involve an operator. All other steps are fully automated.
Step 0. System Setup and Preparation
Step 1. Automated Thaw Embryos in TS
Step 2. Automated Transfer Embryos from TS to DS
Step 3. Automated Wash Embryos and Transfer to Culture Medium
D. Techniques for Automated Vitrification
Techniques for implementing the work flow discussed in the previous section are described below. These techniques include: micropipette/vitrification straw detection, contact detection, autofocusing, embryo detection/tracking, positioning embryos in micropipette, generation of a moderate concentration gradient in VS, removal of excessive VS from the vitrification straw, a vitrification straw sealing machine, and an automated liquid nitrogen storage system. The techniques described below and illustrated in the referred figures may be implemented as machine-accessible instructions utilizing any of different programming codes stored in computer 50.
Detection of Micropipette and Vitrification Straw
The first step may be to binarize an image of the micropipette or vetrification straw and extract target objects from the background. In IVF clinics, stereo up-right microscope is often used for examining the inner morphological detail of embryos. Stereo microscope produces a lighting gradient on the captured images. The system runs a histogram-based thresholding algorithm to discriminate foreground objects (e.g., micropipette, or vitrification straw) from background. At first, a histogram of gray scale value of all pixels may be calculated. The histogram may include two peaks, with one representing the background pixels and the other representing the pixels of the foreground objects. The middle point of the two pixels may be used by the system as threshold to bitwise the image for extracting foreground objects, as shown in
After image binarization, morphological transformations (i.e., erosion and dilation) are performed to remove the noisy objects of small sizes. Since the micropipette is much larger than embryos, the largest object in the binarized image is taken by the system to be the micropipette. In addition to using size as a criterion for detecting micropipette, the shape information is also used by the system to ensure correct defection. The outside walls of the micropipette can be modeled as two lines, therefore, Hough line transform is applied on the contours of detected objects (see
In the robotic system, the micropipette tip points to the right direction. According to this specific system setup, the point (on the detected micropipette object) with the largest distance to the left boundary of the image is considered by the system to be the micropipette tip position. Within the detected micropipette, a smaller region is extracted as a region of interest (ROI) for detecting embryos inside micropipette.
The detection of vitrification straw is similar to micropipette detection. The difference is that the orientation of the vitrification straw (with the tip pointing to the left side) is opposite to the micropipette.
Contact Detection
The relative distance along Z axis between micropipette tip and dish bottom or vitrification straw's surface may be critical for successful automation of picking/placing embryos from/to dish or straw.
The relative distance between micropipette tip and dish bottom may be detected based on detecting micropipette tip's sliding motion on the dish bottom. The system constantly visually tracks the tip's position in the x-y plane, while lowering the micropipette downwards to the dish bottom. As illustrated in
The detection of the micropipette tip's contact on vitrification straw is based on a different principle. The vitrification straw 303 has low stiffness and is easily deflected when contacted by the micropipette tip. Therefore, the robotic system of the present invention detects the contact between micropipette tip and straw by detecting the straw's deformation motion (see
Another method for detecting the tip's contact with vitrification straw is based on detecting the spreading of medium on the straw. When micropipette tip contacts the straw, medium inside the micropipette passively flows out to the hydrophilic straw surface, due to surface tension. Therefore, the contact position can be determined by detecting medium spreading around the micropipette tip. When an object appears around the tip position in the binarized image, contact is considered detected. The robotic manipulator 501 then stops downward movements and records the reference Z position.
Autofocusing
One prerequisite stop for the aforementioned detection processes is to focus on a reference surface (i.e., on dish bottom or on straw surface). Accordingly, the system conducts auto-focusing. The autofocus algorithm uses a focus measure based on Tenenbaum gradient to determine the focus level of the captured image. This algorithm convolves an image with Sobel operators, and then sums the square of the gradient vector components. The system adjusts the microscope focal plane until the focus measure reaches maximum. The first order derivative of focus measure is used to determine the maximal value. When the focal plane moves to the target objects (e.g., embryos or micropipette tip), the focus measure is increasing and the first order derivative is positive. Once the first order derivative turns to negative, the local plane has reached the optimal position at the turning point and starts to move away from the target objects.
In order to avoid the loss of embryos through the entire vitrification process, robustly detecting and tracking embryos is important. With reference to
In order to rapidly locate embryos in the relatively large culture dishes, the embryo detection algorithm further involves the change of the microscope magnifications and the movement of the X-Y stage. The system first detects potential embryo objects under a lower magnification (i.e., larger FOV). Then the potential embryo objects are positioned to the center of FOV by moving the X-Y stage. The system then automatically switches the microscope to a higher magnification, and performs embryo detection again to verify the detection result from under lower magnifications and analyze the inner morphology of the embryo.
When transferring embryo from lower-concentration ES to VS or during the washing steps, embryos can move up and down inside the solutions, due to the buoyancy force and/or fluidic flow caused by micropipette movement/dispensing. In order to tracking embryos in 3D space, the embryo detection algorithm is integrated with the autofocusing algorithm by dynamically moving the focal plane to keep the tracked embryo(s) in focus.
Positioning Embryos in Micropipette
During embryo aspiration and dispensing, the robotic system quickly and accurately positions the embryo in micropipette. The embryo's position is visually tracked to provide position feedback for the system to realize closed-loop visual servo control. The desired position inside the micropipette is at a pre-specified distance to the tip. This distance cannot be too long because it would introduce too large a volume of the solution when dispensing the embryo into the next vitrification solution. On the other hand, the distance cannot be too short because the embryo can possibly ‘escape’ out of the micropipette tip, resulting in oocyte/embryo loss. Based on the error between desired position and detected positions, the system runs a controller (e.g., proportional-integral-derivative (PID) or robust controller) algorithm to control the motorized syringe and aspirate or dispense until the oocyte/embryo reaches the desired position.
Generation of a Moderate Concentration Gradient
In vitrification, ES often has the same solute as VS but at a lower concentration. After equilibrated in ES, the embryo encounters an osmotic shock when transferred into VS due to the concentration change. In order to lessen cell damage, this osmotic shock may be minimized. Therefore, the robotic system may generate a moderate concentration gradient in VS. To generate a moderate concentration change, the system moves the micropipette along the X axis to the left while dispensing ES into VS. ES medium is dispensed at a constant rate (e.g., at 5 μl/sec) while the speed of micropipette movement keeps increasing until micropipette tip is moved to the edge of FOV. Since the mixing rate between ES and VS is constant, a concentration gradient is generated along the micropipette's moving path.
When an embryo is dispensed out from the micropipette tip, they are first positioned at the locations where the concentration is lower (i.e., similar to ES). Then the embryos are gently moved along the concentration gradient increasing direction (e.g., from right to left). After that, the embryos are washed several times by aspirating into and dispensing out of the micropipette.
Another method to minimize osmotic shock pressure is to use a series of vitrification solutions with moderate concentration changes. However, this method involves more types of VS and more steps of embryos pick-and-place.
Removal of Excessive VS from the Vitrification Straw
After robot 501 places a vitrified embryo onto the straw, the system removes excessive medium to achieve a high cooling rate. As shown in
Throughout this step, the system utilizes image processing to monitor droplet volume change and micropipette tip positions. The embryo is also detected from the straw ROI by using the methods described in the previous section “Embryo Detection/Tracking”. Since the embryo is surrounded by the excessive medium, the volume of the detected object is proportional to the volume of the excessive medium. Accordingly, when the detected embryo volume stops changing, it is considered that all excessive medium has been removed from the vitrified embryo.
Vitrification Straw Sealing
The vitrification straw sealing machine is design to automatically place a plastic cap onto the straw tip in order to avoid contamination during the long-term cryopreservation. As shown is
Automated Liquid Nitrogen Storage System
As part of the robotic vitrification system, an automated liquid nitrogen storage system is designed. The automated storage system comprises a liquid nitrogen tank, a built-in 3-axis transport system, and liquid nitrogen sub-containers. The three-axis transport system 903 is automated to locate a sub-container 901 and transfer it into or out of the storage tank 904 containing liquid nitrogen. As shown in
As shown in
E. Advantages
The system of the present invention automates the implementation of well accepted vitrification protocols. Without changing well-established protocols or modifying embryo handling approach, the system of the present invention can be easily accepted in clinics.
Embryos are monitored in real time throughout the entire vitrification process, which can effectively avoid embryo loss.
The systems and methods of the present invention are fully automated including the loading and retrieving embryos (vs. manual loading in microfluidic vitrification), and detecting the embryos position in solutions (as opposed to the manual input in patient by Ru et al. in Chinese Pat. Appl. Publ. No. 202918907U).
The systems and methods of the present invention can generate a moderate concentration gradient which can minimize the osmotic pressure and further increase the cell survival rate.
The systems and methods of the present invention can automatically place vitrified embryos onto vitrifications straws and remove excessive surrounding solutions to achieve ‘minimal volume’, in order to achieve high cooling rates for freezing.
The systems and methods of the present invention is capable of processing multiple embryos with an optimized schedule for saving the total processing time.
The systems and methods of the present invention includes an automated sealing machine that can be used to place caps to vitrification straws, and an automated cryopreservation storage system which can eliminate human errors in embryo handing.
The oocytes, embryos and cells are exposed to the different processing solutions (i.e. washing, equilibrium, vitrification, thawing solutions) as uniformly as in the standard manual protocols.
Through the embodiments that are illustrated and described, the currently contemplated best mode of making and using the invention is described. Without further elaboration, if is believed that one of ordinary skill in the art can, based on the description presented herein, utilize the present invention to the full extent.
Although the description above contains many specificities, these should not be construed as limiting the scope of the invention, but as merely providing illustrations of some of the presently embodiments of this invention. Thus the scope of the invention should be determined by the appended claims and their legal equivalents, rather than by the examples given.