Patent Grant
11280803
An automated system of processing biological specimens.
In various settings, examination of biological specimens is required for diagnostic purposes. Generally speaking, pathologists and other diagnosticians collect and study samples from patients, and utilize microscopic examination, and other devices to assess the samples at cellular levels. Numerous steps typically are involved in pathology and other diagnostic process, including the collection of biological samples such as blood and tissue, processing the samples, preparation of microscope slides, staining, examination, re-testing or re-staining, collecting additional samples, re-examination of the samples, and ultimately the offering of diagnostic findings. Numerous medical or veterinary personnel may be involved in the diagnostic processes, including surgeons, phlebotomists or other operating personnel collecting samples, pathologist, histologists and other personnel processing, transporting and examining the samples and so on. The complexity of the tissue handling procedures from the operating room to the laboratory and back to the diagnosticians or surgeons have become increasingly complex in large medical environments where high volumes of samples need to be handled, processed and examined on a daily basis.
Various steps of the tissue handling procedures have been automated using instruments each of which typically are controlled by a dedicated computer or an on-board computerized controller. In some laboratories, information can be shared between automated instruments and/or a networked laboratory or hospital information system, such as to store patient or tracking data. One example of an automated instrument is an automated tissue processing system in which biological samples are fixed and infiltrated with paraffin in an automated fashion. Exemplary tissue processing systems are the TISSUE-TEK® VIP® and the TISSUE-TEK® XPRESS® processing systems available from Sakura Finetek U.S.A., Inc. of Torrance, Calif.
Another example of automation is an automated slide stainer and coverslipper, which stains microscope slides and applies coverslips to the slides in an automated fashion. Examples of such automated staining and coverslipping systems are TISSUE-TEK® PRISMA® and TISSUE-TEK® FILM® combo system and TISSUE-TEK® PRISMA® and TISSUE-TEK® Glas™ g2 combo system available from Sakura Finetek U.S.A., Inc. of Torrance, Calif.
Despite the assistance of automated instruments, pathologists, other diagnosticians and laboratory personnel typically must be involved in numerous steps during the processing and examination of biological specimens. For example, once a specimen has been stained, the stained specimen on a microscope slide may be physically examined under a microscope. This typically involves transport of the microscope slide to a diagnostician who is located outside the laboratory, or in other cases may involve a diagnostician going to the laboratory to examine the microscope slide. Alternatively, the stained specimen on a microscope slide is imaged with a digital scanner and the image of the specimen is uploaded for examination by a diagnostician.
Following this initial examination step, the diagnostician generally evaluates whether additional testing is required. Such additional testing might involve collecting further specimens (samples) from a patient, or further examination of specimens already collected. For example, the diagnostician may retrieve the physical slide containing the specimen subject to the initial examination and examines the specimens using a traditional microscope. The diagnostician may also require that the existing specimen be sectioned further and a different staining regimen or other protocol be applied. This can result in iterations of one or more of collection, grossing, processing, infiltration, embedding, sectioning, coverslipping, staining, examination etc. In addition, different coverslipped slides may require different drying times. Accordingly, some slides may be ready for examination while others are not. All of this can result in time delays, as well as tissue impairment. Following the iterations of additional tests and procedures, the pathologist repeats the examination process, and may then request still further tests in an iterative fashion until an ultimate finding is reached.
The embodiments of the invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean at least one.
In overview, a system (apparatus) and process for performing a series of automated operations including tissue processing, imaging and tissue storage is disclosed.
At the material handling system, the slide mounted specimen may be processed via automated operations into a condition suitable for a desired examination. In one embodiment, processing includes staining the biological sample and applying a cover slip to the slide (block 104). Staining of the specimen may be optional. The slide having the specimen thereon is then transferred to a transport module (block 106). In some embodiments, the slide is transferred to the transport module using a robotic transferring device as will be discussed in more detail with reference to
Process 100 also includes determining whether the slide is ready for imaging (block 108). Such a determination may be based on, for example, the drying time of the slide. For example, different methods of coverslipping exist and each one requires different drying times. Representatively, a glass cover slip may require about a day to dry while a film cover slip may dry in about an hour. In this aspect, coverslipped slides that are not ready (e.g., not dry) for further processing (e.g., imaging) are transported to a storage module to give them additional time to dry (bock 112). Slides that are dry are determined to be ready for imaging.
Process 100 further includes determining whether a first imager is available for imaging (block 110). In one embodiment, a first imager is an imager operable to capture an image of an entire area of a specimen on a slide in a single specimen plane. One such imager is a slide scanner that provides high speed brightfield slide scanning. Imaging of a specimen on a slide typically takes longer than the amount of time it takes to stain, coverslip and dry the slide, because imaging must be done on individual slides (i.e., one at a time) while staining and coverslipping operations may be done on a number of slides at the same time (e.g., staining a batch of slides). For example, a first imager that is a slide scanner can perform a 20× scan of a 15×15 mm tissue in about two to three minutes. Higher resolution can double that time. This equates to an imager throughput of from about 10-24 slides per hour. In contrast, up to about 500 slides per hour may be processed through a coverslipper and/or stainer. As a result, the imager is often times not ready to image each of the slides as they exit the coverslipper and/or stainer. If the imager is not available, the slides are transported from the coverslipper to a storage module for storing until a first imager is available (block 112).
Once a first imager is available, the slide is transported to the first imager for imaging (block 114). At the first imager, a digital image of the specimen is captured and stored in a computer memory (block 115). In one embodiment, the first imager is a slide scanner that is operable to scan an image of a specimen on a slide and capture and store the scanned image in a computer memory. In one embodiment, process 100 directs the first imager to scan an image of a specimen in a single plane (a single z-direction plane) at a selected magnification (e.g., 2.5×, 5×, 10×, 20×, 40×, 63×). In another embodiment, process 100 directs the first imager to scan an image of a specimen in a single plane at multiple magnifications (e.g., a first magnification of 20× and a second magnification of 40×).
Following the capturing of an image or images of a specimen on the slide by the first imager, the slide may be transported to the storage module (block 116) or to a second imager. In one embodiment, where specimens on a number of slides are processed together and images captured, examination of any one specimen image may take place at a time after the image has been captured. Until such examination, the slide is sent to the storage module.
After an image or images of a specimen, or group of specimens, is captured by the first imager and stored, the image(s) may be examined and the data may be made available to a diagnostician and/or an optional interpretation module which automatically interprets the data (block 117). As used herein, “diagnostician” refers to any person who may wish to view image data, such as pathologists, surgeons, nurses, researchers, technicians and administrators.
The image data is made available for access by a diagnostician if desired, and optionally the diagnostician is notified by electronic notification, such as by an e-mail, computer screen pop-up announcement, banner announcement, pager message or automated phone call. In other embodiments, the image data may also be accessed, or otherwise made available, to an optional interpretation module. In the case of the optional interpretation module, the interpretation module may conduct digital processing, such as by using pattern recognition technology in order to develop a preliminary diagnosis, and generate instructions or recommendations for additional processing if necessary.
In an embodiment, where specimens on a number of slides are processed and an image of each specimen is captured for examination, such examination of the images captured by the first imager may reveal that one or more of the specimens do not present an indication for more detailed analysis that may include, for example, further imaging (e.g., further image capture). As noted above, image capture of a portion of a specimen on a slide (an entire area portion of a specimen or less than an entire area portion) consumes a significant amount of time relative to other tissue processing steps. In one embodiment, the method enhances a processing and analysis time of multiple slides by separating out slides with specimens that do not show an indication of requiring additional processing from those that do show such an indication. Representatively, by capturing an image (e.g., a scan of an image in a single plane), and analyzing (examining) only that image for diagnosis minimizes the analysis time relative to, for example, capturing multiple images (e.g., images taken at different focal distances or in different planes of a specimen).
For those slides with specimens that show an indication for additional processing, the additional processing may include collecting additional biological samples, or performing further processing on samples already collected such as running additional or different test procedures or staining protocols. According to one embodiment, the additional processing includes additional imaging of a specimen. In this embodiment, process 100 is notified of a slide(s) containing specimens that are to receive additional processing by, for example, a signal from the interpretation module or by a diagnostician that has reviewed the slide(s) or queries the interpretation module or diagnostician regarding whether additional imaging is to be performed (block 118).
If additional imaging of a specimen on a slide is indicated or requested, process 100 queries whether a second imager is available (block 120). If a second imager is not available, the process maintains the slide containing the specimen in the storage module if the slide is present therein or directs the transport of the slide to the storage module (block 121). If a second imager is available, the process transports the slide to the available second imager (block 122). At the second imager, the process directs the obtaining of an additional image(s) (block 123). In one embodiment, process 100 directs multiple images of a portion of the specimen to be taken at different focal distances to provide a z-stack of the portion of the specimen or a composite image of the portion of the specimens with a greater depth of field (i.e., a thickness of the plane of focus) than an individual scan of a single plane. In another embodiment, additional images alternatively or additionally include higher magnification or higher resolution images than an image captured by the first imager.
In one embodiment, the second imagers is a digital microscope such as VISIONTEK® digital microscope, commercially available from Sakura Finetek USA, Inc. VISIONTEK® incorporates a multiple camera, brightfield optical system configured to view multiple slides simultaneously or view multiple areas of the same slide at different magnifications (e.g., 2.5×, 5×, 10×, 20×, 40×, 63×).
In one embodiment, the second imager captures multiple images of a portion of a sample and stores such images in a computer memory for review and analysis by the system or a diagnostician. In one embodiment, once the images are stored, the system or diagnostician is notified and the system or diagnostician can review the stored images. In another embodiment, a diagnostician may be notified when a slide is present in the second imager before images are stored. Where the second imager is a VISIONTEK®, the microscope offers a “live” view mode to allow a diagnostician to review a portion of a specimen without storing (saving) the image. Thus, the diagnostician may view a portion of the image at different focal distances, different resolutions (e.g., zoom in/out) and/or different magnifications without saving an image using the VISIONTEK® to analyze a specimen.
Currently, the VISIONTEK® permits the viewing of up to four slides simultaneously. In certain instances, it may be desirable to view or make available for viewing more than four slides. For example, in certain instances, more than four slides may be available for a certain patient case (e.g., seven slides, nine slides, 12 slides, etc.) and a diagnostician (e.g., a pathologist) may want to view all of the slides together. In one embodiment, a system includes multiple digital microscopes. Where each of the multiple digital microscopes is a VISIONTEK®, each microscope can accommodate four slides at one time so a diagnostician can view (examine) slide multiples of four together. In another embodiment, where a microscope is capable of loading and possibly viewing more than four slides simultaneously, for example, 12 slides (see, e.g., U.S. patent application Ser. No. 14/138,740, titled “Microscope with Slide Clamp Assembly,” filed Dec. 23, 2013, incorporated herein by reference), the seven slides, nine slides, or even 12 slides of a patient case can be loaded and viewed together on a single digital microscope.
Examination, imaging and interpretation of a specimen may be continued until the system or diagnostician deems it to be complete. These repeated tests and examinations are referred to herein as iterative processing, testing or examination. In another aspect of the invention, the diagnostician may access reports that are based on the comparison data created by the interpretation module. In a further aspect of the invention, the diagnostician may order or conduct further iterative processing, testing or examination.
In the illustrated examples, the material pathways represent examples of transport paths that may be traveled by a physical sample in a laboratory or hospital. A typical progression of the material from one station or system component to the next is depicted by the direction of the arrow. However, it should be understood that the processing stations are provided as examples, as are the directions of material flow. It shall be appreciated that more, fewer or other processing stations may be used in practice of the present invention, and/or more, fewer or other material paths and directions may be used. In addition, the stations may be in any order and any orientation (e.g. vertically stacked or side by side).
Any form of transport may be used that is sufficient to automatically transport the material as indicated by the material pathways. For example, material may be transported by a robotic device from one station to the next as will be discussed in more detail in reference to
In the embodiment of
At laboratory receiving, information about the sample may be read, such as by a bar code reader reading a bar code associated with an associated sample container. The bar code reader or a processor associated with the bar code reader, in one embodiment, is linked or can be linked to the database containing information about the sample, such as a database associated with a hospital information system 270 and/or laboratory information system 280. In one embodiment, the database finds the record of the shipment and displays a list of the expected contents. In an embodiment where information about the sample is printed on a label, a technician receiving the sample at laboratory receiving may enter the information into a database that, in one embodiment, is linked to hospital information system 270 or laboratory information system 280. In one embodiment, a sample identifier such as a bar code may be added to the container containing the samples.
Following receipt and acknowledgement of the shipment of the sample, the sample may be presented to grossing module 201. At grossing module 201, in one embodiment, any sample identifier (e.g., barcode) is shown to a reader and the reader or a processor associated with the reader is linked to or can be linked to the database containing information about the sample. A screen associated with the reader or processor displays a list of how the sample is to be prepared for the requested test(s). In another embodiment, rather than being linked to a database, paper documentation may be included with the sample indicating the requested test(s). The sample may be divided into the needed number of vials or cassettes, as the case may be. For non-fluid specimens such as tissue, processing may be include treatment with a series of reagents followed by infiltration with paraffin. In one embodiment, any division or preparation of the sample may be directed by the sender or a protocol, such as a protocol associated with hospital information system 270 or laboratory information system 280. In another embodiment, the technician at grossing module 201 makes a determination of any division or preparation of the sample based on his/her experience. Any vials or cassettes into which the sample or a portion of the sample is placed may be individually labeled with an identifier (e.g., a barcode), and the identifier and its association with a vial or cassette may be uploaded into the database.
From grossing module 201, the sample(s) are transferred to microtome module 205. At microtome module 205, the process of showing an identifier (e.g., a bar code) to a reader may be repeated. In one embodiment, the reader or a processor associated with the reader is linked to the database containing information about the sample. A screen associated with the reader or processor displays a list of instructions of how the sample is to be prepared for the requested test(s). In another embodiment, rather than being linked to a database, paper documentation may be included with the sample indicating the requested test(s). The sample may be embedded in paraffin and sectioned into a number of sections each placed on a slide for further processing. In one embodiment, any division or preparation of the sample may be directed by the sender or a protocol, such as a protocol associated with hospital information system 270 or laboratory information system 280. In another embodiment, the technician at microtome module 205 makes a determination of any sections of the sample based on his/her experience.
Specimen sections created in the microtome station are individually positioned on a microscope slide. A specimen mounted on a microscope slide is transported to staining module 210. Slides requiring deparaffinization may be placed in an oven prior to staining, or placed directly in the stainer if the stainer is equipped with a built-in oven or can perform a chemical deparrafinization step.
Any staining or other test protocol may be performed by the staining module 210 as desired. In one embodiment, an automated stainer is used. In an example, staining with hematoxylin and eosin (“H & E”) is performed in staining module 210. Other staining methods such as special stains (SS), immunohistochemistry (IHC), and in situ hybridization (ISH) or fluorescent in situ hybridization (FISH) can also be performed.
In one embodiment, following staining, the samples may be transported along material path 217 to coverslipper module 220 to be coverslipped.
After staining and/or coverslipping, the slide may proceed to first imager module 230 or storage module 202. In some embodiments, where it is desirable for imaging of the slide to be delayed, the slide is transported to storage module 202 for storage until imaging is desired. Representatively, different methods of coverslipping exist and each one requires different drying times. Coverslipped slides that are not ready (e.g. not dry) for further processing are transported to storage module 202 along material path 203. Once the slides are ready, they may then be transported along material path 205 to first imager module 230. In this aspect, the differences in drying times from slide to slide are automatically resolved by the automated system.
In some embodiments, the laboratory can select delay criteria based on the coverslipping technique used and the type of sample (histology v. cytology, monolayer slides v. smears, etc.). For example, the laboratory may determine, based on the coverslipping technique to be used and the type of sample on the slide, that the slide should be stored for a period of time prior to imaging. This information may be contained in an identifier associated with the slide. The automated system may read the identifier and follow the assigned processing protocol. In this aspect, after coverslipping, the slide is transported to storage module 202 and stored for the predetermined period of time. After such time, the system may alert the transport module to retrieve the slide from storage module 202 and transport the slide to first imager module 230 for imaging. In one embodiment, first imager module 230 includes one or more imagers, each imager operable to or configured to capture an image of an entire area of a specimen on a slide in a single plane. A selection of the plane or z-section of the sample may be made by directed by protocol associated with hospital information system 270 or laboratory information system 280. An identifier associated with the slide may be read a first imager module 230 and be linked to the protocol. In another embodiment, a selection of the plane of the sample to image may be determined by a diagnostician responsible for analyzing the slide. In one embodiment, an imager at first imager module 230 is a slide scanner that is operable to capture an image of the specimen and store that image in memory or direct the image to a memory unit or module associated with the system.
In addition to drying times, the availability of an imager at first imager module 230 may further delay imaging. In particular, imaging of a specimen on a slide typically takes longer than the amount of time it takes to stain, coverslip and dry the slide. As a result, an imager at first imager module 230, in certain instances, cannot be ready to image each of the slides as they exit the stainer/coverslipper modules. The identifier associated with the slide may store information regarding the desired imaging protocol for the slide. Upon reading the identifier, the system schedules imaging of the slide with an imager at the first imager module 230 capable of imaging according to a requested protocol. If the desired imager is not available when the slide is otherwise ready for imaging, the slide is transported from staining module 210 and/or coverslipper module 227 along material path 203 to storage module 202 for storing until an imager at first imager module 230 is available.
It is further contemplated that after a specimen is imaged by first imager module 230, the specimen slide may be transported along material path 205 to storage module 202. The slide may be stored in storage module 202 for future testing and/or examination.
Once the specimen is ready for imaging, at least one image of the material specimen is obtained by first imager module 230. In one embodiment, an imaging protocol for each slide which is to be followed dictates one image (e.g., one scan) of a portion (e.g., an entire xy area) of a specimen on a slide in a single z plane). In another embodiment, the imaging protocol for each slide which is to be followed by an imager at first imager module 230 may be flexible and can be defined at any time by, for example, the diagnostician (e.g., pathologist). For example, a diagnostician may examine an image and determine that additional images of the slide are necessary. Representatively, the diagnostician may determine that images at a different magnification are necessary or that an imager should focus deeper into a tissue area (a different z plane). According to the automated system disclosed herein, the pathologist may instruct the system to obtain further images.
First imager module 230 may include one or more imagers. First imager module 230 can be any system that generates images that can be interpreted manually interpreted or, optionally, automatically interpreted by interpretation module 290. In one embodiment, first imager module 230 includes a scanner capable of recording digital images of the microscope's view field. For example an optical charge coupled device (CCD) based camera can be used to generate the digital image data. The digital image data can be stored in any fashion that provides for access to the data as required by interpretation module 290, diagnostician workstation 243 and/or technician workstation 250 and/or as desired by anyone needing access to the image data, such as diagnosticians or laboratory personnel. Examples of suitable data storage are local storage devices associated with first imager module 230 (such as hard drive, removable memory, flash memory, optical memory such as CD or DVD etc.), and/or networked memory such as diagrammatically illustrated by data storage 260. It should be noted that any form of information may be generated by first imager module 230, in addition to the image data. For example, first imager module 230 may optionally associate other types of data, such as a log of patient information associated with the image data and as discussed further herein. Alternatively, another processing system may associate the image data with other data.
In one embodiment, the type of information generated is intended to be sufficient for a diagnostician (e.g., a pathologist) or interpretation module 290 to perform interpretation processing and generate a desired report. Interpretation module 290 may take any desired form, such as for example, a dedicated computing system, or alternatively it may be a module running on a computing system used for multiple purposes. In additional examples, it may be freestanding, a part of first imager module 230, part of hospital information system 270, part of laboratory information system 280, or it may be in any location where data may be received from first imager module 230. Although the figure depicts a single interpretation module 290, it should be understood that plural interpretation modules 290 also may be used. In further examples, diagnostician workstations 243 may include interpretation modules 290 or interpretation module clients that enable the diagnostician to locally conduct an interpretation based on the data available including, without limitation, image data from first imager module 230.
In the embodiment of
Following imaging by an imager at first imager module 230, interpretation module 290 may be configured to determine if the imaged specimen should go to storage module 202, or interpretation module 290 may be configured to determine whether the particular specimen needs to undergo additional processing, such as that it should go to a diagnostician or other personnel for further inspection or additional image capture, in which case the slide containing the specimen proceeds along material pathway 237 to second imager module 240.
In another embodiment, rather than a computer- or machine-automated analysis of a specimen image on a slide, a diagnostician analyzes the specimen image(s) provided by first imager module 230 and makes a determination whether the specimen needs to or should undergo additional processing. If the diagnostician does not believe additional processing is necessary, the slide containing the specimen may remain in storage module 202, if there, or is directed to storage module 202. If the diagnostician believes additional processing is necessary, the slide containing the specimen may be sent to second imager module 240.
If additional imaging of a specimen on a slide is indicated or requested either through interpretation module 290 or a diagnostician, the slide containing the specimen is sent to second imager module 240. Second imager module 240 includes one or more microscopes. In one embodiment, the second imagers is a digital microscope such as VISIONTEK® digital microscope, commercially available from Sakura Finetek U.S.A., Inc. VISIONTEK® incorporates a multiple camera, brightfield optical system configured to view multiple slides simultaneously or view multiple areas of the same slide at different magnifications (e.g., 2.5×, 10×, 20×, 40×). In one embodiment, a second imager at second imager module 240 in operable or configured to automatically capture (obtain) additional image(s) of the specimen on the slide. In one embodiment, a second imager is operable or configured to automatically capture multiple images of a portion of the specimen are taken at different focal distances to provide a z-stack of the portion of the specimen or a composite image of the portion of the specimens with a greater depth of field (i.e., a thickness of the plane of focus) than an individual scan of a single plane. In another embodiment, a second imager is operable or configured to alternatively or additionally automatically capture higher magnification or higher resolution images than an image captured by the first imager.
In one embodiment, the second imager captures multiple images of a portion of a specimen and stores such images in a computer memory for review and analysis by the system or a diagnostician. In one embodiment, once the images are stored, the system or diagnostician is notified and the system or diagnostician can review the stored images. In another embodiment, a diagnostician may be notified when a slide is present in the second imager before images are stored. Where the second imager is a VISIONTEK®, the microscope offers a “live” view mode to allow a diagnostician to review a portion of a specimen without storing (saving) the image. The diagnostician may view a portion of the image at different focal distances, different resolutions (e.g., zoom in/out) and/or different magnifications without saving an image using the VISIONTEK® to analyze a specimen. In one embodiment, as an alternative to storing additional images or in addition to storing additional images, where a diagnostician is available at second imager module 240 at the time the slide containing the specimen is present, the diagnostician may examine the slide and decide images to be captured and stored (saved).
In certain instances, it may be desirable to view or make available for viewing more one slide at a time. In one embodiment, second imager module 240 includes an imager than can contain more than one slide so that specimens on different slides can be displayed simultaneously. Currently, the VISIONTEK® permits the viewing of up to four slides simultaneously. In another embodiment, second imager module 240 can include more than one imager, including more than one imager than can contain more than one slide. For example, in certain instances, more than four slides may be available for a certain patient case (e.g., seven slides, nine slides, 12 slides, etc.) and a diagnostician (e.g., a pathologist) may want to view all of the slides together. In one embodiment, a system includes multiple digital microscopes. Where each of the multiple digital microscopes is a VISIONTEK®, each microscope can accommodate four slides at one time so a diagnostician can view (examine) slide multiples of four together.
In one embodiment, a diagnostician makes an analysis of a specimen based on images captured at second imager module 240. Once images are captured and stored at second imager module 240, the slide containing the specimen may be sent to storage module 202.
In one embodiment, interpretation module 290 is configured or operable to determine if further processing of a sample represented by the imaged specimen is required. In such case, a new specimen of the sample is necessary for interpretation. In one embodiment, additional section(s) (specimens) of the same same may have been placed on slide(s) and those slide(s) sent to storage module 202 with a label that links the slide(s) to the imaged sample. In this embodiment, the slide(s) are not stained or coverslipped, and are kept aside. These slides could be identified as being extra sections that should not be stained and kept in the storage area until called back for staining and coverslipping. For example, these extra slide(s) may have the same identification information as the original or primary, perhaps with an additional indicator (e.g., an additional letter or number) to indicate the slide(s) are extra slide(s). If they are not needed, these slides can be discarded after, for example, a user-defined period of time has elapsed or the case has been completed and signed off. Extra sections are cut and extra slides prepared and stained only when there is a requirement for more staining protocols. In an automated handling system that also includes handling of tissue blocks, the request for more staining would be transferred to microtomy module 205. In one embodiment, a tissue block, including a formalin-fixed tissue section in a paraffin block from which another tissue section may have been taken and placed on a slide, includes an identification tag such as a bar code or RFID tag. In response to a signal from a controller, the tissue block is retrieved and transported automatically from a storage module (e.g., storage module 202) to microtomy module 205. The tissue block is stored and may be retrieved by the identification tag. The tissue block would be forwarded to the microtomy area for more sections to be taken.
Representatively, once a new sample is placed on a slide, the new sample proceeds to stainer module 210 where it may undergo operations such as special staining, immunohistochemistry (“IHC”), in situ hybridization (“ISH”), multiplexing or other staining or testing procedures. Subsequently, the new sample may proceed along the material path, for example, back to first imager module 230 or directly to second imager module 240. Ultimately it is desired that a tested and imaged specimen be stored as indicated by storage module 202. In this example, after inspection by a diagnostician or other person, the original specimen may be designated for storage, such as in storage module 202 and the new specimen from the same tissue section designated and further processed. The new specimen may be processed and inspected and sent to storage. The original specimen and the new specimen are linked by an identification tag. Later, either or both the original and the new specimen can optionally be retrieved from the storage module 202, if desired.
The workstations, such as diagnostician workstations 243 or other workstations, such as technician workstations 250 can have any desired structure, including computing systems serving as controllers in communication via communications infrastructure 200 with other processing stations or components of the system. The workstations may optionally also include other components that might be useful in a work area, such as material storage units, furniture, phones etc. In an embodiment, the workstations 243, 250 provide access to information concerning the processing of biological samples, and the results of the processing, including image data from the first imager module 230 and interpretation data or reports from the interpretation module 290. Technician workstation 250 may be in communication with data storage 260 via path 257. In another embodiment, a system may not include workstations such as diagnostician workstations 243 and/or technician workstations 250.
As the material proceeds along the material pathways and through the processing systems, information may be shared between the numerous devices using various information pathways that form communications infrastructure 200. It should be noted that communications infrastructure 200 may be any form of communication system enabling communications between and amongst individuals, computer systems and/or automated processing systems. Representatively, the communications infrastructure may be a computer network that is wired, wireless or a combination of wired and wireless. For example, information access points may be wired into the network and/or joined to the network via a wireless portal. Although the illustrated example shows a networked system in which communications are performed via a network, direct communications also may be conducted. For example in one embodiment, staining module 210 may have a direct communications link with coverslipper module 220 and may access the communications network via a node in coverslipper module 220, or alternatively it may have a direct network link. It should be understood that any suitable communications pathway structure is envisioned which would enable suitable sharing of information between and amongst various stations. Likewise, it should be understood that, in other embodiments, not all of the stations may have a direct communications path. Furthermore, it should be understood that the communication pathways can take any form, such as digital, analog, wired, wireless, paper, oral, telephonic, etc.
In one embodiment, a laboratory network may be provided as the portion of the communications infrastructure 200 between and amongst the laboratory instruments or modules, depicted with reference numbers 210, 220, 230, 240, 202 and also laboratory information system 280 and other workstations 243 and 250 (which might include a computer system such as for example one or more personal computers and/or computer servers). The laboratory network may be networked with a hospital network that is also a part of communications infrastructure 200. In such an embodiment, other devices may have access to the information available on laboratory information system 280 or other laboratory devices via the communications infrastructure 200. Such other devices include for example, diagnostician or administrator workstations 243, hospital information system 270, and in some embodiments interpretation module 290 as well. It should be understood that the flexibility of the information pathways is directed to enable the necessary information flow to track biological samples being processed however desired, and to distribute the necessary information to the appropriate users. Numerous alternative communications system structures may be selected to meet this need, and the illustrated and discussed examples are provided for illustrative purposes only, not to limit the scope or flexibility of the system.
Referring to the illustrated example, communications pathways 203, 205, 207, 215, 225, 235, 245, 255, 265, 275, 285, 295, represent examples of communications pathways between staining module 210, coverslipper module 220, first imager module 230, storage module 202, diagnostician workstation 243, technician workstation 250, local or remote data storage 260 and/or hospital information system 270, laboratory information system 280, interpretation module 290, or any other desired station or component of the system.
The sharing of information may be automated, manual or conceptual. For example, information may be shared directly by two machines in communication with each other, it may be made available to a user who can manually input it into another device, or a single machine comprising more than one device shown in
In another embodiment, biological specimens, slides, trays, containers, workpieces, and locations throughout the system may be identified with machine understandable codes, such as provided by RFID tags, shape identifiers, color identifiers, numbers or words, other optical codes, barcodes etc. The identifiers can be recorded to generate data provided to a database, such as data maintained in data storage device 260, by a processor (any computing devices), hospital information system 270, laboratory information system 280 or any combination thereof. Examples of data that may be tracked include patient information and history, information regarding biological sample(s) collected, arrival and departure times of biological samples, tests performed on the specimens, processes performed on the specimens, reagents applied to the specimens, diagnoses made, associated images and so on.
In one embodiment, transport module 302 may be a robotic device capable of transporting a slide between stations. In one embodiment, transport module 302 may be an X-Y-Z robotic device dimensioned to transport one or more slides between stations. Representatively, transport module 302 may be a track and elevator system. The track system may be a conveyor belt or plate system that transports the slide horizontally in an “x-” direction. In this aspect, one or more slides may be placed on the conveyor and conveyed between the desired stations, for example, between coverslipper module 306, imager 308 and storage module 322. In one embodiment, the conveyor belt system may have two separate conveyor belts such that one conveyor belt transfers the slide in one direction and the other conveyor belt transfers the slide in the opposite direction as illustrated by arrow 316. Alternatively, as described with reference to
Staining module 304 and coverslipper module 306 may be an integrated slide stainer and coverslipping system. Alternatively, staining module 304 and coverslipper module 306 may be in separate instruments at different locations. In the case of an integrated system, staining module 304 and coverslipper module 306 may be a staining/coverslipping system such as the TISSUE-TEK® PRISMA® and TISSUE-TEK® GLAS™g2 combo system or TISSUE-TEK® PRISMA® and TISSUE-TEK® FILM® combo system commercially available from Sakura Finetek U.S.A., Inc., Torrance, Calif. In one embodiment, staining module 304 may have hematoxylin and eosin stain (H&E) and special staining (SS) capabilities. At H&E/SS staining and coverslipping, a biological sample may undergo H&E or SS staining and optional coverslipping. Other staining or testing protocols also can be performed.
During operation, an individual slide or group of slides placed in a basket may be loaded into staining module 304 and stained according to a desired staining protocol. In the case of a group of slides, the staining protocol can be the same for all slides or selected from a staining protocol menu, either by an operator or automatically by reading a bar code, an RFID or any other protocol identification device. Once the staining protocol is complete, the slide or group of slides within the basket is automatically transferred to coverslipper module 306 for individual cover slipping. The identifier associated with each slide is then read as the slides are coverslipped and either placed as a group in a basket or individually fed onto transport module 302.
In an alternative embodiment, where a group of slides are stained together, the slides may be singulated (separated from the group) in staining module 304 and placed on transport module 316. For example, where a group of slides are stained together in a basket, a pick and place robotic device in staining module 304 may transfer the slides individually to transport module 316. From transport module 316, the slides may be conveyed to coverslipper module 306, or, without a coverslip, to one first imager module 307 or to storage module 322.
In one embodiment, first imager module 307 includes more than one imager (e.g., imager 308, imager 310 and imager 312 shown). Imaging methods (quick-scan, 20×, 40×, etc.) at first imager module 307 can be pre-assigned to a specimen on a slide according to a laboratory default or specific instructions from, for example, a diagnostician. In the case of basket-grouped slides, in one embodiment, each of the slides would be assigned the same scanning method(s). Individual slides or the basket of slides may be assigned to one of imagers 308, 310, 312 based on the imagers availability or according to laboratory defined rules, such as dedicating one or more imagers to a specific scanning method (e.g., quick-scan, 20×, 40×) or a plurality of methods.
In one embodiment, a slide including a biological sample is individually transported by transport module 302 to one of imagers 308, 310, 312 of first imager module 307 and/or storage module 322. If the slide is ready for imaging (e.g., dry), the system checks to see if, for example, imager 308 is available. Imager 308 is determined to be available if, for example, it is properly functioning and not currently imaging another sample on a slide. If imager 308 is not available, the availability of imager 310 is determined. If imager 310 is not available, the availability of imager 312 is determined. This process continues, until an available imager is found. Alternatively, an imaging schedule between the slide and a particular imager may be predetermined. Representatively, information relating to a period of time sufficient to allow the slide to dry may be assigned to the slide and imagers 308, 310 and 312 may be on an imaging schedule. The system may determine which imager will be available after the drying period expires. Once an available imager is determined, the slide is transported by transport module 302 to the available imager. Although three imagers are illustrated in
If none of imagers 308, 310, 312 of first imager module 307 are available or there are other conditions which require delay in imaging (e.g., waiting for slide processing instructions), transport module 302 transports the slide to storage module 322. The slide remains in storage module 322 until one of imagers 308, 310, 312 become available and/or processing instructions are received. Once an imager is determined to be available, the slide is transferred from storage module 322 to transport module 302 using, for example a robotic device, and transported by transport module 302 to the available imager for imaging. Upon completion of imaging, the slide may be transported by transport module 302 from imager 308, 310 or 312 to storage module 322. The image may be communicated to a diagnostician, for example a diagnostician, for immediate examination. Via a computer (e.g., a personal computer), the diagnostician can then examine an image of a sample on a slide for viewing and recall a slide for more imaging work if desired. Alternatively, if it is determined that no further examination of the slide is desired, the slide may be removed from the storage module 322. Alternatively, an interpretation module (not shown) may automatically examine a specimen on a slide using, for example, pattern recognition software, and make a determination whether a slide should be recalled for additional imaging work.
If additional imaging of a specimen on a slide is indicated or requested either through an interpretation module or a diagnostician, the slide containing the specimen is transported via transport module 302 from, for example, storage module 322 to second imager module 315. Second imager module 315 includes one or more imagers or microscopes (e.g., imager 316, imager 318, imager 320). In one embodiment, the one or more second imagers are a digital microscope such as VISIONTEK® digital microscope, commercially available from Sakura Finetek USA, Inc. In one embodiment, a second imager (e.g., imager 316, imager 318 or imager 320) is operable or configured to automatically capture multiple images of a portion of the specimen are taken at different focal distances to provide a z-stack of the portion of the specimen or a composite image of the portion of the specimens with a greater depth of field (i.e., a thickness of the plane of focus) than an individual scan of a single plane. In another embodiment, a second imager is operable or configured to alternatively or additionally automatically capture higher magnification or higher resolution images than an image captured by the first imager.
In one embodiment, a second imager (e.g., imager 316, imager 318 or imager 320) captures multiple images of a portion of a specimen and stores such images in a computer memory for review and analysis by the system or a diagnostician. In one embodiment, once the images are stored, the system or diagnostician is notified and the system or diagnostician can review the stored images. In another embodiment, a diagnostician may be notified when a slide is present in the second imager before images are stored. Where the second imager is a VISIONTEK®, the microscope offers a “live” view mode to allow a diagnostician to review a portion of a specimen without storing (saving) the image. The diagnostician may view a portion of the image at different focal distances, different resolutions (e.g., zoom in/out) and/or different magnifications without saving an image using the VISIONTEK® to analyze a specimen. In one embodiment, as an alternative to storing additional images or in addition to storing additional images, where a diagnostician is available at second imager module 315 at the time the slide containing the specimen is present, the diagnostician may examine the slide and decide images to be captured and stored (saved).
In certain instances, it may be desirable to view or make available for viewing more one slide at a time. In one embodiment, multiple imagers (e.g., imager 316, imager 318 and/or imager 320) at second imager module 315 are used at the same time to each contain one or more slides related, for example, to a single patient case. In this manner, specimens on different slides can be displayed simultaneously.
In one embodiment, storage module 322 may include more than one storage module. In this aspect, one or more of the storage modules may act as short-term storage areas for slides likely to need more imaging work. In addition, one or more of the storage modules may act as long-term storage areas for slides which are unlikely to require more imaging work in the near future. The long-term storage modules may be located within the laboratory or remotely.
In one embodiment, storage module 322 is configured to group slides (and tissue blocks for the block storage system) according to user-defined criteria. For example, slides pertaining to a patient case could be placed in the same area. Then cases or blocks can be located by date of production, by physician, by provenance, or by a combination of these criteria. Representatively, as noted above, a slide may contain an identifier that may be read by a reader (e.g., RFID reader, bar code reader). That identifier (e.g., RFID, bar code) may contain information (e.g., letters, numbers and/or symbols) indicating a date of production, a physician and/or a provenance. When the information is read by a reader, the information may be sent to a controller or to other devices through the communications infrastructure.
Automated system 300 as illustrated in
In order to automate the movement of the loading container in the coverslipper module 306 between coverslipper module 306 and stainer module 304, software instructions and a data link between coverslipper module 306 and stainer module 304 are provided. Such instructions and link may be solely between coverslipper module 306 and stainer module 304. Alternatively, a control system may be connected to each of strainer module 304, coverslipper module 306, first imagers 308, 310, 312 of first imager module 307, second imagers 316, 318, 320 of second imager module 315, storage module 322 and transport module 302 that may be used to transport a slide between imagers and the modules.
Referring again to movement of a loading container from stainer module 304 to coverslipper module 306, the loading container sits on a plate that is connected to wires that move the plate and the loading container in an x- and y-direction, respectively, by two-step motors. The plate may move the loading container in an x-direction into the stainer.
In operation, a transfer arm of stainer module 304 retrieves a rack of slides and moves the rack along an xy axis to one or more individual staining stations. The transfer arm transfers a rack of slides to an appropriate staining station and then lowers the rack into that staining station for staining (a z-direction). Following staining, the transfer arm removes the rack of slides from the staining station and moves in x- and y-direction to another staining station or, when all staining operations are complete, to a transfer station where the rack of slides is to be transferred from the stainer to the Film® coverslipper module (coverslipper module 306).
For a transfer operation between stainer module 304 and coverslipper module 306, the loading container in coverslipper module 306 receives instructions to move from coverslipper module 306 to stainer module 304 through the adjacent doorways in each device. The loading container is moved by the plate on which it sits along a single plane (xy plane) from the coverslipper to a position inside the stainer adjacent the doorway of the stainer module. Once inside the stainer module, the transfer arm lowers the rack of slides into the loading container. The loading container typically contains a solution such as xylene that wets the slides. The loading container then moves on the x-direction plate from the stainer into the coverslipper again through the adjacent doorways. A cover slipping operation including placing a film-type cover slip on individual slides in the basket of slides is then performed in the coverslipper.
Transport module 302 may be a robotic device capable of transporting a slide between stations. In the embodiment shown in
Attached to the pallet support members are pallets. The pallets are designed to overlap one another and are secured to the pallet support members to form a flexible surface. The overlap configuration of the pallets allows them to slide relative to each other as the pallets travel around the corners of the tracks. The leading edge of the pallets are secured to the support members by fasteners. Each of the pallets may have a slight bend to negotiate the curves in the unit.
In the embodiment shown in
As noted earlier, in this embodiment, multiple slides are brought to converslipper module 306 from stainer module 304 in a rack. In coverslipper module 306, the slides are singulated (separated from other slides in a rack) for coverslipping. In one embodiment, all stained slides in coverslipper module 306 are coverslipped. In another embodiment, a coverslipping operation may be bypassed. Such bypass can occur at the singulation point in coverslipper module 306. According to this embodiment, a slide is singulated and either directed to be directly discharged onto conveyor 402 or to be coverslipped and then discharged.
In one embodiment, a slide retaining device is positioned adjacent to or connected to conveyor 402. Slide retaining device 420, in one embodiment, is an oval-shaped chain or belt (e.g., a continuous loop) having projections 422 extending outwardly therefrom. Projections 422 are spaced from one another at approximately a width of a slide.
As shown in
Slide retaining device 420 is rotated by a pulley and moves at the same rate as conveyor 402.
As illustrated in
Conveyor 402 transports a slide to first imagers 308, 310, 312 of first imager module 307. Imagers 308, 310, 312 are, for example, scanners and may further each contain a reader (e.g., RFID reader, bar code reader) connected with controller 400 to read an identifier on a slide indicate to controller 400 that a slide is in the imager and to associate a digital image with the identifier. In one embodiment, conveyor 402 stops at each imager and controller 400 assesses the availability of the imager (e.g., receives a signal that indicates whether or not an imager is available). If an imager is available and control system (e.g., controller 400) determines that a slide may be imaged at this time (e.g., the slide is dry), the slide is placed in the imager.
In one embodiment, a slide is placed in an imager by applying a pushing force to the slide. In this embodiment, associated with each first imager 308, 310, 312 and controlled by controller 400 is a plunger assembly.
Each plunger assembly 408, 410, 412 includes an actuator such as an electrical motor or air piston that drives a corresponding plunger to extend or retract. A plunger, when actuated, moves outward from the plunger assembly toward the respective imager. The plunger may be a bar or rod having a thickness equivalent to or greater than a thickness of a slide. Each plunger assembly is positioned adjacent conveyor 402 such that when a plunger is extended from a plunger assembly, the plunger will contact a surface of conveyor 402 or extend over conveyor 402 a slight distance (e.g., 0.1 to 0.25 mm). Plunger must be close enough to conveyor 402 that it is capable of contacting an edge of a slide on the conveyor and pushing the slide off conveyor 402 as it extends. To the extent a height of slide retaining device 420 would otherwise prevent a plunger form contacting an edge of a slide, plunger is made of a material having sufficient weight or density to deflect slide retaining device 420. For example, a plunger comprised of a steel bar or rod can be made of a sufficient weight to deflect slide retaining device 420 of a synthetic rubber belt downward. In another embodiment, a plunger may extend from a plunger assembly at an angle slightly less than horizontal (e.g., less than 5°) so that the plunger will deflect slide retaining device 420 to be at most parallel with a surface of conveyor 402.
If a slide is positioned in front of first imager module 307 (in front of one of first imagers 308, 310, 312) and the imager is available, the plunger will push the slide into imager. Thus, the plunger is oriented such that it will contact an edge of a slide on conveyor 402.
As noted above, in one embodiment, controller 400 is connected to stainer module 304, coverslipper module 306, first imagers 308, 310, 312 of first imager module 307, second imagers 316, 318, 320 of second imager module 315, plunger assemblies 408, 410, 412, 418, 421 corresponding plunger assemblies associated with each imager, storage module 322 and conveyor 402. In addition to optionally controlling a staining of slides in stainer module 304 and coverslipping slides in coverslipper module 306, controller 400 includes instructions (e.g., a computer program) for controlling a discharge of a slide from coverslipper module 306 or, optionally, stainer module 304 onto conveyor 402 and the movement of conveyor 402 to bring a slide to first imagers 308, 310, 312 or first imager module 307 and/or to second imagers 316, 318, 320 of second imager module 315.
To control discharging of a slide onto conveyor 402 from coverslipper module 306, controller 400 receives data from coverslipper module 306 whether a slide is ready for discharge. In one embodiment, this data is provided to controller 400 in the form of a signal when a slide is positioned in a designated area in coverslipper module 306. The slide may or may not have proceeded through a coverslip operation in coverslipper module 306. Controller 400 checks to see if a position on conveyor 402 is free to receive a slide. In the embodiment shown in
In one embodiment, control system stops conveyor 402 for a brief moment (e.g., three to five seconds) in periodic intervals each time a slide could be positioned in front of an imager (any of the first imagers or second imagers). Controller 402 receives a signal whether the imager is available for receiving a slide for imaging. It may receive this signal in an unsolicited fashion (e.g., a sensor associated with the sensor sends a signal whenever the imager is available) or it may solicit the signal (e.g., controller sends a signal to a sensor associated with the imager and receives a reply to the sent signal from the sensor). If a slide is present on conveyor 402 and an imager is available, control system will activate a corresponding plunger assembly to place a slide in the imager. Similarly, controller 400 checks when an imaging of a slide is complete and subsequently discharges the slide onto conveyor 402. In one embodiment, a sensor such as a photoelectric sensor may be associated with, including connected or adjacent to, each of plunger assembly 408, 410, 412 to sense whether a slide is present on conveyor 402 or conveyor 402 is free to receive a slide from one of first imagers 308, 310, 312 of first imager module 307 or one of second imagers 316, 318, 320 of second imager module.
Although plunger assemblies are described for transferring slides between conveyor 402 and imagers, it is contemplated that any other type of robot device suitable for transferring a slide between processing stations may be used. Representatively, a robotic arm capable of grasping slide 424 and transferring slides between imagers and conveyor 402 may be used. For example, in embodiments where a group of slides are transported within a basket, slides must be removed individually from the basket for imaging. In this aspect, a Gantry or Cartesian coordinate type robot, a selective compliant assembly robot arm (SCARA) type robot, an articulated arm type robot, or a combination thereof (e.g., a SCARA type robot coupled in a Gantry type robot configuration) may be used to retrieve and deposit individual slides within the basket.
In one embodiment described with reference to
In an embodiment where storage module 322 is a cabinet, storage module 322 may have a plurality of slide stations 602. Each of slide stations 602 may be dimensioned to receive and store a slide. In one embodiment, slide stations 602 are dimensioned to receive and store a slide individually or a group of slides. In the case of a group of slides, the slide group may be stored in slide stations 602 in a tray or basket. For example, a tray or basket holding 10 slides may be stored within one of slide stations 602. In this aspect, slide stations 602 are dimensioned to store the tray or basket having the sides therein.
In one embodiment, stations 602 may be formed in a grid pattern as illustrated in
Transport module 302 may include one or more elevator devices positioned in storage module 322 to place slides within or retrieve slides from slide stations 602 and transfer slides between slide stations 602 and conveyor 402.
Elevator device 614 may be used to move the slide vertically between conveyor 402 (a y-direction in reference to
Elevator device 614 may further include slide platform 622 positioned within lift member 620. Slide platform 622 may be movably coupled to lift member 620 such that it slides horizontally in an x-direction to eject the slide from or receive the slide within elevator device 614. Slide platform 622 is dimensioned to receive and retain slide 624 within lift member 620. In one embodiment, slide platform 622 may be a rectangularly shaped box having open ends and of a size configured to contain a single slide (e.g., 1 in.×1 in×3 in.). Slide platform 622 may be at least as wide as a width of the slide so that the slide may be positioned thereon. The slide may be inserted into and retrieved through either side of slide platform 622. Alternatively, slide platform 622 may be a planar member (a true platform) upon which the slide can be supported by slide platform 622.
Elevator device 614 may be used to transfer slide 624 between conveyor 402 and slide stations 602. Representatively, conveyor 402 may transport slide 624 from, for example, coverslipper module 306 or imagers 308, 310, 312, to slide stations 602. Conveyor 402 moves slide 624 horizontally in the x-direction until slide 624 is aligned with slide platform 622. In this aspect, lift member 620 moves vertically in the y-direction along frame member 618 until slide platform 622 is aligned with slide 624. Once slide platform 622 is aligned with slide 624, slide platform 622 moves in the x-direction toward conveyor 402 until it is positioned around slide 624. In one or more embodiments, slide platform 622 may include pincers, claws, jaws, hook-like structures or another gripping member. Slide platform 622 then moves in the opposite direction (i.e., away from conveyor 402) with slide 624 inside. Lift member 620 raises slide platform 622 having slide 624 therein until slide 624 is aligned with opening 428 of slide stations 602. Slide platform 622 then moves in the “X” direction toward slide stations 602 to insert slide 624 within the opening of the slide station. Once slide 624 is within the opening, slide platform 622 releases slide 624 and retracts (i.e. moves away from slide stations 602) thereby leaving slide 624 within slide stations 602 for storage.
Once storage is complete, elevator device 614 may be used to remove slide 624 from slide stations 602 and place it back on conveyor 402 for transport to, for example, first imager module 307 or second imager module 315.
Although elevator device 614 is described for transferring slide 624 between conveyor 402 and slide stations 602, it is contemplated that any other type of robot device suitable for transferring a slide between processing stations may be used. Representatively, a robotic arm capable of grasping slide 624 and transferring slide 624 between slide stations 602 and transport module 302 may be used. For example, in embodiments where a group of slides are transported within a basket, slides must be removed individually from the basket for imaging. In this aspect, a Gantry or Cartesian coordinate type robot, a selective compliant assembly robot arm (SCARA) type robot, an articulated arm type robot, or a combination thereof (e.g., a SCARA type robot coupled in a Gantry type robot configuration) may be used to retrieve and deposit individual slides within the basket.
As previously discussed, a slide may be inserted and stored in any of stations 602 which are positioned in a grid pattern. In this aspect, the robotic device for inserting and retrieving the slides must be able to move both vertically in the y-direction and horizontally in the x-direction. To store a slide or retrieve a slide stored in station 602A, lift member 620 of elevator device 614 moves vertically as illustrated by arrow 616 up to the top row (e.g. row A) of storage module 322. Frame member 618 then moves horizontally as illustrated by arrow 606 to the first column (e.g., column 1).
To store slide 424 within station 602A, slide platform 422 moves in the “z-” direction toward storage module 322 and inserts slide 424 within station 602A. Once slide 424 is positioned within station 602A, slide platform 622 moves in a direction away from storage module 322 leaving slide 624 behind within station 602A. To retrieve slide 624 from station 602A, slide platform 622 is inserted within station 602A and around slide 624. Movement of slide platform 622 away from station 602A pulls slide 624 out of station 602A and into elevator device 614. Lift member 620 of elevator device 614 may then be raised or lowered to transfer slide 624 to conveyor 402. Conveyor 402 may then be used to convey slide 624 to imager 308, 310, 312.
The identification, placement and retrieval of a slide within storage module 322 may be controlled by controller 400 that is electrically or communicatively linked to transport module 302. In one or more embodiments, movement or operation of transport module 302 may be based on signals exchanged between the controller and storage module 322. For example, in one embodiment, such a controller may receive a signal from coverslipper module 306 indicating that a slide is ready for storage. In response, the controller may signal transport module 302 to retrieve the slide from the coverslipper module and transfer the slide to storage module 322. A reader (e.g., an RFID or bar code reader) may be positioned at the entrance to storage module to read an identifier associated with the slide. This information is transmitted to controller 400. The controller may identify an open slide station within storage module 322 and signal transport module 302 to insert the slide within the open slide station. The slide location information may be stored by the system. In one embodiment, the slide location may be selected based on a criteria such as patient case, a physician or hospital, term of storage, etc. When retrieval of the slide is desired, for example where a pathologist instructs the system to perform further imaging of the slide, controller 400 may determine the location information of the desired slide and signal transport module 302 to retrieve the slide from the appropriate slide station within storage module 322.
An automated system for slide transport between processing stations is disclosed. One skilled in the art will appreciate that the present invention can be practiced by other than the preferred embodiments which are presented in this description for purposes of illustration and not of limitation, and the present invention is limited only by the claims that follow. It is noted that equivalents for the particular embodiments discussed in this description may practice the invention as well. Representatively, in one embodiment, a “Reflex Staining” procedure may be implemented. In that procedure, the system recommends specific staining and/or testing of biological samples based on pattern recognition reports of an interpretation module. The reflex stainer may include a staining system, imager and interpretation module, which are grouped together and implemented as a single automated instrument. Alternatively, they may be in separate instruments at different locations. Processing may be performed by instruments both inside and outside the reflex stainer, such as for example in grossing, processing and embedding, microtomy and staining and coverslipping.
In some embodiments, the above-described transport module may be implemented in connection with tissue block processing. Representatively, a sample of tissue, which has potentially been grossed and/or fixated in a block of paraffin, may be transported by the transport module between a microtome, imager and storage module. For example, the block having the tissue embedded therein and an identifier may be sectioned by the microtome and then transported to the storage module. If, upon examination of the tissue section, it is determined that another tissue section is needed, a controller may signal the transport module to retrieve the block from the storage module and transport it back to the microtome for additional sectioning.
One or more embodiments of the invention may be provided as a program product or other article of manufacture that may include a machine-readable computer medium having stored thereon one or more instructions. The medium may provide instructions, which, if executed by a machine such as a robot or integration unit, may result in and/or cause the machine to perform one or more of the operations or methods disclosed herein. Suitable machines include, but are not limited to, robots, integration units, computer systems, laboratory equipment, and a wide variety of other machines, to name just a few examples. Representatively, the medium may include recordable mediums, such as, for example, floppy diskette, optical storage medium, optical disk, CD-ROM, magnetic disk, magneto-optical disk, read only memory (ROM), programmable ROM (PROM), erasable-and-programmable ROM (EPROM), electrically-erasable-and-programmable ROM (EEPROM), random access memory (RAM), static-RAM (SRAM), dynamic-RAM (DRAM), Flash memory, other types of memory, other machine-readable medium within programmable logic units used to control robots, and combinations thereof.
Example 1 is an apparatus including a first imaging module; a second imaging module; a storage module; an automated transport module operable to transport at least one slide between at least one of the first imaging module, the storage module and the second imaging module; and a controller operable to direct transport of the at least one slide by the transport module into the first imaging module and into the second imaging module.
In Example 2, the controller of the apparatus of Example 1 is operable to direct transport of the at least one slide into the first imaging module and then subsequently into the second imaging module.
In Example 3, the controller of the apparatus of Example 1 directs the first imaging module to capture an image of an entire area of a specimen on a slide.
In Example 4, the controller of the apparatus of Example 3 directs the first imaging module to capture an image of a specimen on a slide in a single plane.
In Example 5, the first imaging module of the apparatus of Example 3 is operable to scan an image of a specimen on a slide.
In Example 6, the second imaging module of the apparatus of Example 1 includes a digital microscope.
In Example 7, the second imaging module of the apparatus of Example 1 includes at least one image sensor, a computer operable to direct an image capture by the at least one image sensor of a portion of a microscope slide and a display coupled to the computer and the computer is operable to display the image captured by the at least one sensor.
In Example 8, the computer of the apparatus of Example 7 is operable to display the image captured without saving an image.
In Example 9, the controller of the apparatus of Example 1 is operable to direct transport of the at least one slide by the transport module from the first imaging module to storage.
In Example 10, the controller of the apparatus of Example 9 is operable to direct transport of the at least slide by the transport module from storage to the second imaging module.
Example 11 is a method including transporting at least one slide to a first imaging module using an automated transport module; capturing by the first imaging module of an image of a specimen on the at least one slide; transporting the at least one slide to a storage module using the automated transport module; and transporting the at least one slide to a second imaging module in response to a request for capturing an image in addition to the image captured by the first imaging module.
In Example 12, transporting the at least one slide to a second imaging module of the method of Example 11 includes transporting the at least one slide from the storage module.
In Example 13, capturing by the first imaging module of an image of a specimen on the at least one slide of the method of Example 11 includes capturing an image of an entire area of the specimen.
In Example 14, capturing by the first imaging module of the method of Example 11 includes capturing of an image of a specimen on a slide in a single plane.
In Example 15, capturing by first imaging module of the method of Example 11 includes scanning of an image of a specimen on a slide.
In Example 16, the second imaging module of the method of Example 11 includes a digital microscope.
In Example 17, the second imaging module of the method of Example 11 includes at least one image sensor, a computer operable to direct an image capture by the at least one image sensor of a portion of a microscope slide and a display coupled to the computer and the computer is operable to display the image captured by the at least one sensor.
In Example 18, the computer of the method of Example 17 is operable to display the image captured without saving an image.
In Example 19, the method of Example 11 further includes capturing by the second imaging module of an image of the specimen.
In Example 20, capturing by the second imaging module of an image of the specimen of the method of Example 19 includes transporting the at least one slide to a storage module.
Example 21 is a machine-readable medium including program instructions that when executed by a controller linked to a first imaging module, a storage module, and a second imaging module cause the controller to perform a method including delivering the at least one slide to the first imaging module; capturing by the first imaging module of an image of a specimen on the at least one slide; transporting the at least one slide to a storage; and transporting the at least one slide to a second imaging module in response to a request for capturing an image in addition to the image captured by the first imaging module.
In Example 22, transporting the at least one slide to a second imaging module of the machine-readable medium of Example 21 includes transporting the at least one slide from the storage module.
In Example 23, capturing by the first imaging module of an image of a specimen on the at least one slide of the machine-readable medium of Example 21 includes capturing an image of an entire area of the specimen.
In Example 24, capturing by the first imaging module of the machine-readable medium of Example 21 includes capturing of an image of a specimen on a slide in a single plane.
In Example 25, capturing by first imaging module of the machine-readable medium of Example 21 includes scanning of an image of a specimen on a slide.
In Example 26, the second imaging module of the machine-readable medium of Example 21 includes a digital microscope.
In Example 27, the second imaging module of the machine-readable medium of Example 21 includes at least one image sensor, a computer operable to direct an image capture by the at least one image sensor of a portion of a microscope slide and a display coupled to the computer and the computer is operable to display the image captured by the at least one sensor.
In Example 28, the computer of the machine-readable medium of Example 27 is operable to display the image captured without saving an image.
In Example 29, the program instructions cause the controller to perform a method of the machine-readable medium of Example 27 further includes delivering the at least one slide to a storage module after the image capture at the second imaging module.
It should also be appreciated that reference throughout this specification to “one embodiment”, “an embodiment”, or “one or more embodiments”, for example, means that a particular feature may be included in the practice of the invention. Similarly, it should be appreciated that in the description various features are sometimes grouped together in a single embodiment, Figure, or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the invention requires more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects may lie in less than all features of a single disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of the invention.
In the foregoing specification, the invention has been described with reference to specific embodiments thereof. It will, however, be evident that various modifications and changes can be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
| Number | Name | Date | Kind |
|---|---|---|---|
| 2051051 | Lilienfeld | Aug 1936 | A |
| 3309262 | Copeland et al. | Mar 1967 | A |
| 3525803 | Smart | Aug 1970 | A |
| 3762798 | Grubb et al. | Oct 1973 | A |
| 3765851 | White | Oct 1973 | A |
| 3862909 | Copeland | Jan 1975 | A |
| 4000417 | Adkisson et al. | Dec 1976 | A |
| 4079248 | Lehureau et al. | Mar 1978 | A |
| 4089989 | White et al. | May 1978 | A |
| 4148752 | Burger et al. | Apr 1979 | A |
| 4404683 | Kobayashi et al. | Sep 1983 | A |
| 4477185 | Berger et al. | Oct 1984 | A |
| 4595829 | Neumann et al. | Jun 1986 | A |
| 4673988 | Jansson et al. | Jun 1987 | A |
| 4684799 | Emoto et al. | Aug 1987 | A |
| 4737022 | Faltermeier et al. | Apr 1988 | A |
| 4760385 | Jansson et al. | Jul 1988 | A |
| 4761075 | Matsushita et al. | Aug 1988 | A |
| 4836667 | Ozeki | Jun 1989 | A |
| 4849177 | Jordan | Jul 1989 | A |
| 4958920 | Jorgens et al. | Sep 1990 | A |
| 4962264 | Forester | Oct 1990 | A |
| 5180606 | Stokes et al. | Jan 1993 | A |
| 5287272 | Rutenberg et al. | Feb 1994 | A |
| 5297034 | Weinstien | Mar 1994 | A |
| 5297215 | Yamagishi | Mar 1994 | A |
| 5311426 | Donohue et al. | May 1994 | A |
| 5367401 | Saulietis | Nov 1994 | A |
| 5428690 | Bacus et al. | Jun 1995 | A |
| 5473706 | Bacus et al. | Dec 1995 | A |
| 5530237 | Sato et al. | Jun 1996 | A |
| 5532874 | Stein | Jul 1996 | A |
| 5546323 | Bacus et al. | Aug 1996 | A |
| 5561556 | Weissman et al. | Oct 1996 | A |
| 5581637 | Cass et al. | Dec 1996 | A |
| 5655028 | Soll et al. | Aug 1997 | A |
| 5659174 | Kaneoka et al. | Aug 1997 | A |
| 5675141 | Kukihara | Oct 1997 | A |
| 5686960 | Sussman et al. | Nov 1997 | A |
| 5696589 | Bernacki | Dec 1997 | A |
| 5737084 | Ishihara | Apr 1998 | A |
| 5768033 | Brock | Jun 1998 | A |
| 5793969 | Kamentsky et al. | Aug 1998 | A |
| 5836877 | Zavislan | Nov 1998 | A |
| 5864138 | Miyata et al. | Jan 1999 | A |
| 5891619 | Zakim et al. | Apr 1999 | A |
| 5924074 | Evans | Jun 1999 | A |
| 5947167 | Bogen et al. | Sep 1999 | A |
| 6008892 | Kain et al. | Dec 1999 | A |
| 6031930 | Bacus et al. | Feb 2000 | A |
| 6043475 | Shimada et al. | Mar 2000 | A |
| 6061176 | Shih | May 2000 | A |
| 6078681 | Silver | Jun 2000 | A |
| 6091075 | Shibata et al. | Jul 2000 | A |
| 6091842 | Domanik et al. | Jul 2000 | A |
| 6101265 | Bacus et al. | Aug 2000 | A |
| 6130745 | Manian et al. | Oct 2000 | A |
| 6147797 | Lee | Nov 2000 | A |
| 6205235 | Roberts | Mar 2001 | B1 |
| 6208374 | Clinch | Mar 2001 | B1 |
| 6215892 | Douglass et al. | Apr 2001 | B1 |
| 6226352 | Salb | May 2001 | B1 |
| 6226392 | Bacus et al. | May 2001 | B1 |
| 6248995 | Tanaami et al. | Jun 2001 | B1 |
| 6272235 | Bacus et al. | Aug 2001 | B1 |
| 6309607 | Johnston et al. | Oct 2001 | B1 |
| 6396941 | Bacus et al. | May 2002 | B1 |
| 6404906 | Bacus et al. | Jun 2002 | B2 |
| 6466690 | Bacus et al. | Oct 2002 | B2 |
| 6522774 | Bacus et al. | Feb 2003 | B1 |
| 6529271 | Engelhardt | Mar 2003 | B1 |
| 6606413 | Zeineh | Aug 2003 | B1 |
| 6671393 | Hays et al. | Dec 2003 | B2 |
| 6674881 | Bacus et al. | Jan 2004 | B2 |
| 6674884 | Bacus et al. | Jan 2004 | B2 |
| 6678398 | Wolters et al. | Jan 2004 | B2 |
| 6684092 | Zavislan | Jan 2004 | B2 |
| 6711283 | Soenksen | Mar 2004 | B1 |
| 6735531 | Rhett et al. | May 2004 | B2 |
| 6745916 | Plank | Jun 2004 | B2 |
| 6775402 | Bacus et al. | Aug 2004 | B2 |
| 6800249 | de la Torre-Bueno | Oct 2004 | B2 |
| 6800853 | Ohkura | Oct 2004 | B2 |
| 6812446 | Kreh | Nov 2004 | B2 |
| 6834237 | Noergaard et al. | Dec 2004 | B2 |
| 6838650 | Toh | Jan 2005 | B1 |
| 6847481 | Ludl et al. | Jan 2005 | B1 |
| 6847729 | Clinch et al. | Jan 2005 | B1 |
| 6947583 | Ellis et al. | Sep 2005 | B2 |
| 6959720 | Kurihara et al. | Nov 2005 | B2 |
| 6982741 | Fiedler | Jan 2006 | B2 |
| 6993169 | Wetzel et al. | Jan 2006 | B2 |
| 7009638 | Gruber et al. | Mar 2006 | B2 |
| 7016109 | Nakagawa | Mar 2006 | B2 |
| 7027627 | Levin et al. | Apr 2006 | B2 |
| 7031507 | Bacus et al. | Apr 2006 | B2 |
| 7071969 | Stimson | Jul 2006 | B1 |
| 7098634 | Yu | Aug 2006 | B1 |
| 7110586 | Bacus et al. | Sep 2006 | B2 |
| 7110645 | Birk et al. | Sep 2006 | B2 |
| 7133545 | Douglass et al. | Nov 2006 | B2 |
| 7136518 | Griffin et al. | Nov 2006 | B2 |
| 7141802 | Takeyama et al. | Nov 2006 | B2 |
| 7146372 | Bacus et al. | Dec 2006 | B2 |
| 7149332 | Bacus et al. | Dec 2006 | B2 |
| 7171030 | Foran et al. | Jan 2007 | B2 |
| 7194118 | Harris et al. | Mar 2007 | B1 |
| 7196300 | Watkins et al. | Mar 2007 | B2 |
| 7209287 | Lauer | Apr 2007 | B2 |
| 7212660 | Wetzel et al. | May 2007 | B2 |
| 7224839 | Zeineh | May 2007 | B2 |
| 7233340 | Hughes et al. | Jun 2007 | B2 |
| 7248403 | Nakagawa | Jul 2007 | B2 |
| 7250963 | Yuri et al. | Jul 2007 | B2 |
| 7292251 | Gu | Nov 2007 | B1 |
| 7297910 | Fomitchov | Nov 2007 | B2 |
| 7301133 | Weiss | Nov 2007 | B2 |
| 7349482 | Kim | Mar 2008 | B2 |
| 7359548 | Douglass et al. | Apr 2008 | B2 |
| 7391894 | Zeineh | Jun 2008 | B2 |
| 7394482 | Olschewski | Jul 2008 | B2 |
| 7394979 | Luther et al. | Jul 2008 | B2 |
| 7396508 | Richards et al. | Jul 2008 | B1 |
| 7400342 | Gaida et al. | Jul 2008 | B2 |
| 7400983 | Feingold et al. | Jul 2008 | B2 |
| 7406215 | Clune et al. | Jul 2008 | B2 |
| 7421102 | Wetzel et al. | Sep 2008 | B2 |
| 7426345 | Takamatsu et al. | Sep 2008 | B2 |
| 7428325 | Douglass et al. | Sep 2008 | B2 |
| 7433026 | Wolpert et al. | Oct 2008 | B2 |
| 7456377 | Zeineh et al. | Nov 2008 | B2 |
| 7463761 | Eichhorn et al. | Dec 2008 | B2 |
| 7470541 | Copeland et al. | Dec 2008 | B2 |
| 7482600 | Seyfried | Jan 2009 | B2 |
| 7483554 | Kotsianti et al. | Jan 2009 | B2 |
| 7486329 | Endo | Feb 2009 | B2 |
| 7502519 | Eichhorn et al. | Mar 2009 | B2 |
| 7542596 | Bacus et al. | Jun 2009 | B2 |
| 7550699 | Marshall | Jun 2009 | B1 |
| 7584019 | Feingold et al. | Sep 2009 | B2 |
| 7596249 | Bacus et al. | Sep 2009 | B2 |
| 7602524 | Eichhorn et al. | Oct 2009 | B2 |
| 7623697 | Hughes et al. | Nov 2009 | B1 |
| 7630113 | Sase et al. | Dec 2009 | B2 |
| 7633616 | Hing | Dec 2009 | B2 |
| 7642093 | Tseung et al. | Jan 2010 | B2 |
| 7653300 | Fujiyoshi et al. | Jan 2010 | B2 |
| 7657070 | Lefebvre | Feb 2010 | B2 |
| 7663078 | Virag et al. | Feb 2010 | B2 |
| 7677289 | Hayworth et al. | Mar 2010 | B2 |
| 7689024 | Eichhorn et al. | Mar 2010 | B2 |
| 7738688 | Eichhorn et al. | Jun 2010 | B2 |
| 7756309 | Gholap et al. | Jul 2010 | B2 |
| 7756357 | Yoneyama | Jul 2010 | B2 |
| 7778485 | Zeineh et al. | Aug 2010 | B2 |
| 7822257 | Endo et al. | Oct 2010 | B2 |
| 7840300 | Harker | Nov 2010 | B2 |
| 7856131 | Bacus et al. | Dec 2010 | B2 |
| 7860292 | Eichhorn et al. | Dec 2010 | B2 |
| 7864414 | Sase et al. | Jan 2011 | B2 |
| 7869641 | Wetzel et al. | Jan 2011 | B2 |
| 7873193 | De La Torre-Bueno et al. | Jan 2011 | B2 |
| 7876948 | Wetzel et al. | Jan 2011 | B2 |
| RE42220 | Clinch et al. | Mar 2011 | E |
| 7901941 | Tseung et al. | Mar 2011 | B2 |
| 7912267 | Kawano et al. | Mar 2011 | B2 |
| 7916916 | Zeineh | Mar 2011 | B2 |
| 7920163 | Kossin | Apr 2011 | B1 |
| 7925067 | Bacus et al. | Apr 2011 | B2 |
| 7944608 | Hayashi et al. | May 2011 | B2 |
| 7949161 | Kawanabe et al. | May 2011 | B2 |
| 7957057 | Sase et al. | Jun 2011 | B2 |
| 7967057 | Kunii et al. | Jun 2011 | B2 |
| 7978894 | Soenksen et al. | Jul 2011 | B2 |
| 8000560 | Shirota | Aug 2011 | B2 |
| 8000562 | Morales et al. | Aug 2011 | B2 |
| 8036868 | Zeineh et al. | Oct 2011 | B2 |
| 8074547 | Ito et al. | Dec 2011 | B2 |
| 8077959 | Dekel et al. | Dec 2011 | B2 |
| 8085296 | Yuguchi et al. | Dec 2011 | B2 |
| 8094902 | Crandall et al. | Jan 2012 | B2 |
| 8094914 | Iki et al. | Jan 2012 | B2 |
| 8098279 | Sase et al. | Jan 2012 | B2 |
| 8098956 | Tatke et al. | Jan 2012 | B2 |
| 8103082 | Olson et al. | Jan 2012 | B2 |
| 8125534 | Shimonaka | Feb 2012 | B2 |
| 8159547 | Kawashima | Apr 2012 | B2 |
| 8174763 | Guiney et al. | May 2012 | B2 |
| 8187536 | Graupner et al. | May 2012 | B2 |
| 8199358 | Eichhorn et al. | Jun 2012 | B2 |
| 8203575 | Molnar et al. | Jun 2012 | B2 |
| 8283176 | Bland et al. | Oct 2012 | B2 |
| 8304704 | Hing et al. | Nov 2012 | B2 |
| 8305434 | Nakatsuka et al. | Nov 2012 | B2 |
| 8306298 | Bacus et al. | Nov 2012 | B2 |
| 8306300 | Bacus et al. | Nov 2012 | B2 |
| 8339703 | Knebel | Dec 2012 | B2 |
| 8350904 | Fujimoto et al. | Jan 2013 | B2 |
| 8366857 | Hayworth et al. | Feb 2013 | B2 |
| 8385619 | Soenksen | Feb 2013 | B2 |
| 8385686 | Sano | Feb 2013 | B2 |
| 8388891 | Lefebvre | Mar 2013 | B2 |
| 8394635 | Key et al. | Mar 2013 | B2 |
| 8396669 | Cocks | Mar 2013 | B2 |
| 8463741 | Ehlke et al. | Jun 2013 | B2 |
| 8473035 | Frangioni | Jun 2013 | B2 |
| 8476585 | Galloway | Jul 2013 | B2 |
| 8501435 | Gustafsson et al. | Aug 2013 | B2 |
| 8565480 | Eichhorn et al. | Oct 2013 | B2 |
| 8565503 | Eichhorn et al. | Oct 2013 | B2 |
| 8582489 | Eichhorn et al. | Nov 2013 | B2 |
| 8582849 | Eichhorn et al. | Nov 2013 | B2 |
| 8673642 | Key et al. | Mar 2014 | B2 |
| 8687858 | Walter et al. | Apr 2014 | B2 |
| 8725237 | Bryant-Greenwood et al. | May 2014 | B2 |
| 8730315 | Yoneyama | May 2014 | B2 |
| 8744213 | Tatke et al. | Jun 2014 | B2 |
| 8747746 | Lefebvre | Jun 2014 | B2 |
| 8771978 | Ragan | Jul 2014 | B2 |
| 8788217 | Feingold et al. | Jul 2014 | B2 |
| 8796038 | Williamson, IV et al. | Aug 2014 | B2 |
| 8827760 | Ushibo et al. | Sep 2014 | B2 |
| 8923597 | Eichhorn et al. | Dec 2014 | B2 |
| 9310598 | Hing et al. | Apr 2016 | B2 |
| 20010035752 | Kormos et al. | Nov 2001 | A1 |
| 20020169512 | Stewart | Nov 2002 | A1 |
| 20020176160 | Suzuki et al. | Nov 2002 | A1 |
| 20020176161 | Yoneyama et al. | Nov 2002 | A1 |
| 20030048931 | Johnson et al. | Mar 2003 | A1 |
| 20030098921 | Endo | May 2003 | A1 |
| 20030112330 | Yuri et al. | Jun 2003 | A1 |
| 20030112504 | Czarnetzki et al. | Jun 2003 | A1 |
| 20030124729 | Christensen et al. | Jul 2003 | A1 |
| 20030133009 | Brown | Jul 2003 | A1 |
| 20030142882 | Beged-Dov et al. | Jul 2003 | A1 |
| 20030156276 | Bowes | Aug 2003 | A1 |
| 20040021936 | Czarnetzki et al. | Feb 2004 | A1 |
| 20040027462 | Hing | Feb 2004 | A1 |
| 20040080758 | Ban et al. | Apr 2004 | A1 |
| 20040090667 | Gartner et al. | May 2004 | A1 |
| 20040113043 | Ishikawa et al. | Jun 2004 | A1 |
| 20040129858 | Czarnetzki et al. | Jul 2004 | A1 |
| 20040135061 | Kreh | Jul 2004 | A1 |
| 20040141660 | Barth et al. | Jul 2004 | A1 |
| 20050057812 | Raber | Mar 2005 | A1 |
| 20050073649 | Spector | Apr 2005 | A1 |
| 20050090017 | Morales | Apr 2005 | A1 |
| 20050094262 | Spediacci et al. | May 2005 | A1 |
| 20050112537 | Wu | May 2005 | A1 |
| 20050211874 | Takeyama et al. | Sep 2005 | A1 |
| 20050219688 | Kawano et al. | Oct 2005 | A1 |
| 20050221351 | Jekwam | Oct 2005 | A1 |
| 20050239113 | Ryu et al. | Oct 2005 | A1 |
| 20050248837 | Sase | Nov 2005 | A1 |
| 20050258335 | Oshiro et al. | Nov 2005 | A1 |
| 20060039583 | Bickert et al. | Feb 2006 | A1 |
| 20060045388 | Zeineh | Mar 2006 | A1 |
| 20060077536 | Bromage et al. | Apr 2006 | A1 |
| 20060088940 | Feingold et al. | Apr 2006 | A1 |
| 20060098861 | See et al. | May 2006 | A1 |
| 20060146283 | Baumann et al. | Jul 2006 | A1 |
| 20060164623 | Wagner et al. | Jul 2006 | A1 |
| 20060171560 | Manus | Aug 2006 | A1 |
| 20060179992 | Kermani | Aug 2006 | A1 |
| 20070025606 | Gholap et al. | Feb 2007 | A1 |
| 20070091324 | Paul et al. | Apr 2007 | A1 |
| 20070098237 | Yoo et al. | May 2007 | A1 |
| 20070102620 | Bublitz et al. | May 2007 | A1 |
| 20070164194 | Kurata et al. | Jul 2007 | A1 |
| 20070198001 | Bauch et al. | Aug 2007 | A1 |
| 20070206096 | Cooke et al. | Sep 2007 | A1 |
| 20070207061 | Yang et al. | Sep 2007 | A1 |
| 20070224699 | Gates | Sep 2007 | A1 |
| 20070285768 | Kawanabe et al. | Dec 2007 | A1 |
| 20080002252 | Weiss et al. | Jan 2008 | A1 |
| 20080020128 | van Ryper et al. | Jan 2008 | A1 |
| 20080054156 | Fomitchov | Mar 2008 | A1 |
| 20080095424 | Iki et al. | Apr 2008 | A1 |
| 20080095467 | Olszak et al. | Apr 2008 | A1 |
| 20080142708 | Workman et al. | Jun 2008 | A1 |
| 20080180794 | Tafas et al. | Jul 2008 | A1 |
| 20080240613 | Dietz et al. | Oct 2008 | A1 |
| 20080283722 | Uchiyama et al. | Nov 2008 | A1 |
| 20090040322 | Leberl et al. | Feb 2009 | A1 |
| 20090046298 | Betzig | Feb 2009 | A1 |
| 20090116101 | Tafas et al. | May 2009 | A1 |
| 20090140169 | Niehren | Jun 2009 | A1 |
| 20090195688 | Henderson | Aug 2009 | A1 |
| 20100000383 | Koos et al. | Jan 2010 | A1 |
| 20100020157 | Jelinek et al. | Jan 2010 | A1 |
| 20100039507 | Imade | Feb 2010 | A1 |
| 20100074489 | Bacus et al. | Mar 2010 | A1 |
| 20100093022 | Hayworth et al. | Apr 2010 | A1 |
| 20100102571 | Yang | Apr 2010 | A1 |
| 20100109725 | Yun et al. | May 2010 | A1 |
| 20100118133 | Walter et al. | May 2010 | A1 |
| 20100118393 | Lin | May 2010 | A1 |
| 20100134655 | Kuroiwa | Jun 2010 | A1 |
| 20100141751 | Uchida | Jun 2010 | A1 |
| 20100141752 | Yamada | Jun 2010 | A1 |
| 20100141753 | Olson et al. | Jun 2010 | A1 |
| 20100171809 | Fujiyoshi | Jul 2010 | A1 |
| 20100177166 | Eichhorn et al. | Jul 2010 | A1 |
| 20100188738 | Epple et al. | Jul 2010 | A1 |
| 20100194873 | Viereck et al. | Aug 2010 | A1 |
| 20100201800 | Yamamoto et al. | Aug 2010 | A1 |
| 20100225668 | Tatke et al. | Sep 2010 | A1 |
| 20100260407 | Eichhorn et al. | Oct 2010 | A1 |
| 20100279342 | Kijima et al. | Nov 2010 | A1 |
| 20100295932 | Yakomachi et al. | Nov 2010 | A1 |
| 20100310139 | Kimura | Dec 2010 | A1 |
| 20110017902 | Hing et al. | Jan 2011 | A1 |
| 20110037847 | Soenksen | Feb 2011 | A1 |
| 20110038523 | Boardman | Feb 2011 | A1 |
| 20110043663 | Tsuchiya | Feb 2011 | A1 |
| 20110064296 | Dixon | Mar 2011 | A1 |
| 20110074817 | Shinichi et al. | Mar 2011 | A1 |
| 20110102571 | Yoneyama | May 2011 | A1 |
| 20110109735 | Otsuka | May 2011 | A1 |
| 20110145755 | Bacus et al. | Jun 2011 | A1 |
| 20110181622 | Bacus et al. | Jul 2011 | A1 |
| 20110221881 | Shirota et al. | Sep 2011 | A1 |
| 20110316993 | Chen et al. | Dec 2011 | A1 |
| 20110316999 | Yoneyama et al. | Dec 2011 | A1 |
| 20120002043 | Nitta | Jan 2012 | A1 |
| 20120002892 | Eichhorn et al. | Jan 2012 | A1 |
| 20120038979 | Hing et al. | Feb 2012 | A1 |
| 20120044342 | Hing et al. | Feb 2012 | A1 |
| 20120069171 | Kodaira et al. | Mar 2012 | A1 |
| 20120069344 | Liu | Mar 2012 | A1 |
| 20120076391 | Dietz et al. | Mar 2012 | A1 |
| 20120076411 | Dietz et al. | Mar 2012 | A1 |
| 20120076436 | Dietz et al. | Mar 2012 | A1 |
| 20120081536 | Kuppig et al. | Apr 2012 | A1 |
| 20120114204 | Olson et al. | May 2012 | A1 |
| 20120120225 | Maddison | May 2012 | A1 |
| 20120127297 | Baxi et al. | May 2012 | A1 |
| 20120163680 | Lefebvre | Jun 2012 | A1 |
| 20120208184 | Ragan | Aug 2012 | A1 |
| 20120281931 | Eichhorn et al. | Nov 2012 | A1 |
| 20130003172 | Widzgowski et al. | Jan 2013 | A1 |
| 20130076886 | Ikeno et al. | Mar 2013 | A1 |
| 20130140459 | Galloway | Jun 2013 | A1 |
| 20130162802 | Soenksen | Jun 2013 | A1 |
| 20130164781 | Lefebvre | Jun 2013 | A1 |
| 20130182922 | Kil | Jul 2013 | A1 |
| 20130216451 | Hayworth et al. | Aug 2013 | A1 |
| 20130250090 | Morimoto | Sep 2013 | A1 |
| 20140030757 | Schiffenbauer | Jan 2014 | A1 |
| 20140049632 | Hemmer | Feb 2014 | A1 |
| 20140051158 | Nakajima et al. | Feb 2014 | A1 |
| 20140085453 | Yamane | Mar 2014 | A1 |
| 20140086463 | Meetz et al. | Mar 2014 | A1 |
| 20140087411 | Chow et al. | Mar 2014 | A1 |
| 20140098376 | Hashimshony et al. | Apr 2014 | A1 |
| 20140112560 | Soenksen | Apr 2014 | A1 |
| 20140118528 | Wolff et al. | May 2014 | A1 |
| 20140130613 | Adiga et al. | May 2014 | A1 |
| 20140137715 | Sneyders et al. | May 2014 | A1 |
| 20140273086 | Lefebvre | Sep 2014 | A1 |
| 20150015578 | Eichhorn et al. | Jan 2015 | A1 |
| 20150153552 | Loney et al. | Jun 2015 | A1 |
| 20150153555 | Loney et al. | Jun 2015 | A1 |
| 20150153556 | Loney et al. | Jun 2015 | A1 |
| 20150177504 | Bickert | Jun 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 2504245 | Nov 2006 | CA |
| 102565428 | Jul 2012 | CN |
| 102782557 | Nov 2012 | CN |
| 102841079 | Dec 2012 | CN |
| 102009012293 | Mar 2009 | DE |
| 1447699 | Aug 2004 | EP |
| 2051051 | Apr 2009 | EP |
| 2169379 | Mar 2010 | EP |
| 2110696 | Oct 2009 | ER |
| 2620537 | Mar 1989 | FR |
| 03092 | Nov 1906 | GB |
| 59071018 | Apr 1984 | JP |
| 61248168 | Nov 1986 | JP |
| S63206793 | Aug 1988 | JP |
| 1992335787 | Nov 1992 | JP |
| 09080138 | Mar 1997 | JP |
| 09133856 | May 1997 | JP |
| 9161068 | Jun 1997 | JP |
| 09218354 | Aug 1997 | JP |
| 2000055635 | Feb 2000 | JP |
| 2001281553 | Oct 2001 | JP |
| 2002-031513 | Jan 2002 | JP |
| 200284554 | Mar 2002 | JP |
| 2003248176 | Sep 2003 | JP |
| 2006003543 | Jan 2006 | JP |
| 2006220654 | Aug 2006 | JP |
| 2006292999 | Oct 2006 | JP |
| 2006343595 | Dec 2006 | JP |
| 2007516447 | Jun 2007 | JP |
| 2007233093 | Sep 2007 | JP |
| 2009192824 | Feb 2008 | JP |
| 2008262100 | Oct 2008 | JP |
| 2009-036969 | Feb 2009 | JP |
| 2011008245 | Jan 2011 | JP |
| 2011008397 | Jan 2011 | JP |
| 2011232762 | Nov 2011 | JP |
| 2012085252 | Apr 2012 | JP |
| 2012141287 | Jul 2012 | JP |
| 201201392 | Jan 2012 | TW |
| WO-0154052 | Jul 2001 | WO |
| WO-2005015120 | Feb 2005 | WO |
| 2006028439 | Mar 2006 | WO |
| WO-2008118886 | Oct 2008 | WO |
| WO-2008141009 | Nov 2008 | WO |
| WO-2010105015 | Sep 2010 | WO |
| WO-2012024627 | Feb 2012 | WO |
| Entry |
|---|
| Sakura Finetek U.S.A., Inc., “Final Office Action”, U.S. Appl. No. 14/138,740, (dated Jan. 26, 2017). |
| Sakura Finetek U.S.A., Inc., “First Office Action”, EP Application No. 15194968.2, (dated Mar. 10, 2017). |
| Sakura Finetek USA Inc., “Office Action”, EP Application No. 15154503.5, (dated Feb. 28, 2017). |
| Requirement for Restriction/Election dated Nov. 3, 2014 for U.S. Appl. No. 13/255,827. |
| Office Action received for Japanese Patent Application No. 2016-507909, dated Apr. 28, 2017. |
| Office Action received for European Patent Application No. 14784707.3, dated Mar. 5, 2018. |
| Office Action received for Eurasian Patent Application No. 201001786, dated Jan. 21, 2014. |
| Office Action received for Chinese Patent Application No. 201480021837.2, dated Apr. 8, 2018. |
| Office Action received for Chinese Patent Application No. 201410415253.5, dated Feb. 9, 2018. |
| Office Action received for Canadian Patent Application No. 2908058, dated Jul. 24, 2017. |
| Office Action received for Canadian Patent Application No. 2808105, dated Jun. 12, 2017. |
| Notice of Allowance received for U.S. Appl. No. 14/138,740, dated Feb. 13, 2018. |
| Notice of Allowance and Fees Due (PTOL-85) dated Dec. 16, 2015 for U.S. Appl. No. 13/255,827. |
| Non-Final Office Action received for U.S. Appl. No. 14/779,550, dated Dec. 22, 2017. |
| Non-Final Office Action received for U.S. Appl. No. 13/212,955, dated Aug. 9, 2017. |
| International Search Report and Written Opinion received for PCT Patent Application No. PCT/IB2010/000518, dated Jul. 16, 2010, 16 pages (8 pages of English Translation and 8 pages of Original Document). |
| Final Office Action received for U.S. Appl. No. 14/779,550, dated May 24, 2017. |
| Final Office Action received for U.S. Appl. No. 14/138,740, dated Jun. 20, 2017. |
| Extended European Search Report and Written Opinion received for EP Patent Application No. 17202516, dated Apr. 24, 2018. |
| Forrest, K., et al., “Tunneling calculations for GaAs—AlxGa(1-x)As graded band-gap sawtooth superlattices”, IEEE Journal of Quantum Electronics, vol. 26, No. 6, (Jun. 1990), 1067-1074. |
| Haruhisa, S., et al., “Application of telepathology for improvement of therapeutic effects and economic efficiency, and development of new equipment for it”, Science Links Japan; http://sciencelinks.jp/j-east/article/200516/000020051605A0431066.php, Journal Code: N20051113, (2005), 166-125. |
| Sakura Finetek U.S.A., “Non-Final Office Action”, U.S. Appl. No. 13/212,955, (dated May 3, 2016). |
| Sakura Finetek U.S.A., Inc., “Examination Report”, AU Application No. 2011291517, (dated Dec. 24, 2013). |
| Sakura Finetek U.S.A., Inc., “Examination Report”, AU Application No. 2011291517, (dated Jun. 19, 2014). |
| Sakura Finetek U.S.A., Inc., et al., “Examination Report”, AU Application No. 2011291517, (dated Feb. 2, 2015). |
| Sakura Finetek U.S.A., Inc., et al., “Examiner's Report”, CA Application No. 2,755,164, (dated Dec. 7, 2012). |
| Sakura Finetek U.S.A., Inc., “Extended European Search Report”, EP Application No. 15194968.2, (dated Mar. 18, 2016). |
| Sakura Finetek U.S.A., Inc., “Extended Search Report”, EP Application No. 15154503, (dated Jun. 19, 2015). |
| Sakura Finetek U.S.A., Inc., “Final Office Action”, U.S. Appl. No. 13/212,955, (dated Apr. 15, 2015). |
| Sakura Finetek U.S.A., Inc., “Final Office Action”, U.S. Appl. No. 13/255,827, (dated Oct. 20, 2015). |
| Sakura Finetek U.S.A., Inc., “Final Office Action”, JP Application No. 2011-553548, (dated Sep. 2, 2014). |
| Sakura Finetek U.S.A., Inc., “Final Office Action”, CN Application No. 201080017649,4, (dated Jul. 3, 2014). |
| Sakura Finetek U.S.A., Inc., et al., “International Preliminary Report on Patentability”, International Application No. PCT/US2011/048488, (dated Mar. 7, 2013). |
| Sakura Finetek U.S.A., Inc., “International Search Report and Written Opinion”, International Application No. PCT/US2011/048488, (dated Oct. 13, 2011). |
| Sakura Finetek U.S.A., Inc., “Non final office action”, U.S. Appl. No. 14/138,740, (dated Jul. 1, 2016). |
| Sakura Finetek U.S.A., Inc., “Non-Final Office Action”, U.S. Appl. No. 13/255,827, (dated Apr. 8, 2015). |
| Sakura Finetek U.S.A., Inc., “Non-Final Office Action”, U.S. Appl. No. 13/212,955, (dated Oct. 31, 2014). |
| Sakura Finetek U.S.A., Inc., “Notice of rejection”, JP Application No. 2013-525005, (dated Feb. 9, 2016). |
| Sakura Finetek U.S.A., Inc., et al., “Office Action”, CN Application No. 201180047558.X, (dated Nov. 15, 2014). |
| Sakura Finetek U.S.A., Inc., et al., “Office Action”, EP Application No. 11749675.2, (dated Jan. 30, 2015). |
| Sakura Finetek U.S.A., Inc., “Office Action”, EP Application No. 10719379.9, (dated Jul. 30, 2013). |
| Sakura Finetek U.S.A., Inc., “Office Action”, AU Application No. 2010222633, (dated Nov. 26, 2013). |
| Sakura Finetek U.S.A., Inc., “Office Action”, JP Application No. P2011-553548, (dated Dec. 10, 2013). |
| Sakura Finetek U.S.A., Inc., “Partial Search Report”, EP Application No. 14198636.4, (dated Apr. 28, 2015). |
| Sakura Finetek U.S.A., Inc., “PCT Search Report and Written Opinion”, International Application No. PCT/US2014/034477, (dated Sep. 22, 2014). |
| Sakura Finetek U.S.A., Inc., “Second Office Action”, EP Application No. 10719379.9, (dated Nov. 6, 2014). |
| Sakura Finetek U.S.A., Inc., “Second Office Action”, CN Application No. 201080017649.4, (dated Dec. 27, 2013). |
| Sakura Finetek U.S.A., Inc., “Second office action”, CN Application No. 201180047558.X, (dated Jul. 6, 2015). |
| Sakura Finetek U.S.A., Inc., “Third Office Action”, CN Application No. 201180047558X, (dated Apr. 1, 2016). |
| Sakura Finetek USA, Inc., “Office Action”, JP Application No. 2016-507909, (dated Sep. 15, 2016). |
| Sensovation AG, “International Preliminary Reporton Patentability”, International Application No. PCT/IB2010/000518., (dated Sep. 20, 2011), 7 pages. |
| Sakura Finetek U.S.A., Inc., “Second Office Action”, CN Application No. 2014800218372, dated Aug. 1, 2017. |
| Sakura Finetek U.S.A., “Extended Search Report”, European Application No. 14198636, (dated Sep. 30, 2015). |
| Sakura Finetek U.S.A., Inc., “EP Supplementary Search Report”, EP Application No. 14784707.3, (dated Oct. 4, 2016). |
| Sakura Finetek U.S.A., Inc., “Examination Report”, CA Application No. 2908058, (dated Nov. 16, 2016). |
| Sakura Finetek U.S.A., Inc., “Final Rejection”, JP Application No. P2013-525005, (dated Dec. 27, 2016). |
| Sakura Finetek U.S.A., Inc., “First Office Action with search report”, CN Application No. 2014800218372, (dated Nov. 30, 2016). |
| Sakura Finetek U.S.A., Inc., “Fourth Office Action”, CN Application No. 201180047558X, (dated Oct. 24, 2016). |
| Sakura Finetek U.S.A., Inc., “International Preliminary Report on Patentability”, International Application No. PCT/US2014/034477, (dated Oct. 29, 2015). |
| Sakura Finetek U.S.A., Inc., “International Search Report and Written Opinion”, International Application No. PCT/US2014/034477, (dated Sep. 22, 2014). |
| Sakura Finetek U.S.A., Inc., “Non-Final Office Action”, U.S. Appl. No. 14/779,550, (dated Jan. 19, 2017). |
| Sakura Finetek U.S.A., Inc., “Patent Examination Report No. 1”, AU Application No. 201453889, (dated May 18, 2016). |
| Sakura Finetek U.S.A., Inc., Related Application, European Patent Application No. 17202516.5, EPO Communication, dated May 28, 2018. |
| Sakura Finetek U.S.A., Inc., Related Application, Japanese Patent Application No. P2017-219675, Notice of Rejection, dated Jan. 22, 2019. |
| Sakura Finetek U.S.A., Inc., Related Application, Japanese Patent Application No. P2017-219675, Notice of Rejection, dated Sep. 3, 2019. |
| Sakura Finetek U.S.A., Inc., Related Application, Chinese Patent Application No. 201711081542.6, First Office Action, dated Apr. 16, 2020. |
| Sakura Finetek U.S.A., Inc., Related Application, Australian Patent Application No. 2019216682, Examination Report No. 1, dated May 1, 2020. |
| Sakura Finetek U.S.A., Inc., Related Application, Japanese Patent Application No. P2017-219675, Decision of Rejection, dated May 12, 2020. |
| Sakura Finetek U.S.A., Inc., Related Application, Australian Patent Application No. 2019216682, Examination Report No. 2, dated Jul. 10, 2020. |
| Sakura Finetek U.S.A., Inc., Related Application, European Patent Application No. 17202516.5, Communication pursuant to Article 94(3) EPC, dated Oct. 14, 2020. |
| Sakura Finetek U.S.A., Inc., Related Application, Chinese Patent Application No. 201711081542.6, Second Office Action, dated Dec. 3, 2020. |
| Sakura Finetek U.S.A., Inc., Related Application, Japanese Patent Application No. 2017-219675, Notice of Reasons for Refusal, dated Mar. 12, 2021. |
| Sakura Finetek U.S.A., Inc., Related Application, Chinese Patent Application No. 201111081542.6, Third Office Action , dated Jun. 2, 2021. |
| Number | Date | Country | |
|---|---|---|---|
| 20180143214 A1 | May 2018 | US |
