A technique of the present disclosure relates to an information processing apparatus, a method for operating an information processing apparatus, and an operation program for an information processing apparatus.
As a method of measuring a micro surface rugged shape of an object to be observed, a light interference measurement method is known. In the light interference measurement method, first, from an interference fringe image that is a two-dimensional distribution of intensity of interference fringes of object light as illumination light diffracted by an object to be observed and reference light without passing through the object to be observed, a phase difference image that is a two-dimensional distribution of a phase difference between the object light and the reference light is obtained. Then, a method that obtains, as the shape of the object to be observed, a height of the object to be observed along an irradiation direction of the illumination light based on the obtained phase difference image.
A pixel value of the phase difference image is obtained as a function of arctan. For this reason, the pixel value of the phase difference image is obtained to be folded in a range of −π to π (wrapping) that is a range of arctan. For this reason, an actual phase difference is hardly known only with the pixel value, such as a case where the pixel value to be obtained is π/4 even though an actual phase difference is 5π/4. Accordingly, as the pixel value is obtained to be folded in the range of −π to π, in the phase difference image, the pixel value may have a phase jump of about ±2π in places.
In a case where there is such a phase jump, it is not possible to accurately obtain the height of the object to be observed. Accordingly, before obtaining the height of the object to be observed, processing of connecting the phases by adding or subtracting 2π to a portion of the phase difference image where there is a phase jump is executed. Such processing of connecting the phases is referred to as phase connection (or referred to as phase unwrapping). Since the phase connection is performed while repeating trial and error by selecting an optimum path of the phase connection in the phase difference image or the like, an appropriate time is spent.
JP2006-284186A describes a technique that actually measures a rough shape of an object to be observed and performs phase connection with respect to a phase difference image with reference to a measurement result. According to the technique described in JP2006-284186A, since the measurement result of the rough shape of the object to be observed is an important clue, it is possible to reduce a processing time of the phase connection compared to a case where the phase connection is performed without any information.
In JP2006-284186A, there is a need for a mechanism for measuring the rough shape of the object to be observed. An excess measurement time is spent.
An object of the technique of the present disclosure is to provide an information processing apparatus, a method of operating an information processing apparatus, and an operation program for an information processing apparatus capable of reducing a processing time of phase connection without needing for a special mechanism for measuring a rough shape of an object to be observed and without spending an excess measurement time.
To attain the above-described object, there is provided an information processing apparatus of the present disclosure that executes processing of obtaining, from an interference fringe image that is a two-dimensional distribution of intensity of interference fringes of object light as illumination light diffracted by an object to be observed and reference light without passing through the object to be observed, a phase difference image that is a two-dimensional distribution of a phase difference between the object light and the reference light, and obtaining a shape of the object to be observed based on the phase difference image, the information processing apparatus comprising at least one processor configured to acquire object-related information regarding the object to be observed, read out, from a storage unit in which the object-related information and a shape profile indicating the shape of the object to be observed are stored in association with each other, the shape profile corresponding to the acquired object-related information, and perform phase connection with respect to the phase difference image with reference to the read-out shape profile.
It is preferable that the at least one processor is configured to extract a presence region of the object to be observed from the interference fringe image, and selectively perform the phase connection with respect to the presence region.
It is preferable that the at least one processor is configured to perform control of displaying a calculation result of the shape of the object to be observed.
It is preferable that the at least one processor is configured to generate a reproduction image representing any tomographic plane of the object to be observed from the interference fringe image, and display the calculation result of the shape of the object to be observed on the reproduction image in a superimposed manner.
It is preferable that the shape of the object to be observed is a height of the object to be observed along an irradiation direction of the illumination light.
It is preferable that the object to be observed is a cell during culture. In this case, it is preferable that the at least one processor is configured to acquire culture surface height-related information regarding a height from a bottom surface to a culture surface of a culture vessel of the cell.
It is preferable that the object-related information is a type of the cell and a culture condition of the cell. In this case, it is preferable that the culture condition includes at least any one of the number of days of culture, a type of a culture medium, a temperature of a culture environment, or a carbon dioxide concentration of the culture environment.
There is provided a method for operating an information processing apparatus of the present disclosure that executes processing of obtaining, from an interference fringe image that is a two-dimensional distribution of intensity of interference fringes of object light as illumination light diffracted by an object to be observed and reference light without passing through the object to be observed, a phase difference image that is a two-dimensional distribution of a phase difference between the object light and the reference light, and obtaining a shape of the object to be observed based on the phase difference image, the method comprising an acquisition step of acquiring object-related information regarding the object to be observed, a readout step of reading out, from a storage unit in which the object-related information and a shape profile indicating the shape of the object to be observed are stored in association with each other, the shape profile corresponding to the object-related information acquired in the acquisition step, and a phase connection step of performing phase connection with respect to the phase difference image with reference to the shape profile read out in the readout step.
There is provided an operation program for an information processing apparatus of the present disclosure that executes processing of obtaining, from an interference fringe image that is a two-dimensional distribution of intensity of interference fringes of object light as illumination light diffracted by an object to be observed and reference light without passing through the object to be observed, a phase difference image that is a two-dimensional distribution of a phase difference between the object light and the reference light, and obtaining a shape of the object to be observed based on the phase difference image, the operation program causing a computer to function as an acquisition unit that acquires object-related information regarding the object to be observed, a readout unit that reads out, from a storage unit in which the object-related information and a shape profile indicating the shape of the object to be observed are stored in association with each other, the shape profile corresponding to the object-related information acquired in the acquisition unit, and a phase connection unit that performs phase connection with respect to the phase difference image with reference to the shape profile read out in the readout unit.
According to the technique of the present disclosure, it is possible to provide an information processing apparatus, a method for operating an information processing apparatus, and an operation program for an information processing apparatus capable of reducing a processing time of phase connection without needing for a special mechanism for measuring a rough shape of an object to be observed and without spending an excess measurement time.
Exemplary embodiments according to the technique of the present disclosure will be described in detail based on the following figures, wherein:
In
In
As shown in
A pixel value of each pixel of the interference fringe image 34 is intensity I of the interference fringes 33. That is, the interference fringe image 34 is a two-dimensional distribution of the intensity I of the interference fringes 33. The intensity I of the interference fringes 33 is represented by Expression (1) described below where a phase difference between the object light 30 and the transmitted light 31 is φ.
I(X,Y)=A+B cos φ(X,Y) (1)
A and B are constants. (X,Y) is an X coordinate and a Y coordinate of each pixel of the interference fringe image 34.
As also shown in
As shown in
In this way, a method of outputting the interference fringe image 34 while shifting the optical path difference between the object light 30 and the transmitted light 31 by a specified amount is referred to as a phase shift method. In particular, a method of setting the specified amount to π/2 and the four interference fringe images 34 in total as described above is referred to as a four-step method. In the following description, unless there is no need for particular distinction, the first interference fringe image 34A, the second interference fringe image 34B, the third interference fringe image 34C, and the fourth interference fringe image 34D are collectively referred to as the interference fringe images 34.
Intensity of I_1(X,Y) of the interference fringes 33 shown in the first interference fringe image 34A is represented by Expression (2) described below.
I_1(X,Y)=A+B cos φ(X,Y) (2)
Similarly, intensity I_2(X,Y) of the interference fringes 33 shown in the second interference fringe image 34B, intensity I_3(X,Y) of the interference fringes 33 shown in the third interference fringe image 34C, and intensity I_4(X,Y) of the interference fringes 33 shown in the fourth interference fringe image 34D are represented by Expressions (3), (4), and (5) described below.
I_2(X,Y)=A+B cos{φ(X,Y)+(π/2)} (3)
I_3(X,Y)=A+B cos{φ(X,Y)+π} (4)
I_4(X,Y)=A+B cos{φ(X,Y)+(3π/2)} (5)
Expressions (3), (4), and (5) are rewritten to Expressions (3A), (4A), and (5A) described below.
I_2(X,Y)=A−B sin φ(X,Y) (3A)
I_3(X,Y)=A−B cos φ(X,Y) (4A)
I_4(X,Y)=A+B sin φ(X,Y) (5A)
In
{I_4(X,Y)−I_2(X,Y)}/{I_1(X,Y)−I_3(X,Y)}=tan φ(X,Y) (6)
Accordingly, a phase difference φ(X,Y) between the object light 30 and the transmitted light 31 is represented by Expression (7) described below.
φ(X,Y)=arctan{I_4(X,Y)−I_2(X,Y)}/{I_1(X,Y)−I_3(X,Y)} (7)
That is, the phase difference φ(X,Y) can be obtained from simple computation using the intensity I_1(X,Y) to I_4(X,Y) of the interference fringes 33 of the first interference fringe image 34A to the fourth interference fringe image 34D. The phase difference φ(X,Y) obtained in this manner is set as a pixel value of a pixel corresponding to the pixel of the interference fringe image 34, whereby a phase difference image 40 that is a two-dimensional distribution of the phase difference φ is obtained.
As shown in
H(X,Y)={λφ(X,Y)/4π}−h (8)
λ is a wavelength of the coherent light 23, and is, for example, 640 nm. h is a height (hereinafter, simply referred to as a culture surface height) 83 (see
In
The storage device 50 is an example of a “storage unit” according to the technique of the present disclosure. The storage device 50 is a hard disk drive incorporated in the computer that configures the information processing apparatus 10 or connected to the computer through a cable or a network. Alternatively, the storage device 50 is a disk array where a plurality of hard disk drives are connected. In the storage device 50, a control program, such as an operating system, various application programs, various kinds of data accompanied with such programs, and the like are stored. A solid state drive may be used instead of the hard disk drive.
The memory 51 is a work memory on which the CPU 52 executes processing. The CPU 52 loads the programs stored in the storage device 50 to the memory 51 and executes processing depending on the programs, thereby integrally controlling each unit of the computer. The CPU 52 is an example of a “processor” according to the technique of the present disclosure.
The communication unit 53 is a network interface that performs transmission control of various kinds of information through a network, such as a local area network (LAN). The display 54 displays various screens. The computer that configures the information processing apparatus 10 receives an input an operation instruction from the input device 55 through various screens. The input device 55 is a keyboard, a mouse, a touch panel, and the like.
In
In a case where the operation program 60 is started, the CPU 52 of the computer that configures the information processing apparatus 10 functions as a read write (hereinafter, abbreviated as RW) controller 70, an acquisition unit 71, a processing unit 72, and a display controller 73 in cooperation with the memory 51 and the like.
The RW controller 70 controls storage of various kinds of data in the storage device 50 and readout of various kinds of data in the storage device 50. For example, the RW controller 70 receives the interference fringe image 34 from the measurement apparatus 11 and stores the interference fringe image 34 in the storage device 50. The RW controller 70 reads out the interference fringe image 34 from the storage device 50 and outputs the interference fringe image 34 to the processing unit 72.
The acquisition unit 71 acquires object-related information 80 and culture surface height-related information 81 that are input from a user through the input device 55. The object-related information 80 is information regarding the cell 12 that is an object to be observed. The culture surface height-related information 81 is information regarding the culture surface height 83. The acquisition unit 71 outputs the object-related information 80 and the culture surface height-related information 81 to the RW controller 70.
The RW controller 70 reads out a shape profile 82 corresponding to the object-related information 80 from the acquisition unit 71, from the shape profile table 61 of the storage device 50. That is, the RW controller 70 is an example of a “readout unit” according to the technique of the present disclosure. The RW controller 70 outputs the shape profile 82 to the processing unit 72.
The RW controller 70 reads out the culture surface height 83 corresponding to the culture surface height-related information 81 from the acquisition unit 71, from the culture surface height table 62 of the storage device 50. The RW controller 70 outputs the culture surface height 83 to the processing unit 72.
The processing unit 72 calculates the height H(X,Y) of the cell 12 based on the interference fringe image 34, the shape profile 82, and the culture surface height 83. The processing unit 72 outputs the height calculation result 63 to the RW controller 70.
The RW controller 70 stores the height calculation result 63 from the processing unit 72 in the storage device 50. The RW controller 70 reads out the height calculation result 63 from the storage device 50 and outputs the height calculation result 63 to the display controller 73.
The display controller 73 controls the display of various screens on the display 54. Various screens include an information input screen 90 (see
As shown in
The user operates the pull-down menus 91 to 93 to select desired options and selects an OK button 94. With this, the object-related information 80 and the culture surface height-related information 81 are acquired in the acquisition unit 71. In
In
The shape of the cell 12 is different depending on the type of cell 12. For example, in a case of a red blood cell of a human, a central portion is recessed by several μm, and in a case of a mesenchymal stem cell, a region of a nucleus is thick, about 10 μm, and a region of cytoplasm is thin, about several μm. In a case where the culture condition, such as the number of days of culture, is different, the shape of the cell 12 is also naturally different. The shape profile 82 is data representing such a feature of the shape of the cell 12.
In
In
As shown in
The phase difference φ(X,Y) that is the pixel value of the phase difference image 40 is a function of arctan as shown in Expression (7). For this reason, as shown in a graph of a phase difference φ(X,Ys) in a certain row Ys of the phase difference image 40, the phase difference φ(X,Y) is obtained to be folded in a range of −π to π, and a phase jump of about ±2π occurs in places. The phase connection unit 101 performs the phase connection for connecting the phase difference φ(X,Y) where there is such a phase jump, with respect to the phase difference image 40. In this case, the phase connection unit 101 refers to the shape profile 82. The phase connection unit 101 outputs the phase difference image 40P after the phase connection to the height calculation unit 102.
As shown in
φE=φS+ΣΔφ (9)
Note that, in a case where the phase difference of the pixel 110 on the pixel 110S side among the two adjacent pixels 110 connected by the path 111 is φi, and the phase difference of the pixel 110 on the pixel 110E side is φj, Δφ changes depending on the value of φj−φi as Expressions (10), (11), and (12). That is,
In a case where −π<φj−φi≤π,Δφ=φj−φi (10)
In a case where φj−φi≤−π,Δφ=φj−φi+2π (11)
In a case where φj−φi>π,Δφ=φj−φi−2π (12)
In the case of Expression (10), a phase jump does not occur in the two adjacent pixels 110 connected by the path 111. For this reason, φj− φi is employed as Δφ as it is. In contrast, in the cases of Expressions (11) and (12), a phase jump occurs in the two adjacent pixels 110 connected by the path 111. For this reason, 2π is added to or subtracted from φj− φi.
In this way, the phase connection is processing of successively adding the difference Δφ between the phase differences φ of the two adjacent pixels 110 connected by the path 111. For this reason, miscalculation of Δφ in the middle of the path 111 influences the calculation result of the phase difference φE. Accordingly, actually, the phase connection unit 101 selects the path 111 while avoiding a place 120 where Δφ is likely to be miscalculated as shown in
In
As shown in
Next, the operations of the above-described configuration will be described referring to a flowchart of
First, the object-related information 80 and the culture surface height-related information 81 input from the user through the information input screen 90 shown in
The RW controller 70 reads out the shape profile 82 corresponding to the object-related information 80 from the acquisition unit 71, from the shape profile table 61 of the storage device 50. Similarly, the RW controller 70 reads out the culture surface height 83 corresponding to the culture surface height-related information 81 from the acquisition unit 71, from the culture surface height table 62 of the storage device 50 (Step ST110). The shape profile 82 and the culture surface height 83 are output from the RW controller 70 to the processing unit 72. Step ST110 is an example of a “readout step” according to the technique of the present disclosure.
In the measurement apparatus 11, as shown in
The RW controller 70 reads out the first interference fringe image 34A to the fourth interference fringe image 34D from the storage device 50 and outputs the first interference fringe image 34A to the fourth interference fringe image 34D to the processing unit 72 (Step ST130). In the processing unit 72, as shown in
As shown in
As shown in
The RW controller 70 reads out the height calculation result 63 from the storage device 50 and outputs the height calculation result 63 to the display controller 73. Then, the display controller 73 displays the measurement result display screen 130 shown in
As described above, the acquisition unit 71 of the CPU 52 of the information processing apparatus 10 acquires the object-related information 80 regarding the cell 12 that is an object to be observed. The RW controller 70 reads out the shape profile 82 corresponding to the acquired object-related information 80 from the shape profile table 61 of the storage device 50. The phase connection unit 101 of the processing unit 72 performs the phase connection with respect to the phase difference image 40 with reference to the read-out shape profile 82 before obtaining the height H(X,Y) of the cell 12. For this reason, unlike JP2006-284186A of the related art, it is possible to reduce a processing time of the phase connection without needing a special mechanism for measuring a rough shape of the cell 12 and without spending an excess measurement time.
The CPU 52 performs control for displaying the height calculation result 63 of the cell 12. For this reason, it is possible to notify the user of the height calculation result 63.
A field of cell culture is recently highlighted due to the appearance of an induced pluripotent stem (iPS) cell or the like. For this reason, there is a demand for a technique for analyzing the cell 12 during culture in detail. In the technique of the present disclosure, an object to be observed is the cell 12 during culture. Accordingly, it can be said that the technique of the present disclosure is a technique capable of meeting a recent demand.
The CPU 52 acquires the culture surface height-related information 81 regarding the culture surface height 83. For this reason, it is possible to determine the reference of the height H(X,Y) of the cell 12 to the culture surface 13B, and to more accurately calculate the height H(X,Y) of the cell 12.
The object-related information 80 is the type of the cell 12 and the culture condition of the cell. For this reason, it is possible to obtain the shape profile 82 appropriate for the type of the cell 12 and the culture condition of the cell.
The culture condition of the cell included in the object-related information is not limited to the number of days of culture illustrated. Like object-related information 140 shown in
The culture condition of the cell included in the object-related information may include at least any one of the number of days of culture, the type of the culture medium, the temperature of the culture environment, or the carbon dioxide concentration of the culture environment illustrated above. In addition to these, a type of a culture solution, pH, osmotic pressure, an oxygen concentration of the culture environment, and the like may be further added.
The culture surface height-related information 81 may be a model number or the like of the culture vessel 13. The culture surface height-related information 81 may be the culture surface height 83 itself. In this case, the culture surface height table 62 is not required.
In the first embodiment described above, although an example where the phase connection is performed with respect to the entire phase difference image 40 has been shown, the technique of the present disclosure is not limited thereto. Like a second embodiment shown in
In
In this way, in the second embodiment, the region extraction unit 150 extracts the presence region 151 of the cell 12 from the interference fringe image 34, and the phase connection unit 101 selectively performs the phase connection with respect to the presence region 151. For this reason, it is possible to further reduce the processing time of the phase connection.
In the first embodiment described above, although the three-dimensional color map 133 has been illustrated as a display method of the height calculation result 63, the technique of the present disclosure is not limited thereto. Like a third embodiment shown in
In
The generation unit 160 selects one best focused reproduction image 161 from among a plurality of reproduction images 161 of the reproduction image group 165. The generation unit 160 outputs the selected reproduction image 161 to the display controller 73. As a method of selecting the best focused reproduction image 161, a method of calculating a contrast value of each of a plurality of reproduction images 161 and selecting the reproduction image 161 having the highest contrast value as the best focused reproduction image 161, or the like can be employed.
As shown in
In this way, in the third embodiment, the generation unit 160 generates the reproduction image 161 from the interference fringe image 34, and the display controller 73 displays the height calculation result 63 on the reproduction image 161 in a superimposed manner. For this reason, the user can confirm the height calculation result 63 in conjunction with the reproduction image 161, and the analysis of the cell 12 is advanced.
The culture surface height-related information 81 may not necessarily be acquired. In a case where the culture surface height-related information 81 is not acquired, the height H(X,Y)+h including the culture surface height 83 is calculated as the height of the cell 12.
The shape of the object to be observed is not limited to the height H(X,Y) along the Z direction illustrated. The widths of the X direction and the Y direction may be used instead of or in addition to the height H(X,Y).
Although the four-step method has been described as an example of the phase shift method, the technique of the present disclosure is not limited thereto. A three-step method, a five-step method, a seven-step method, or the like may be used. The technique of the present disclosure is not limited to the phase shift method. A vertical scanning method that applies white light as illumination light and captures a plurality of interference fringe images 34 while moving an objective lens in the Z direction may be used. Alternatively, a method that inclines a reference plane of reference light to produce carrier fringes may be used.
The object to be observed is not limited to the cell 12 illustrated. A bacterium, a virus, or the like may be applied as the object to be observed. The object light is not limited to the object light 30 transmitted through the object to be observed, and may be object light reflected by the object to be observed. The coherent light 23 from the light source 20 may be split into light for object light and light for reference light using a beam splitter and the like. The illumination light may not be the coherent light 23, and any light may be applied as long as light produces interference fringes 33 to withstand observation.
The hardware configuration of the computer that configures the information processing apparatus 10 can be modified in various ways. For example, the information processing apparatus 10 can also be configured with a plurality of computers separated as hardware for the purpose of improvement of processing capacity and reliability. For example, the functions of the RW controller 70, the acquisition unit 71, and the display controller 73 and the function of the processing unit 72 are distributed to two computers. In this case, the information processing apparatus 10 is configured with two computers.
In this way, the hardware configuration of information processing apparatus 10 can be appropriately changed depending on required performance, such as processing capacity, safety, or reliability. Not only hardware but also an application program, such as the operation program 60, can be of course duplicated or distributed and stored in a plurality of storage devices for the purpose of ensuring safety and reliability.
In each embodiment described above, for example, as the hardware structures of processing units that execute various kinds of processing, such as the RW controller 70, the acquisition unit 71, the processing unit 72, the display controller 73, the phase difference image generation unit 100, the phase connection unit 101, the height calculation unit 102, the region extraction unit 150, and the generation unit 160, various processors described below can be used. Various processors include a programmable logic device (PLD) that is a processor capable of changing a circuit configuration after manufacture, such as a field programmable gate array (FPGA), a dedicated electric circuit that is a processor having a circuit configuration dedicatedly designed for executing specific processing, such as an application specific integrated circuit (ASIC), and the like, in addition to the CPU 52 that is a general-purpose processor configured to execute software (operation program 60) to function as various processing units, as described above.
One processing unit may be configured with one of various processors described above or may be configured with a combination of two or more processors (for example, a combination of a plurality of FPGAs and/or a combination of a CPU and an FPGA) of the same type or different types. A plurality of processing units may be configured with one processor.
As an example where a plurality of processing units are configured with one processor, first, as represented by a computer, such as a client or a server, there is a form in which one processor is configured with a combination of one or more CPUs and software, and the processor functions as a plurality of processing units. Second, as represented by system on chip (SoC) or the like, there is a form in which a processor that implements all functions of a system including a plurality of processing units into one integrated circuit (IC) chip is used. In this way, various processing units may be configured using one or more processors among various processors described above as a hardware structure.
In addition, as the hardware structure of various processors, more specifically, an electric circuit (circuitry), in which circuit elements, such as semiconductor elements, are combined, can be used.
In the technique of the present disclosure, various embodiments and various modification examples described above can also be appropriately combined. The technique of the present disclosure is not limited to the above-described embodiments, and various configurations can be of course employed without departing from the spirit and scope of the technique of the present disclosure. In addition to the program, the technique of the present disclosure extends to a storage medium that stores the program in a non-transitory manner. The content of the above description and the content of the drawings are detailed description of portions according to the technique of the present disclosure, and are merely examples of the technique of the present disclosure. For example, the above description relating to configuration, function, operation, and advantageous effects is description relating to configuration, function, operation, and advantageous effects of the portions according to the technique of the present disclosure. Thus, it is needless to say that unnecessary portions may be deleted, new elements may be added, or replacement may be made to the content of the above description and the content of the drawings without departing from the gist of the technique of the present disclosure. Furthermore, to avoid confusion and to facilitate understanding of the portions according to the technique of the present disclosure, description relating to common technical knowledge and the like that does not require particular description to enable implementation of the technique of the present disclosure is omitted from the content of the above description and the content of the drawings.
In the specification, “A and/or B” is synonymous with “at least one of A or B”. That is, “A and/or B” may refer to A alone, B alone, or a combination of A and B. Furthermore, in the specification, a similar concept to “A and/or B” applies to a case in which three or more matters are expressed by linking the matters with “and/or”.
All cited documents, patent applications, and technical standards described in the specification are incorporated by reference in the specification to the same extent as in a case where each individual cited document, patent application, or technical standard is specifically and individually indicated to be incorporated by reference.
Number | Date | Country | Kind |
---|---|---|---|
2020-006381 | Jan 2020 | JP | national |
This application is a continuation application of International Application No. PCT/JP2020/038539 filed on Oct. 12, 2020, the disclosure of which is incorporated herein by reference in its entirety. Further, this application claims priority from Japanese Patent Application No. 2020-006381 filed on Jan. 17, 2020, the disclosure of which is incorporated herein by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/JP2020/038539 | Oct 2020 | US |
Child | 17860511 | US |