Patent Application
20240188825
The present application relates to laser scanning imaging technologies and, in particular, to a laser scanning optical imaging method, a system, a storage medium and a computer program.
Cancer related cases are increasing in modern lifestyle. A laser scanning imaging system has a very bright prospect in the application of early detection, screening and diagnosis of cancer due to its ability to provide in real time tissue information at subcellular resolution. The laser scanning imaging system includes main parts such as a laser, a laser scanning device, an objective lens, a pinhole, and an optical sensor.
In the laser scanning imaging process, the laser of the laser scanning imaging system emits a laser beam. The laser scanning device deflects the laser beam for scanning, the objective lens focuses the laser beam on the biological tissues, the irradiated biological tissues generate fluorescence information, and the fluorescence information passes through the objective lens, the laser scanning device and the pinhole to reach the optical sensor and is received by the optical sensor, which forms the fluorescence information of the irradiated area.
A resonant galvanometer is used as a fast mirror in the laser scanning device and usually performs a nonlinear motion. Along with the motion of the resonant galvanometer, the laser scanning device outputs a galvanometer scanning synchronization signal, which is a square wave representing a change in the direction of the motion of the galvanometer. A hardware clock board is typically used to receive the galvanometer scanning synchronization signal and generate a clock signal for sampling the fluorescence signal.
Currently, the clock signal is generated by the hardware clock board, in this way, the clock signal is generated only in a fixed portion of the motion of the resonant galvanometer that is relatively linear, so that the corresponding imaging range of the resonant galvanometer is relatively small and the imaging position is fixed, and the characteristics of the clock signal generated in this way are fixed and cannot be flexibly altered with the change of the scanning range, which results in a relatively small and fixed area of the fluorescence image. Even if the imaging area is subsequently changed by means of amplification processing and the like, the actual resolution of the fluorescence image still can't be increased.
In view of the above problem existing in the prior art, the present application provides a laser scanning imaging method, a system, a storage medium and a computer program, which can realize that the generated clock signal can be flexibly altered with the change in the range of the effective area, so as to present a user with a scanning imaging result that meets requirements.
An embodiment of the present application provides a laser scanning imaging method, including: determining parameter information related to an effective area, where the effective area includes an imaging area satisfying a preset condition; receiving a galvanometer scanning synchronization signal generated by a driving unit; generating, based on the galvanometer scanning synchronization signal and the parameter information related to the effective area, a clock signal; and sampling, according to the clock signal, a fluorescence signal received through galvanometer scanning, and obtaining fluorescence image information of the effective area.
An embodiment of the present application also provides a laser scanning imaging system, including: a galvanometer; and a processor, connected to the galvanometer, for executing the above-mentioned laser scanning imaging method.
An embodiment of the present application further provides a storage medium, storing a computer program, and the computer program, when executed by a processor, executes the laser scanning imaging method according to any embodiment of the present application.
An embodiment of the present application further provides a computer program, and the computer program, when executed by a processor, executes the laser scanning imaging method according to any embodiment of the present application.
Compared with the prior art, the present application generates a clock signal based on the received galvanometer scanning synchronization signal and parameter information related to the effective area, samples a fluorescence signal received from the galvanometer scanning according to the clock signal, obtains the fluorescence image information of the effective area, and realizes that the generated clock signal can be flexibly altered with the changes in the range of the effective area, so as to obtain the fluorescence image of the effective area, and presents the user with the scanning imaging result that satisfies the requirement.
In the accompanying drawings, which are not necessarily drawn to scale, the same reference signs may depict similar components in different figures. The same reference signs with letter suffixes or different letter suffixes may represent different instances of similar components. The drawings illustrate various embodiments generally by way of example rather than limitation, and are used in conjunction with the description and the claims to illustrate the disclosed embodiments. Where appropriate, the same reference signs are used in all of the drawings to refer to the same or similar parts. Such embodiments are illustrative and are not intended to be exhaustive or exclusive embodiments of the present device or method.
With reference to the accompanying drawings, various embodiments as well as features of the present application are described.
It should be understood that various modifications can be made to the embodiments of the present application. Accordingly, the foregoing specification should not be regarded as limiting, but merely as examples of embodiments. Those skilled in the art will think of other modifications within the scope and spirit of this application.
The accompanying drawings, which are included in and form a part of the specification, illustrate embodiments of the present application and are used to explain the principles of the application in conjunction with the general description of the application given above and the detailed description of the embodiments given below.
These and other features of the present application will become apparent by the following description of alternative forms of the embodiments given as non-limiting examples with reference to the accompanying drawings.
It should also be understood that although the present application has been described with reference to some specific embodiments, many other equivalent forms of the present application can be realized with certainty by those skilled in the art.
The aforementioned and other aspects, features and advantages of the present application will become more apparent in view of the following detailed description when taken in conjunction with the accompanying drawings.
Specific embodiments of the present application are described with reference to the accompanying drawings. However, it should be understood that the embodiments are merely examples of the present application, which may be implemented in a variety of ways. Well-known and/or repeated functions and structures are not described in detail to avoid unnecessary or redundant details that would lead to ambiguity of the present application. Accordingly, the specific structural and functional details invented herein are not intended to be limiting, but are merely intended to serve as a basis for the claims and as a representative basis for teaching those skilled in the art to substantially utilize the application in a variety of ways with any suitable detailed structure.
This specification may use the phrases “in one embodiment”, “in another embodiment”, “in yet another embodiment”, or “in other embodiments”, which refer to one or more of the same or different embodiments according to the present application.
Embodiments of the present application provide a laser scanning imaging method that can be applied to a laser scanning imaging system. As shown in
Specifically, the galvanometer may be a linear galvanometer, a nonlinear galvanometer, or a composition of both. If the galvanometer is a linear galvanometer or a nonlinear galvanometer, the one-dimensional fluorescence image information of the effective area is obtained by the steps S101 to S104; if the galvanometer includes a linear galvanometer and a nonlinear galvanometer, the two-dimensional fluorescence image information of the effective area is obtained by the steps S101 to S104. The nonlinear galvanometer is the resonant galvanometer mentioned before.
Specifically, an imaging area that satisfies a preset condition, for example an area that needs fluorescence imaging, may be set by a user. The effective area can be understood as the imaging area that needs to be displayed for the user. It should be noted that the effective area can be the entire scanning area of the laser scanning imaging system, or a part of the entire scanning area, depending on the user's requirement.
Specifically, the driving unit may be a nonlinear driving unit. The nonlinear galvanometer in the laser scanning imaging system performs natural oscillation and nonlinear motion after powered up. The nonlinear driving unit generates a galvanometer scanning synchronization signal of nonlinear galvanometer, and the galvanometer scanning synchronization signal synchronizes with the motion of the nonlinear galvanometer, where the specific expression of the galvanometer scanning synchronization signal may be a square wave signal. The laser scanning imaging system receives the galvanometer scanning synchronization signal and generates a clock signal based on the galvanometer scanning synchronization signal and the parameter information related to the effective area. In this way, the generated clock signal can be flexibly changed with the changes in the range of the effective area of imaging, which can meet the user's imaging needs for various areas and resolutions and avoid the problem in the prior art that the hardware clock board can only generate a fixed clock signal for fluorescence signal sampling through receiving the galvanometer scanning synchronization signal, resulting in only a fixed area for imaging.
Through the solution of the present application, the user can set the imaging area (i.e., the effective area) according to the imaging requirements, and it can be realized that the generated clock signal can be flexibly altered with the changes in the range of the effective area, and then the sampling is performed based on the clock signal to obtain the fluorescence image of the effective area, so as to present the user with a scanning imaging result that satisfies the requirements.
In some embodiments, the parameter information includes area range information of the effective area.
In some embodiments, the area range information at least includes a starting position, an end position, and the number of pixel points.
Specifically, for the imaging area is displayed for the user according to the requirements, a starting position, an end position and the number of pixel points of the effective area may be determined, where the imaging area displayed for the user is the area with an imaging requirement of the user. The above mentioned starting position, the end position and the number of pixels are parameter information related to the effective area, where the starting position and the end position may be expressed by means of coordinates (one-dimensional, two-dimensional or three-dimensional coordinates) or a deflection angle of the galvanometer.
Alternatively, for example, when the user needs to zoom in on a result of the scanning imaging, the zoomed in result of the scanning imaging is displayed by increasing the number of pixels in the effective area without changing the preset values of the starting position and the end position, or by shortening the distance between the starting position and the end position without changing the number of pixels, thus realizing that the displayed result of the scanning imaging is zoomed in. Alternatively, for example, when the user needs to zoom out on the result of the scanning imaging, the zoomed out result of the scanning imaging is displayed by decreasing the number of pixels in the effective area without changing the preset values of the starting position and the end position, or by increasing the distance between the starting position and the end position without changing the number of pixels, thus realizing that the displayed result of the scanning imaging is zoomed out. The number of pixels refers to the number of pixel points.
A nonlinear galvanometer is taken as an example of the galvanometer in the following.
In some embodiments, as shown in
In some embodiments, the galvanometer motion parameter information includes at least one of the followings: motion speed information of the galvanometer, motion acceleration information of the galvanometer, motion frequency information of the galvanometer, deflection angle information of the galvanometer, spatial position information at an edge of the galvanometer, and spatial position information of a mirror of the galvanometer. The motion frequency information of the galvanometer can also be understood as the swing frequency information of the galvanometer.
In some embodiments, the image scanning parameter information includes at least one of the followings: sampling position information corresponding to each of the pixel points, sampling moment information corresponding to each of the pixel points, and sampling time information corresponding to each of the pixel points.
In some embodiments, the scanning time information includes at least any two scanning moments corresponding to the effective area. Alternatively, when the scanning time information includes any two scanning moments corresponding to the effective area, the any two scanning moments corresponding to the effective area may be a starting moment and an end moment, a starting moment and any one scanning moment other than the starting moment, or an end moment and any one scanning moment other than the end moment.
In some embodiments, the area range information at least includes the starting position, the end position, and the number of pixel points. The following takes the galvanometer motion parameter information including the motion speed information of the galvanometer, and the image scanning parameter information including the sampling time information corresponding to each pixel point, as an example, as shown in
Specifically, in this embodiment, the scanning time information includes the starting moment and the end moment.
Specifically, when the starting position and the end position are expressed in the form of one-dimensional coordinates, the corresponding coordinate values can be transformed into the corresponding deflection angle θ of the nonlinear galvanometer by the formula x=d*tan θ, where d is a galvanometer distance parameter, such as a perpendicular distance from the center of the nonlinear galvanometer to the scanning plane, and x is the coordinate value of the current pixel point. Then, the galvanometer motion speed information corresponding to each pixel point can be obtained by taking the derivative of the galvanometer deflection angle θ corresponding to each pixel point.
Specifically, when the starting position and the end position are expressed in the form of the deflection angle θ of the galvanometer, the motion speed of the nonlinear galvanometer corresponding to each pixel point in the effective area may be calculated according to a motion function that may be a trigonometric function describing simple harmonic oscillation. For a more accurate description, the function may be obtained by measurement, and the parameters used for the calculation in the motion function at least includes the maximum motion position of the nonlinear galvanometer (e.g., the maximum deflection angle), the motion frequency of the nonlinear galvanometer (which is reciprocal relationship to the oscillation period of the nonlinear galvanometer), where the deflection angle of the galvanometer can also be understood as the motion angle of the galvanometer, then the maximum deflection angle of the galvanometer can be the maximum motion angle of the galvanometer.
Alternatively, it can be concluded from the above mentioned that the motion speed of the nonlinear galvanometer corresponding to each pixel point in the effective area is V(Pn), where V is the speed, P is the pixel point, n ranges from 1 to N, and N is the number of pixel points in the effective area. Then taking the reciprocal of the motion speed of the nonlinear galvanometer corresponding to each pixel point, the sampling time information
corresponding to each pixel point can be obtained. Next, taking the sampling time information corresponding to the pixel point P(m) with the largest speed as a standard, the sampling time information corresponding to each pixel point is quantized to obtain the quantized sampling time information corresponding to each pixel point; then by performing quantization processing on the above quantized sampling time information, the starting moment and the end moment, a clock signal corresponding to the effective area is generated.
In some embodiments, the area range information at least includes a starting position, an end position, and the number of pixel points. The following takes the galvanometer motion parameter information including the galvanometer deflection angle information, and the image scanning parameter information including the sampling time information corresponding to each pixel point, as an example, as shown in
Specifically, in this embodiment, the scanning time information includes a starting moment and an end moment.
Specifically, according to the starting position, the end position and the number of pixel points, a relative coordinate of each pixel point in the effective area relative to the starting position is determined as x(Pn), where x is the relative coordinate, P is a pixel point, n ranges from 1 to N, and N is the number of pixel points in the effective area. By means of the galvanometer distance parameter, such as the perpendicular distance from the center of the nonlinear galvanometer to the scanning plane, the deflection angel information of the nonlinear galvanometer corresponding to each pixel point is determined as θ(Pn). According to the deflection angle information θ(Pn) of the nonlinear galvanometer corresponding to each pixel point, the sampling time information T(Pn) corresponding to each pixel point can be obtained by utilizing the description formula of the deflection angle of the nonlinear galvanometer. Then by performing quantization processing on the sampling time information, the starting moment and the end moment, the clock signal associated with the effective area can be generated.
The above specific scheme of generating the clock signal is illustrative and not limiting. The scheme of generating the clock signal by the galvanometer scanning synchronization signal and the parameter information related to effective area is within the protection scope of this application.
In some embodiments, the area range information at least includes the starting position and the end position, where the step S203: determining, based on the galvanometer scanning synchronization signal and the area range information, scanning time information corresponding to the effective area includes:
Specifically, the following formula can be used to calculate the starting moment and end moment corresponding to the effective area: θ=θmax*P(f*t), where P is a function describing the change of the angle of the nonlinear galvanometer over time, and this function can be a trigonometric function describing the simple harmonic oscillation; f is the motion frequency of the nonlinear galvanometer, which can also be called as the swing frequency of the nonlinear galvanometer and can be determined based on the period of the galvanometer scanning synchronization signal; and θmax is the maximum deflection angle (i.e., the maximum motion angle) of the nonlinear galvanometer. The above formula describes the one-to-one correspondence between the motion angle of the nonlinear galvanometer and time. Therefore, the starting moment corresponding to the effective area can be calculated from the starting position, and the end moment corresponding to the effective area can be calculated from the end position.
In some embodiments, the galvanometer includes a first galvanometer and a second galvanometer, where the first galvanometer and the second galvanometer are set vertically, and the laser scanning method further includes:
Specifically, the driving unit may be a nonlinear driving unit. The first galvanometer may be the nonlinear galvanometer as previously described, and the second galvanometer may be the linear galvanometer as previously described. The first direction may be a direction perpendicular to the second direction, e.g., the first direction is an X-axis direction in the rectangular coordinate system, and the second direction is a Y-axis direction in the rectangular coordinate system, that is, the above-described non-linear galvanometer can move in the X-axis direction, and the above-described linear galvanometer can move in the Y-axis direction. Alternatively, the X-axis direction can be a horizontal direction and the Y-axis direction can be a vertical direction.
Taking the first direction being the horizontal direction, the second direction being the vertical direction, and the effective area being a part of the entire scanning area as an example, the following explains the process of obtaining fluorescence image information in the present embodiment: from a scanning starting position of the entire scanning area, the nonlinear galvanometer is first driven to move along the horizontal direction of the scanning starting position, and when the nonlinear galvanometer is moved to a maximum scanning position along the horizontal direction, the linear galvanometer is driven to move along the vertical direction. When the linear galvanometer moves to the next position along the vertical direction, the nonlinear galvanometer is driven to move along the horizontal direction. The above motion process is repeated until the first galvanometer and the second galvanometer cooperate to complete the motion in the entire scanning area. During the above motion process of the two galvanometers, the fluorescence signal in the effective area is sampled according to the generated clock signal related to the effective area, and the two-dimensional fluorescence image information of the effective area can be obtained.
By the scheme of the present embodiment, the two-dimensional fluorescence image information can ultimately be obtained by the two galvanometers moving in two mutually perpendicular directions respectively, which further improves the accuracy of the obtained fluorescence image information, obtains image information with more dimensions, and provides a guarantee of the accuracy of the subsequent diagnosis.
In some embodiments, the determining, based on the galvanometer scanning synchronization signal and the effective area, a control signal for the second galvanometer includes: determining, based on the galvanometer scanning synchronization signal and the parameter information related to the effective area, the control signal for the second galvanometer.
Specifically, the parameter information may include a starting position, an end position, and the number of pixel points, these parameters are related to the effective area, and based on these information, the motion of the linear galvanometer can be determined, thereby realizing the control of the motion thereof, so as to ultimately obtain a fluorescence image of the effective area.
Embodiments of the present application also provide a laser scanning imaging system including a galvanometer and a processor connected to the galvanometer, where the processor is used to execute the laser scanning imaging method provided in any one of the above embodiments.
The processor 510, the memory 520, the input device 530 and the output device 540 may be connected via a bus or otherwise. In
The memory 520 serves as a non-volatile computer-readable storage medium that can be used to store a non-volatile software program, a non-volatile computer-executable program and a module, such as the program instructions/module corresponding to the laser scanning imaging method in embodiments of the present application. The processor 510 executes various functional applications and data processing of the server by running the non-volatile software program, instructions, and module stored in the memory 520, that is, realizing the laser scanning imaging method in the above embodiments.
The memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk memory device, a flash memory device, or other non-volatile solid state memory device. Example of the aforementioned network includes, but not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and a combination thereof.
With the schemes of the present application, the user can set the effective area according to the imaging requirement, and realize that the generated clock signal can be flexibly altered with the change of the range of the effective area, so as to perform the sampling based on of the clock signal to obtain the fluorescence image within the effective area, presenting the user with the scanning imaging result that satisfies the requirement.
Embodiments of the present application also provide a non-volatile storage medium storing a computer program, which is executed by a processor to realize the steps of the laser scanning imaging method according to any one embodiment of the present application. A person of ordinary skill in the art may understand that realizing all or some of the steps carried by the method of the above embodiments can be accomplished by a program to instruct the associated hardware, and the program may be stored in a medium, and when the program is executed, one of the steps of the method embodiments or a combination thereof is realized. The medium may be a storage medium or a signal medium. The storage medium may be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the foregoing, but is not limited thereto; specifically, it may be: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM) (or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any combination of the foregoing. The signal medium may include, but is not limited to, an electrical signal medium, an optical signal medium, a radio wave (electromagnetic wave) medium, or any combination of the foregoing.
In addition, the various functional units in various embodiments of the present application may be integrated in a single processing module, or the individual unit may physically exist separately, or two or more units may be integrated in a single module. The integrated modules may be implemented either in the form of hardware or in the form of software functional modules. The integrated module may also be stored in a computer-readable storage medium if it is realized in the form of a software function module and sold or used as a separate product.
The above mentioned storage medium may be a read-only memory, a disk or a CD-ROM, etc.
It should be noted that the various units according to various embodiments of the present application may be realized as computer-executable instructions stored on a memory which, when executed by a processor, can realize the corresponding steps; or as hardware with corresponding logical computational capabilities; or as a combination of software and hardware (firmware). In some embodiments, the processor may be implemented as any one of a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a digital signal processor (DSP) chip, System on Chip (SOC), a microprocessor unit (MPU) (for example but not limited to, the Cortex), and the like.
It should be noted that in the various components of the system of the present application, the components are logically divided according to the functions to be realized thereof, however, the present application is not limited to that. The various components may be re-divided or combined as desired, for example, some components may be combined into a single component, or some components may be optically broken down into more sub-components.
The various component embodiments of the present application may be implemented in hardware, or in software modules running on one or more processors, or in combination thereof. Those skilled in the art should understand that microprocessors or digital signal processors (DSPs) may be used in practice to implement some or all of the functionality of some or all of the components of a system according to embodiments of the present application. The present application may also be implemented as an apparatus or device program (e.g., computer program and computer program product) for performing some or all of the methods described herein. The program implementing the present application may be stored on a computer-readable medium or may have the form of one or more signals. Such signals may be available through downloading from an Internet site, or provided on a carrier signal or in any other form. In addition, the present application may be realized with by means of hardware including a number of different elements and by means of a suitably programmed computer. In unit claims enumerating a number of devices, several of these devices may be embodied by means of the same hardware item. The use of the words first, second and third, etc. does not indicate any order. The words may be construed as names.
Furthermore, although exemplary embodiments have been described herein, the scope thereof includes any and all embodiments having an equivalent element, modification, omission, combination (e.g., schemes in which various embodiments intersect), adaptation, or alteration based on the present application. The elements in the claims will be construed broadly based on the language employed in the claims and are not limited to the examples described in this specification or during the implementation of this application, and the embodiments will be construed as non-exclusive. Accordingly, this specification and the embodiments are intended to be considered merely exemplary, and the true scope and spirit are indicated by the full scope of the following claims, as well as their equivalents.
The above description is intended to be illustrative and not limiting. For example, the above examples (or one or more schemes thereof) may be used in combination with one another. For example, a person of ordinary skill in the art may use other embodiments when reading the above description. Also, in the specific embodiments described above, various features may be grouped together to simplify the present application. This should not be construed as an intent that features of the disclosure that do not require protection are necessary for any claim. Rather, the subject matter of the present application may be less than all of the features of a particularly disclosed embodiment. Thereby, the following claims are incorporated here as examples or embodiments in specific implementation mode, where each claim independently serves as a separate embodiment, and it is contemplated that these embodiments may be combined with each other in various combinations or permutations. The scope of the present application shall be determined by reference to the full scope of the appended claims and the equivalent forms conferred by those claims.
The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is limited by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application in the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202110852227.9 | Jul 2021 | CN | national |
This application is a continuation of International Application No. PCT/CN2022/108039, filed on Jul. 26, 2022, which claims priority to Chinese Patent Application No. 2021110852227.9, filed on Jul. 27, 2021. The disclosures of the aforementioned applications are hereby incorporated by reference in their entireties.
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2022/108039 | Jul 2022 | WO |
| Child | 18424401 | US |
