The present disclosure relates to a method and device for non-invasive tomographic characterisation of a sample comprising a plurality of differentiated tissues.
Mainstream analytical chemistry and biosensor approaches to spectroscopy information complexity are to use physical separation unit operations [1,2], reaction-specific labelling [3,4], or more recently label-free reaction specificity [5]. State-of-the-art chemometrics makes use of macroscopic hyperspectral or multispectral imaging to relate surface characteristics of plant tissues (e.g., fruits or leaves) to pigmentation and disease symptoms [6]. The previous art methods are cumbersome to be used in vivo, particularly in an open field context:
These facts are disclosed in order to illustrate the technical problem addressed by the present disclosure.
The present document discloses a computer-implemented method for non-invasive tomographic metabolite characterisation of a vegetable or animal sample to be characterised, the sample comprising a plurality of differentiated tissues from a set of previously-characterised differentiated tissues, by using a multidimensional latent structure model of differentiated tissues, which was previously obtained by: scanning tissue samples belonging to each differentiated tissue of the set with a hyperspectral microscope and spectrometer; acquiring hyperspectral spectrum or spectra for each differentiated tissue sample from the hyperspectral microscope and spectrometer; analysing the same tissue samples for the metabolites to be characterised; calculating the latent structure model from the acquired spectra and analysed metabolites; the method comprising the steps of: spatially scanning the sample to be characterised with a hyperspectral point-of-measurement, POM, optical device and a spectrometer; acquiring hyperspectral spectrum or spectra from each of a plurality of spatial positions and/or orientations of the POM optical device and spectrometer in respect of the sample to be characterised; using the previously obtained model to deconvolute the metabolites to be characterised for each of the differentiated tissues from the acquired spectra.
In an embodiment, the scanning of tissue samples belonging to each tissue of the set with a hyperspectral microscope and spectrometer is microscopic, and the spatially scanning the sample to be characterised with a hyperspectral POM optical device and spectrometer is macroscopic.
In an embodiment, the spatially scanning comprises varying one or more of the following to obtain a spatial diversity of the POM spatial scanning: displacement of the POM optical device; point-of-view angle of the POM optical device; different scales of magnification of the POM optical device; moving the POM optical device lines of sight to cause a parallax effect.
In an embodiment, the multidimensional latent structure model is a hierarchical latent structure model.
In an embodiment, the hierarchical latent structure model comprises a latent variable model comprising a latent variable super-space and corresponding latent variable sub-spaces.
In an embodiment, the hierarchical relationship of the hierarchical latent structure model comprises super-space and corresponding sub-spaces: a latent sub-space for spectral data of each of the differentiated tissues; a super-space for each of the metabolites to be characterised, translating the combination of hyperspectral data to the metabolites to be characterised.
In an embodiment, the sample to be characterised is a three-dimensional tissue structure or organ.
In an embodiment, the metabolites to be characterised comprise pigments, tannins, photosynthesis, energy storage, nutrient uptake, necrosis, infection probability, or combinations thereof.
In an embodiment, the sample to be characterised is a vegetable sample, in particular the sample to be characterised is a tomato or grape tissue sample.
In an embodiment, the sample to be characterised is vegetable sample and the differentiated tissues comprise skin, pulp, and seed tissue.
In an embodiment, the scanning of tissue samples belonging to each differentiated tissue is carried out along a sampling grid.
In an embodiment, the scanning of tissue samples belonging to each differentiated tissue is further carried out along discrete sampling depths.
In an embodiment, the spectra are UV-VIS-NIR.
It is further disclosed a device for non-invasive tomographic metabolite characterisation of a vegetable or animal sample to be characterised, the sample comprising a plurality of differentiated tissues from a set of previously-characterised differentiated tissues, including a computer processor and a non-transitory computer-readable medium comprising a multidimensional latent structure model of differentiated tissues, which was previously obtained by: scanning tissue samples belonging to each differentiated tissue of the set with a hyperspectral microscope and spectrometer; acquiring hyperspectral spectrum or spectra for differentiated each tissue sample from the microscope and spectrometer; analysing the same tissue samples for the metabolites to be characterised; calculating the latent structure model from the acquired spectra and analysed metabolites; wherein the computer processor is configured for: spatially scanning the sample to be characterised with a hyperspectral point-of-measurement, POM, optical device and spectrometer; acquiring hyperspectral spectra from a plurality of spatial positions and/or orientations of the POM optical device and spectrometer in respect of the sample to be characterised; using the previously obtained model to deconvolute the metabolites to be characterised for each of the differentiated tissues from the acquired spectra.
It is also presented a non-transitory computer-readable medium comprising computer program instructions for implementing a device for non-invasive tomographic metabolite characterisation of a vegetable or animal sample to be characterised, which when executed by a processor, cause the processor to carry out the disclosed method.
One of the present disclosure aims is to develop a non-invasive spectroscopy tomography system for compositional imaging of plant and animal tissues. This technology is demonstrated in application to precision agriculture and plant physiology. The present disclosure is based on image reconstruction using latent hierarchical information fusion methods to unscramble the observed spectroscopy signal into its different components of the observed tridimensional structures, particularly the different plant tissues being recorded, such as in grapes the skin, pulp, and seeds. Hierarchical relations exploit the parallax effect, which changes the spectral fingerprint depending upon the point-of-view angle of the spectroscopy probe. Reconstruction is performed by the hierarchical latent relationships between spectral patterns of the observed tissues. This principle is used to develop:
Results show the feasibility of relationship reconstruction from latent structures, as well as, providing relationship between macroscopic and microscopic spectral data.
Spectral data carries both physical and chemical information. The observed spectrum of plant tissue is the superposition of all structures in the field of view, scattering, matrix effects, and multi-scaled interference. Being able to use these complex interactions is paramount to unscrambling the information present in each recorded spectrum. The complexity of spectral data has been the foremost hurdle to the existence of in vivo, reagent-less, and non-invasive chemical or metabolic diagnosis.
The macroscopic spectra are recorded and related to the hyperspectral image by their corresponding latent space.
The capacity of internal structures reconstruction, compositional, and disease diagnosis at very early stages centres molecular biology and plant physiology as the main driver of information in precision agriculture and plant biotechnology. The present disclosure bridges the information between in vitro laboratory and in vivo field studies by allowing high-throughput and multi-scaled monitoring of plants in their ecosystem without disrupting the interaction of plants with soil, climate, and ecosystem. Allowing information transfer between the field and laboratory tests is a critical enabling transformation in precision agriculture that allows evaluating to be much faster, thus promoting scientific advances in both phytopharmaceuticals and agricultural practices for early-stage and low-impact practices.
One of the main objectives of the disclosure is to provide a macroscopic, microscopic, and multi-scaled spectral reconstruction, compositional, and infection diagnosis through spectroscopy POM devices and advanced signal/information processing, demonstrating the feasibility of tomographic reconstruction by hierarchical latent structures methods.
The following figures provide preferred embodiments for illustrating the disclosure. They should not be seen as limiting the scope of the invention.
In an embodiment, the present disclosure is deployed/integrated on existing systems and devices, thus extending their technical capabilities. It enables the existing system to explore in detail plant internal structures and composition for a new omics approach to precision agriculture, where molecular biology and plant physiology are key enablers of new diagnosis and agricultural practices. The disclosure includes,
In an embodiment, the two are related by hierarchical latent structures models in
A similar strategy is used for hyperspectral microscopy, as exemplified in
In an embodiment, a scanning hyperspectral microscope was implemented to scan at 200× magnification inoculated tomato leaves, taking the spectra along the x, y and z axis. The 3D dataset is stored in the tensor format, to each space coordinate has a corresponding spectrum. The evolution of spectral fingerprints of infection and control are used to create classification models to be used in both hyperspectral microscopy diagnosis and multi-scaled diagnosis.
Multi-scaled reconstruction is presented in
Tomato (Solanum lycopersicum L.) plants of the cultivar Cherry were grown in 200 ml pots containing a commercial potting substrate, in a walk-in plant growth chamber under controlled conditions (t=25-27° C., humidity≈60%, and photoperiod of 12/12 h). Plants were divided into three groups: control, Pst and Xeu; control was used as the control (submitted to sterile water treatment), Pst was the group for inoculation with Pseudomonas syringae pv. tomato, whereas Xeu was the group for inoculation with Xanthomonas euvesicatoria LMG 905. Plants were inoculated in the laboratory, at the growth stage of 5-6 fully expanded leaves, by liquid aerosolisation until saturation was achieved with subsequent run-off. Differently inoculated plants were segregated from each other, in order to avoid any possible cross-contamination.
The bacterial suspensions used for these inoculation assays consisted of 1×108 cells·mL−1 for Pst and 2×108 cells·mL−1 for Xeu, prepared from a 48 h old culture, grown on KB medium (peptone, 20.0 g; glycerol, 10.0 mL; MgSO4, 1.5 g; K2PO4, 1.5 g; agar, 15.0 g; distilled water, up to 1.0 liter), for Pst, and YDC medium, for Xeu (yeast extract, 10.0 g; dextrose, 20.0 g; CaCO3, 20.0 g; agar, 15.0 g; distilled water, up to 1.0 liter). Afterwards, the inoculated plants were covered with transparent polythene bags for 48 h in order to increase the relative humidity that fosters bacterial entry into plant tissues, through natural openings, i.e., stomata.
Plants were monitored daily for symptomatic evolution for 7 days. Leaf samples of all groups (control, Pst and Xeu) were collected once every 24 h, until the 4th day, and one final sampling moment at 144 h, in a total of 5 sampling moments per group.
Simultaneously and in order to verify if the bacteria cultures used in the inoculation assays were viable, 20 μL of Pst and Xeu solution was cultured in Petri dishes containing KB and YDC media, respectively. After 48 h, it was possible to observe bacterial growth in both nutrient media, supporting the bacterial viability at inoculation.
After the inoculation assays, plant leaves from each group were collected and stored in separate plastic bags. Afterwards, for sample disinfection, the surface of each sample was washed in 70% alcohol for 5 min, followed by rinsing with sterile distilled water for 1 min; this procedure was repeated 5 times. Disinfected samples were placed in different homogenising bags containing a side filter. The samples were grounded in 2 ml of sterile distilled water until full tissue maceration was achieved. The obtained mixture was diluted with sterile distilled water to a final volume of 6 mL and collected after passing the filter bag. Twenty microliters of each sample solution were plated in separate Petri dishes containing KB (Pst extracts) and YDC medium (Xeu extracts), respectively, and incubated for 48 h at 28° C. Bacterial colonies that presented characteristic Pst and Xeu morphology were isolated and used in colony PCR.
Primer pairs designed by Vieira et al. [7] were selected for PCR validation of leaves infected with Pst. Similarly, infection of leaves infected with Xeu was confirmed by PCR using primer pairs designed by Albuquerque et al. [8]. Polymerase chain reactions were carried out in 20 μL reactions containing 1× DreamTaq Buffer (Thermo Scientific, Vilnius, Lithuania), 0.2 mM of each dNTP (Thermo Scientific), 0.2 UM of each primer (STAB Vida, Lisbon, Portugal), 1 U of DreamTaq DNA Polymerase (Thermo Scientific), and 10 μL of a dilution of bacterial cells in sterile water. The PCR conditions for the confirmation of Pst were as follows: an initial denaturation step of 3 min at 95° C., followed by 35 cycles of 30 s at 95° C., 30 s at 63° C. and 30 s at 72° C. with a final extension step of 10 min at 72° C. [7]. Similarly, PCR conditions for the confirmation of Xeu were as follow: an initial denaturation step of 5 min at 95° C., followed by 35 cycles of 30 s at 95° C., 30 s at 61° C. and 30 s at 72° C. with a final extension step of 10 min at 72° C. [8]. PCR products were separated on 0.8% agarose gels stained with GreenSage Premium (Nzytech, Lisbon, Portugal).
In an embodiment, macroscopic tomography reconstruction is performed by recording the spectra to the whole grape in two different apparatus: i. direct transmittance using the described device; and ii. full transmittance using a power led light source as opposed to the transmittance probe with light collection fibers, using a reflectance probe and HR4000 spectrometer from Ocean Insight. The same grape was afterward separated into its different tissues (skin, pulp, and seeds) for the spectral recording of these grape components. A total of 192 grapes were used to provide statistical support to macroscopic reconstruction.
Leaf samples of each plant were excised of the plant and readily analysed: each leaf was deposited on a single glass slide, supported by a platform featuring a pinhole slit (<0.5 mm) from which white light was projected (>65000 lux). The stainless-steel probe was then placed perpendicularly to the leaf (adaxial side) on select spots (3×) throughout each leaf (radial orientation) for spectral acquisition. This configuration minimises light scattering, allowing light collimation towards the fiber within the probe, maintaining a minimal measurement area (200 μm), and providing maximum discrimination by leaf scanning. No further sample preparation was required nor performed.
Spectroscopy measurements were performed using UV-Vis-NIR fiber optic (I.D. 200 μm) stainless steel probe connected to an Ocean Insight model HR4000 spectrometer, with dynamic acquisition times, so that spectral data was fully comprehended within the linear range of the spectrometer. Data acquisition was performed within the 200-1100 nm range, and every 24 h, until the 4th day, and one final sampling moment at 144 h, in a total of 5 sampling moments per group.
Leaf samples of each plant were excised of the plant and readily analysed: samples were laid between glass slides, and placed on the microscopical setup for data acquisition. A 5×5 grid (millimetre spacing) sampling grid was devised on each leaf, comprising of 25 data points of spectroscopic data; sampling grid initial coordinates were randomly set, yet assuring the avoidance of the leaf main stem line. No further sample preparation was required nor performed.
The hyperspectral microscopy system, consisting of a Zeiss Axiovert.A1, in transmission mode, was modified and customised to perform 3D hyperspectral microscopy for sequential x, y, z mapping. The following modifications were performed:
Microscopical imaging was performed using Best Fit microscopy option for macro (Red, Green, Blue—RGB) images, under the same conditions as for microscopical hyperspectral imaging, every 24 h, until the 4th day, and one final sampling moment at 144 h, in a total of 5 sampling moments per group.
The basis of information relationship within different scales of spectral measurements is performed by hierarchical relationship of super-space and corresponding sub-spaces. The sub-spaces conserve the latent space variables of the different layers of a hyperspectral microscope image, that is, the features of these images are linear combinations of the vectors spawning the sub-space at a given space coordinate [9,10]. These characteristics are used to derive a super-space of characteristics, translating the combination of a particular hyperspectral image or tomography. The super-space characteristics are afterwards possible to be related to the macroscopic feature space to establish the relationship between macroscopic and microscopic measurements.
Results of the reconstruction of the macroscopic tomography using hierarchical latent variables are presented in
Spectra of the grapes along the maturation follow the displayed parabola of
The tomographic fusion space is obtained by the information fusion of the variance sub-spaces of spectral characteristics of the skin, pulp, and seeds, obtained by their observed spectra, as seen on
This case-study shows how spectral data is correctly represented by its latent structures, spaces, and sub-spaces. The spectral data in tomography, as well as, between different physical scales, is well represented by a hierarchical relationship of the latent structures from each scale. The use of latent structures is highly computationally efficient, as these methods perform all computations in a compressed space, and therefore, as the computational cost is low, the proposed method is ideal for running in low-cost IoT (Internet of Things) hardware, allowing the development of very cost-effective hardware/software relation for metabolic tomography.
Results of tomato leaf bacterial infection diagnosis by P. syringae and X. euvesicatoria are presented in
At this stage, P. syringae exhibits a higher degree of discriminance than X. euvesicatoria, when compared to the control samples. As the infection progresses, significant differences between the control and infection agent begin to diverge in terms of spectral patterns and corresponding chemical composition, leading to the isolated classifications of each sample type in the linear discriminant analysis presented in
At the naked eye or RGB camera, visible superficial damage appears only after 72 hours after inoculation. Therefore, human or machine vision algorithms (e.g., Deep Learning) will not be able to diagnose the disease before it enters this late stage. Ground truth polymerase chain reaction (PCR) tests shown positive identifications of the presence of P. syringae and X. euvesicatoria after 24 h of inoculation, as shown in
Results for hyperspectral microscopy tomography are presented in
To our best knowledge, the state-of-the-art does not provide a tool similar to the present disclosure. Results show that the disclosed technology has high potential to revolutionise precision agriculture in its dimensions by being able to monitor the omics of each plant in vast plantations, providing information for the study of the complex relationship of plant-soil-climate-ecosystem and agricultural practices.
The benchmarks presented in Table 1 show how this type of technology can improve precision agriculture's state of the art. By focusing precision agriculture on plant physiology and its molecular biology, new horizons can be achieved by: i. developing new agricultural practices with omics scientific evidence; ii. introducing a modern way of developing agricultural practices by the use of in silico models from bioinformatics and systems biology; iii. linking plant biotechnology molecular biology in vitro results towards in vivo field applications; and iv. understanding how the physiology of plants inserted into a natural ecosystem responds when compared to laboratory observations.
Flow diagrams of particular embodiments of the presently disclosed methods are depicted in figures. The flow diagrams illustrate the functional information one of ordinary skill in the art requires to perform said methods required in accordance with the present disclosure.
It will be appreciated by those of ordinary skill in the art that unless otherwise indicated herein, the particular sequence of steps described is illustrative only and can be varied without departing from the disclosure. Thus, unless otherwise stated, the steps described are so unordered meaning that, when possible, the steps can be performed in any convenient or desirable order.
It is to be appreciated that certain embodiments of the disclosure as described herein may be incorporated as code (e.g., a software algorithm or program) residing in firmware and/or on computer useable medium having control logic for enabling execution on a computer system having a computer processor, such as any of the servers described herein. Such a computer system typically includes memory storage configured to provide output from execution of the code which configures a processor in accordance with the execution. The code can be arranged as firmware or software and can be organised as a set of modules, including the various modules and algorithms described herein, such as discrete code modules, function calls, procedure calls or objects in an object-oriented programming environment. If implemented using modules, the code can comprise a single module or a plurality of modules that operate in cooperation with one another to configure the machine in which it is executed to perform the associated functions, as described herein.
The disclosure should not be seen in any way restricted to the embodiments described and a person with ordinary skill in the art will foresee many possibilities to modifications thereof. The above-described embodiments are combinable. The following claims further set out particular embodiments of the disclosure.
It is acknowledged the contribution of Fundação Amadeus Dias, Santander Universidades, and the University of Porto in respect of the BIP PROOF 2021 AWARD.
This work was supported by Project “OmicBots: High-Throughput Integrative Omic-Robots Platform for a Generation Physiology-based Precision Viticulture/PTDC/ASP-HOR/1338/2021”, financed by the National Funds through the FCT-Fundação para a Ciência e a Tecnologia, I.P. (Portuguese Foundation for Science and Technology).
O. E. de Noord, Theobald E. H., Multilevel component analysis and multilevel PLS of chemical process data. J. Chemometrics, 19 (2005) 301-307. https://doi.org/10.1002/cem.933
J. Chan, S. C. Raju, E. Topol. Towards a tricorder for diagnosing paediatric conditions. The Lancet (British Edition), 394 (2019) 10202, 907-907.
H. Waters, New $10 million X prize launched for tricorder-style medical device. Nature Medicine, 17 (2011) 754-754
Number | Date | Country | Kind |
---|---|---|---|
117727 | Dec 2021 | PT | national |
117732 | Jan 2022 | PT | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/EP2022/088102 | 12/30/2022 | WO |