This specification concerns systems and methods for evaluating a variable of interest concerning a sample having aquatic organisms and more particularly relates to such systems and methods which involve computer vision in evaluating the variable of interest.
Aquaculture involves cultivating populations of aquatic organisms (e.g. fish) as they grow over time. Controlling the cultivated organisms and/or the amount of feed given to the cultivated organisms (sometimes in the form of smaller aquatic organisms), can be required to achieve satisfactory efficiency in terms of growth, survival rate and costs.
Published PCT application WO 2012/083461 describes methods and systems for estimating a relatively large quantity of organisms in a sample using the quantity of attenuation, by the sample, of a light signal. These methods and systems were satisfactory to a certain degree, but there always remains room for improvement. For instance, in order to provide a satisfactory degree of precision in the determination of the quantity of organisms it was known to perform a calibration beforehand to determine the quantity of attenuation associated to a known quantity of organisms. The quantity of organisms used in the calibration step was determined by hand-counting, which was time consuming.
One specific need occurs when it is desired to count a relatively large number of organisms. For example, fish egg producers typically counted the fish eggs one-by-one, in a time consuming process, in order to adequately evaluate the number of fish eggs which are present in a sample. In some cases, such as in the case of fish eggs for instance, these organisms can be amassed in a superposed manner to one another rather than being dispersed in a water medium during the step of counting. It was found that the volume which such amassed organisms occupy can be correlated to a quantity of organisms. A calibration process can be used beforehand, which can involve the determination of a (mean) unitary volume of organisms and a filling factor of the organisms (i.e. the amount of volume which is unused when the organisms are amassed, which depends on the shape and deformability of the organisms).
Accordingly, in accordance with an aspect, there is provided a method of determining a variable of interest associated with a sample of organisms received in a closed container having a contour wall extending upwardly from a closed bottom, the method comprising the steps of: receiving a given volume of the sample of organisms in the container such that the organisms are amassed to form a depth of organisms extending from the closed bottom to a given level of the contour wall; using a camera, acquiring an image of the sample of organisms received in the container; measuring an imaged level, in the image, corresponding to the given level of the contour wall to which the depth of organisms extends; and determining the quantity of organisms associated with the imaged level based on calibration data.
One other specific need is associated to the determination of a variable of interest concerning aquatic organisms of a sample, where the variable of interest can be correlated to an appearance of the aquatic organisms. For instance, the color of the organisms can be indicative of the health of the organisms (e.g. some bacterial, fungal and/or parasite diseases change the color of the organisms). In another example, the mean size and/or the size distribution of the organisms can be a variable of interest which is determined based on the appearance of the organisms, and more specifically by measuring the size of the organisms in a sample, for instance. There is thus a need for systems and methods which can be used to automate the determination of appearance-related variables of interest concerning the organisms in a sample.
In accordance with an aspect, there is provided a method of determining an appearance-related variable of interest associated with a sample of organisms received in a container, the method comprising the steps of: receiving a given volume of the sample of organisms in the container; using a camera at a fixed distance relative to the container, acquiring an image of the sample of organisms received in the container; and determining a value of the appearance-related variable of interest associated with the organisms using the image taken by the camera.
In accordance with another aspect, it was known to calibrate a system such as the one described in published PCT application WO 2012/083461 to subsequently allow satisfactory correlationship between an amount of received light (received light=emitted light−attenuation) and a quantity of organisms. More specifically, the calibration can include the determination of a biomass attenuation relationship (relationship indicative of amount of light absorbed by each individual organism), and can require the manual counting of a relatively large amount of organisms. There is thus a need to automate the determination of the biomass attenuation relationship associated with an organism, which, in turn, can help automating the calibration of the photometry system described in published PCT application WO 2012/083461.
In accordance with an aspect, there is provided a method of determining a biomass attenuation relationship associated with a sample of organisms received in a container, the method comprising the steps of: receiving a given volume of the sample of organisms in the container; using a camera, acquiring an image of the sample of organisms received in the container; determining a value of the quantity of organisms associated with the sample using the image; while a sample having the quantity of organisms previously determined is received in the container; emitting an initial intensity of diffused light onto the sample; the container receiving the diffused light and reflecting the diffused light through the sample, the sample thereby attenuating the initial intensity; measuring a reflected intensity of the diffused light; and comparing the reflected intensity to the quantity of organisms of the sample to obtain a biomass attenuation relation.
One need occurs when a large number of aquatic organisms need to be counted. For example, producers sometimes need to have a relatively good estimation of the number of aquatic organisms which are raised in order to provide the right amount of feed in order to increase growth and survival rate without wasting resources. Once the right amount of feed has been determined, the next challenge resides in actually providing the right amount of feed to the aquatic organisms, which can also require counting of organisms since the feed can be provided in the form of smaller living organisms (e.g. plankton). There is thus a need for systems and methods which can be used to automate the counting of the organisms in a sample. Accordingly, there are a very large number of organisms which remains to be counted for properly managing the production of organisms in the aquaculture industry.
In accordance with an aspect, there is provided a method of determining a variable of interest associated with a sample of organisms received in a container, the method comprising the steps of: receiving a given volume of the sample of organisms in the container; using a camera, acquiring an image of the sample received in the container; and determining a value of the variable of interest associated with the sample using the image.
The step of receiving the given volume can further comprise receiving the given volume of the sample of organisms in the container such that at least some of the organisms have a distinctive feature which do not overlap with the distinctive features of the other organisms, the method further comprising the step of: localizing the distinctive features of the organisms in the image; wherein said determining the value of the variable of interest associated with the volume received in the container is based on the localized distinctive features in the image.
The container can be a closed container having a contour wall extending upwardly from a closed bottom and known dimensions; wherein said receiving further comprises receiving the given volume of the sample in the container such that the organisms overlap with one another to form a layer of organisms extending upwardly from the closed bottom to a level of the contour wall, the method further comprising the steps of: obtaining a unitary volume associated with the organisms, the image comprising the sample and the interior of the container such that the image shows the level of the contour wall to which the layer of organisms extends, wherein the variable of interest is a volume of the layer of organisms; inferring the volume of the layer of organisms inside the container based on the level of the contour wall and the known dimensions of the container; and determining a quantity of organisms associated with the layer of organisms based on the unitary volume and the inferred volume of the layer of organisms.
The container can be an open container having an inlet, an outlet and a conduit between the inlet and the outlet, said receiving a volume of a sample of organisms further comprising receiving a flow of the sample of organisms at the inlet and flowing the volume of the sample across the conduit towards the outlet, the flow of the sample being such that at least some of the organisms have distinctive features which do not overlap with the distinctive features of the other organisms of the flow; localizing the distinctive features of the organisms in the image, the method further comprising the steps of determining a value of the variable of interest associated with the volume received in the container based on the localized distinctive features; and wherein said determining the value of the variable of interest associated with the volume received in the container is based on the localized distinctive features in the image.
In accordance with another aspect, there is provide a system for determining a value of a variable of interest concerning a sample of organisms, the system comprising: a container for receiving the sample; a structure mounted to the container and having a camera oriented towards the sample for acquiring an image of the sample received in the container; and a processor in communication with the camera, the processor being coupled with a computer-readable memory being configured for storing computer executable instructions that, when executed by the processor, perform the step of: determining a value of the variable of interest associated with the sample using the image.
In accordance with another aspect, there is provided a method of determining a variable of interest associated with a sample having a quantity of organisms received in a container, the method comprising the steps of: acquiring, from a camera, an image the sample in the container; and determining a value of the variable of interest associated with the sample using the image.
In accordance with another aspect, there is provided a method of determining a volume of a sample received in a container, the method comprising the steps of: receiving the sample in the container; acquiring, from a camera, an image of the sample received in the container; measuring an imaged level, in the image, corresponding to a level to which the sample extends in the container; determining a volume of the sample using the imaged level and calibration data.
Many further features and combinations thereof concerning the present improvements will appear to those skilled in the art following a reading of the instant disclosure.
In the figures,
These figures depict example embodiments for illustrative purposes, and variations, alternative configurations, alternative components and modifications may be made to these example embodiments.
This disclosure describes methods and systems for determining a value of a variable of interest concerning a sample of aquatic organisms. Depending on the circumstances and on the embodiment, the variable of interest can be a quantity, an estimated unitary volume, a biomass, an appearance-related variable of interest such as a color, pigmentation or a presence of a disease, a depth, a position, a volume, a length, a width, an area and other variables of interest used in the field of aquaculture. The aquatic organisms can be fish, fish eggs, plankton and the like, depending on the application. As will be understood, although specific embodiments are described, embodiments which are best suited for determining given variables of interest associated with given organisms will be apparent for the skilled reader.
More specifically, the structure 108 is used to maintain the camera 110 at a given distance d from the closed bottom 114 of the container 106. As illustrated in
The camera 110 is so positioned that, when the container 106 receives the sample 102 of organisms 104, the camera 110 can image the sample 102, or a portion thereof, for further analysis by the system 100. In other words, the camera 110 has a field of view 120 which is oriented towards the sample 102. As depicted, the field of view 120 of the system 100 shown in
Referring now to
In this embodiment, the processing module 112 is used to localize distinctive features of the organisms 104 in the image 128 and the value of the variable of interest is determined based on the localized distinctive features of the organisms 104. In order for the determination of the distinctive features of the organisms 104 to be satisfactory, the organisms 104 are dispersed in a layer of liquid medium 126 (e.g. water) or in no liquid medium.
As illustrated in the exemplary image 128 taken by the camera 110 of the system 100, the distinctive features are contours 130 of the organisms 104, but it is understood that the distinctive features can alternately be eyes, guts or any suitable imaged characteristic feature of the anatomy of the organisms 104 in question. Also, this embodiment can also be used with fish eggs or other suitable marine organisms.
In the illustrated embodiment shown in
In the event where some organisms 104 of the sample 102 overlap, the system 100 can be specifically adapted to diagnose occurrences of overlapping and trigger an alarm and/or modify the value of the variable of interest. For instance, if the distinctive feature is the contour, the system 100 can be adapted to estimate the possible combinations of two overlapping contours and to localize such overlapping contours in the image 128. When overlapping contours are localized, the value of the variable of interest can be modified accordingly. Such modification can also apply for more than two overlapping organisms, depending on the circumstances.
It is understood that the image 128 can be digitally processed in order to enhance its contrast, for instance, to allow a more efficient localization of the distinctive features 130. For instance, the image 128 can be thresholded using a given intensity threshold such that any pixel of the image having an intensity lower than the given intensity threshold is set to black and that any pixel of the image having an intensity higher or equal to the given intensity threshold is set to white. In an alternate embodiment, the image 128 is also segmented in different portions such that the image is partitioned into multiple segments which help analyzing the image 128 by the system 100. As will be understood, other image processing techniques can be used.
It is contemplated that although only one image 128 is shown in
In another embodiment, the system 100 can be used to determine another variable of interest such as a size, a depth, a length or a unitary volume of the organisms. In such an embodiment, the system 100 references the image 128 in space and factors in known dimensions of the container 106, a known field of view 120 of the camera 110 as well as a known distance d separating the camera 110 relative to the container 106 such that the size and/or volume of one organism 104 can be estimated using the image 128.
More specifically, the focal length of the camera 110 can be optimized for a specific distance between the camera 110 and the organisms 104 (e.g. the closed bottom 114) so that the sharpness of the image 128 may change as a function of the distance of the organism 104 relative to the camera 110. Accordingly, by quantifying the sharpness of the image, information about the depth of the organisms can be estimated.
Now referring to
The system 200 is configured to measure the level L of the contour wall 216 which is reached by the volume 232 of organisms 204 using an image 228 such as the one shown in
In an embodiment, the calibration data associate the imaged level Li to the quantity of organisms 204. Such calibration data can be obtained by performing a calibration process which can include, for instance, placing a known quantity Nj of organisms 204 in the container, acquiring a calibration image and measuring a corresponding imaged level Li,j of the contour wall 216 using the calibration image. By repeating these steps at least another time, the calibration data, in this case provided in the form of a relationship Nj=f(Li,j), can be obtained for a given system 200 (e.g. for a given container 206). Accordingly, the system 200 can determine the quantity of organisms 204 in a sample 202 by correlating the imaged level Li, deemed proportional to the level L, to the quantity of organisms 204. The calibration data can also be provided in other suitable forms, as will be described herebelow.
In another embodiment, the system 200 is configured to determine the quantity of organisms 204 using calibration data which comprise known dimensions of the container 206 as well as a unitary volume associated with each organism 204. In this embodiment, the system 200 is configured to determine the imaged level Li in the image 228 and to infer a value of the volume 232 of the amassed organisms 204 based on the imaged level Li and on the known dimensions of the container 205. Once the value of the volume 232 of the amassed organisms 204 is inferred, the system 200 can determine the quantity of organisms 204 by correlating the unitary volume of the calibration data to the value of the volume of amassed organisms 204 inferred from the image 228.
It is noted that the calibration data used by the system 200 can include a filling factor, i.e. an estimated amount of emptiness between the organisms 204, associated with a given type of amassed organisms 204. For instance, fish eggs, which are typically substantially spherical, can have a different filling factor depending if they are amassed in a face-centered cubic (FCC) fashion or in a hexagonal close-packed (HCP) fashion. Accordingly, the system 200 can modify the quantity of organisms 204 based on the filling factor, which can typically bring the quantity of organisms 204 down by a certain extent. Depending on the organisms 204 and on their geometry, considering the filling factor in the determination of the quantity of organisms 204 can be negligible. In alternate embodiments, considering the filling factor of the organisms 204 is appropriate. In another embodiment, the emptiness between the organisms 204 can be filled with a liquid medium. In this specific embodiment, the filling factor can be modified when the amount of liquid medium is greater than the amount of emptiness that the sample would have if the sample had no liquid medium.
Further, the calibration data include a deformation factor associated with the organisms 204. In an embodiment, the deformation factor can cause the filling factor to vary as a function of the level L. For instance, when the deformation factor associated with the organisms 204 is significant, the filling factor of the organisms 204 closer to the closed bottom 214 can be higher than a filling factor of the organisms closer to the top surface 222. Accordingly, the quantity of organisms 204 determined by the system 200 can be modified by the deformation factor when it is believed that the deformability of the organisms 204 may influence the determination of the quantity of organisms 204 present in the sample 202. In another embodiment, when the deformation factor associated with the organisms 204 is low, the filling factor can be relatively constant throughout the sample 202.
In another embodiment, illustrated in
In yet another embodiment, the systems 100 and 200 can be used to determine appearance-related variables of interest using the image(s) taken by the camera(s). The appearance-related variables of interest can include a color distribution, a pigmentation distribution, a size distribution, a presence of defect (e.g. parasites, diseases) and any useful information. Based on such appearance-related variables of interest, the systems 100 and 200 can associate a status to one or more organisms of the sample. For instance, the color of the organisms 204 can help determine if the organisms 204 are healthy or not (presence of bacterial-, fungal- and/or parasite-related diseases), which can be useful. Also, in another embodiment, it can be useful to determine the appearance-related variable of interest to the sample 202 of organisms 204 by either analyzing the sample 202 as a whole or by analyzing each individual organism 204 of the sample 202, for instance. Accordingly, it can also be useful to average the appearance-related variables of interest associated with the organisms 204 using more than one of the organisms present in a single image. In this embodiment, illumination of the sample for imaging purposes is maintained in order for the image to be comparable from one another. In case where the sample comprises a liquid medium, this embodiment can be used to determine a value of a variable of interest concerning the liquid medium.
In order to further determine the appearance-related variables of interest, the system 200 can have a field of view 220 which is limited to a given portion of the top surface of the sample 102. For instance, now referring to
Alternately, a similar method can be used to determine the volume of a liquid or semi-liquid sample (as opposed to a sample having amassed organisms with little or no liquid medium). Still alternately, a 3D image can be obtained using stereoscopic vision. A 3D image of the empty container can be used to determine the shape of the bottom of the container for calibration, and a 3D image of the container with the sample of organisms can be compared to the image of the empty container to determine the volume of the sample, which can in turn be associated to a quantity of organisms. In some embodiments, the camera can be very simple to reduce costs, and can be provided without zoom capability for instance.
When using a system such as the one described in published PCT application WO 2012/083461, determination of the value of the variable of interest requires a correlationship between an amount of received intensity of diffused light and a biomass attenuation relationship. Indeed, for a given system having a given reflective surface (e.g. white walls and bottom of a container), and where the organisms (biomass) of the sample are tested in a given liquid (e.g. water having a given attenuation factor) or tested alone (in a amassed relationship without liquid, e.g. eggs), the only remaining variable which affects the attenuation of the reflected light is the presence of the biomass, and so the attenuation relationship can be simplified to a biomass attenuation relationship. For ease of understanding, it is noted that the biomass attenuation relationship is typically a function which has units of “amount of energy absorbed per number of organism”. Accordingly, to determine the biomass attenuation relation, one had to measure the amount of energy which is absorbed by the organisms of a sample, divide this amount of energy by the total quantity (typically counted manually) of organisms present in that sample, and then repeat these steps for a given number of samples having differing quantity of organisms therein. Accordingly, using the quantity of organisms (determined using computer vision) and the energy absorbed by the organisms 304 (determined using photometry), the biomass attenuation relationship can be determined. In alternate embodiments, a more complex form of calibration can be performed to factor in other variables of a more general attenuation relationship, such as a calibration which factors in the attenuation of the walls and of a liquid medium for instance.
Still referring to
In an embodiment, measuring the amount of energy absorbed by the organisms requires a referencing process in order to isolate the amount of energy which is actually absorbed by the organisms from the amount of energy that can be absorbed by the container, and/or by a liquid medium, if any. For each of the biomass attenuation coefficients of the biomass attenuation relation, the system 300 can be used to emit an initial intensity of diffused light towards a reference container exempt of organisms (for proper referencing, the reference container exempt of organisms can contain a liquid medium in cases where the sample comprises both organisms and a liquid medium) and to measure a reflected intensity of the diffused light which is reflected solely by the reference container (and by the liquid medium, if any). Then, the system 300 can be used to emit the initial intensity of diffused light towards the container 306 containing the sample 302 of organisms 304 and to measure a reflected intensity of the diffused light reflected by the container and by the sample. By comparing the initial intensity of diffused light, the intensity of diffused light reflected by the reference container and the reflected intensity of the diffused light reflected by the container and the sample, the amount of energy absorbed solely by the organisms 304 can be determined. In alternate embodiments, other suitable referencing processes can be used.
As mentioned above, having determined the biomass attenuation relationship associated with the organisms 304 using computer vision, the value of the quantity of organisms 304 can be provided using photometry. In alternate embodiments, it can be preferred to determine the biomass attenuation coefficient for a plurality of samples having different quantities of organisms 304 therein in order to obtain a statistically representative biomass attenuation relation. In another embodiment, once the biomass attenuation relationship is adequately determined using the combination of computer vision and photometry, the value of the variable of interest is determined using photometry since it is typically less power consuming than computer vision.
It is understood that for embodiments which use photometry, and possibly ones which use computer vision, the container 306 and/or the lid 308 are generally made of a material which is opaque for preventing external light from disturbing the lighting inside the container. In addition, the material is chosen to be reflective so that reflection of the diffused light onto the container 306 is increased. The container 306 and/or the lid 308 can thus be made of opaque white polymers. In the illustrated embodiment of
Also shown in
In these embodiments, the flow has a depth 427 (relative to the point of view of the camera 410) chosen relatively to a dimension 429 associated with the organisms 404 in order to limit or prevent overlapping of the organisms 404 in the field of view 420. The dimension 429 is substantially parallel to the depth 427 such that, in the embodiments shown in
Referring now to
As can be understood, the examples described above and illustrated are intended to be exemplary only. For instance, the methods and systems described herein can be used for determining a value of a variable or interest concerning samples comprising crustaceans, molluscs and aquatic plants. In alternate embodiments, a combination of optical components such as lenses and filters can be used to optimize the resolution of the image as a function of the type of organisms interrogated. It is understood that the system can use different wavelengths of illumination, different types of suitable wavelength filters and different polarizing filters. Moreover, it is noted that the open container is not limited to be open solely to the inlet and to the outlet, it can also be open along the longitudinal axis of the open container (e.g. a river and the like). The expression ‘camera’ is used generally to refer to a device which can obtain and record visual images. In some embodiments, the camera can be a device having a digital camera chip have very simple optics and avoid the use of a zoom for instance, in other embodiments, the camera can include a zoom, whereas in still other embodiments, the camera can be provided in the form of a laser scanner, for instance. In most embodiments described above, the camera obtains a 2D image, but it will be noted that in alternate embodiments, the camera can obtain a 3D image. The scope is indicated by the appended claims.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CA2015/050629 | 7/7/2015 | WO | 00 |
Number | Date | Country | |
---|---|---|---|
62021556 | Jul 2014 | US |