Patent Application
20210354096
None.
This invention relates to an emulsification process and the control thereof.
Product quality assessment, during emulsification, has been identified as a challenging task in chemical industries. Emulsions are multi-phase colloidal dispersions obtained by mixing two or more immiscible fluids. One example of an emulsion is a two-phase oil in water emulsion (o/w) which is formed when oil gets dispersed as tiny droplets (dispersed phase) in water (continuous phase). Emulsification involves breaking an immiscible fluid (e.g., oil) into a homogenous distribution of microscopic droplets. The quality of the final emulsion product is highly dependent on the droplet size distribution, which is influenced by the operating conditions and process parameters.
Currently, the quality evaluation of emulsion products in for example the pharma sector is entirely based on manual examination of samples under the microscope. This manual technique has shown high subjectivity, low repeatability and reproducibility and is remarkably slow. The existing techniques which require physical sampling can lead to significant measurement errors causing inaccurate quality assessment. This can lead to over-processing, excessive energy utilization, wastage of resources and increased cost of production.
Other emulsion quality evaluation techniques have been proposed such as laser diffraction, spectroscopic, etc., however, with these techniques dilution of samples is essential. Such techniques add additional complexity to industrial processes, are time consuming and unreliable.
All of the above-mentioned techniques are difficult to automate for the reasons hereinbefore discussed.
There have been several machine vision techniques reported in the literature for the segmentation and analysis of optical microscopic images of emulsions. Zhou et al have conducted a critical review of various image processing approaches in real-time crystal size measurements. ImageJ and Matlab are the most popular and widely used machine vision software for automated droplet detection in emulsions according to previous research. Maaβ et al have developed a machine vision tool in Matlab for automated counting and measurement of particles/droplets in multiphase systems including emulsions. However, the previous methods are limited to circular particles and Maaβ et al have proposed a future extension of the method to additionally detect irregular shapes as well.
The droplet detection techniques previously referenced have used machine vision software mainly to identify the droplet border on the phase transition between oil and water using edge detection.
In addition to that, there has been no previous research published so far on a progressive analysis of droplet characteristics other than droplet diameter during emulsification.
One object of the present invention is to provide an improved system for controlling an emulsification process and overcome the problems encountered heretofore.
According to one aspect of the invention, there is provided a system comprising an apparatus and a method for controlling emulsification of at least two immiscible fluids.
According to another aspect of the invention, there is provided an apparatus for controlling emulsification comprising:
In one embodiment of the invention, the imaging device is adapted for mounting in situ where the emulsification process is taking place. One advantage of having the imaging device in situ where emulsification is taking place is that no sampling, manual or otherwise, is required.
In one embodiment of the invention, the imaging device is an optical imaging device.
In a further embodiment of the invention, the optical imaging device is configured to perform confocal microscopy, line confocal microscopy, deconvolution microscopy, spinning disk microscopy, multi-photon microscopy, planar illumination microscopy, Bessel beam microscopy, differential interference contrast microscopy, phase contrast microscopy, epifluorescent microscopy, bright field microscopy, dark field microscopy, oblique illumination or a combination thereof.
In a preferred embodiment of the invention, the optical imaging device is configured to perform bright field microscopy.
In another embodiment of the invention, the imaging device comprises a charge-coupled device (CCD) camera.
In another embodiment of the invention, the imaging device comprises an endoscope.
In another embodiment of the invention, the imaging device comprises a soft sensor.
In another embodiment of the invention, the image obtained by the imaging device is a micrograph.
In another embodiment of the invention, the micrograph is calibrated to a micrometer scale.
In another embodiment of the invention, the image processor is configured to detect the droplet from the pixels in the image.
In another embodiment of the invention, the image processor is configured to detect the droplets comprise a means for image segmentation.
In another embodiment of the invention, the means for image segmentation is an edge and symmetry filter.
In another embodiment of the invention, the means for image segmentation comprises a histogram-based technique.
In another embodiment of the invention, the image processor configured to detect at least one of the droplet characteristics comprises means to prepare a graphical representation from the pixels in the images.
In a preferred embodiment of the invention, the graphical representation is an edge and symmetry graphic.
In a most preferred embodiment of the invention, the graphical representation is a histogram.
In another embodiment of the invention, the image processor comprises means to calculate the mean pixel intensity from the graphical representation.
In another embodiment of the invention, the apparatus comprises means to threshold the images.
In another embodiment of the invention, the apparatus comprises means to convert the thresholded images to binary.
In another embodiment of the invention, the apparatus comprises means to apply watershed segmentation to the images.
In another embodiment of the invention, the image processor comprises means to convert the images to 8-bit.
In another embodiment, the means to convert the images to 8-bit is positioned after the imaging device and before the means to prepare a graphical representation.
In a preferred embodiment, the means to convert the images to 8-bit is positioned after the imaging device and before the means for image segmentation.
In another embodiment of the invention, the means for comparing the measured droplet characteristics with the desired droplet characteristics specification determines an emulsification quality category of the emulsion at a given pre-set interval during emulsification.
It is to be appreciated that the emulsification quality category is determined from images obtained by the image processor at pre-set intervals during emulsification.
In another embodiment of the invention, the means for comparing the measured droplet characteristics obtained from the image with a desired droplet characteristics specification involves a supervised classification model.
In a further embodiment of the invention, the supervised classification model utilizes an algorithm.
In a preferred embodiment of the invention, the algorithm of the supervised classification model adapts in response to new droplet characteristics data.
One advantage of an algorithm that adapts in response to new droplet characteristics data is that the apparatus becomes capable of machine learning and can optimize operational performances and better control emulsification based on the new data received and avoid overprocessing.
In a more preferred embodiment of the invention, the algorithm is principal component analysis (PCA) based.
In another embodiment of the invention, the apparatus comprises a central processing unit configure to control operation of the apparatus.
In a further embodiment of the invention, the central processing unit monitors the emulsion quality category at the pre-set intervals between the start and the end of emulsification to determine when the desired droplet characteristics are achieved.
In another embodiment of the invention, the apparatus comprises an alerting means for indicating the emulsion quality category of the images obtained by the image processor to the central processing unit corresponding to a quality of the emulsion at the pre-set intervals between the start and the end of emulsification.
In a further embodiment of the invention, the central processing unit sends the signal to the control switch to terminate the emulsification.
It is to be appreciated that the signal to terminate is sent once the central processing unit has determined that the desired droplet characteristics are met.
One advantage of the features as hereinbefore described is that real-time deployment of emulsion quality evaluation is enabled.
Another advantage of the features as hereinbefore described is that emulsion quality evaluation techniques can be automated.
Another advantage of the features as hereinbefore described is that in-line deployment of emulsion quality evaluation in industrial process is made possible.
According to another aspect of the invention, there is provided a method for controlling emulsification of at least two immiscible fluids including the steps:
In one embodiment of the invention, the method includes acquiring images of the emulsification at five minute intervals between a start and an end of the emulsification.
In one embodiment of the invention, the detecting and measuring utilizes a means for image segmentation.
In another embodiment of the invention, each image obtained by the imaging device is a micrograph and the method comprises calibrating the micrograph to a micrometer scale.
In another embodiment of the invention, the method comprises converting the images to 8-bit.
In a preferred embodiment of the invention, conversion of the images to 8-bit occurs after the images have been acquired by the imaging device and before the detecting and measuring of the droplets.
In another embodiment of the invention, the method comprises preparing a graphic representation of the image from the pixels in the images.
In another embodiment of the invention, the method comprises the preparation of the graphic representation wherein the graphic representation is an edge and symmetry graphic.
In a preferred embodiment of the invention, the method comprises the preparation of the graphic representation wherein the graphic representation is a histogram.
In another embodiment of the invention, the method comprises calculating the mean pixel intensity from the graphic representation.
In another embodiment of the invention, the method comprises thresholding the images using the calculated intensity value.
In another embodiment of the invention, the method comprises converting the thresholded images to binary.
In another embodiment of the invention, the method comprises applying watershed segmentation.
In another embodiment of the invention, the method comprises categorizing the emulsion into one of several pre-defined emulsion quality categories.
In a preferred embodiment of the invention, the categorizing is determined by the comparison of the measured droplet characteristics against a desired droplet characteristics specification.
In another embodiment of the invention, the method comprises processing data on the emulsion quality category obtained from the pre-set intervals between the start and the end of emulsification to determine when the desired droplet characteristics are achieved.
In a further embodiment of the invention, the method comprises sending a signal from the central processing unit to a control switch to terminate the emulsification when the desired droplet characteristics are achieved.
In another embodiment, the invention provides a system for controlling an emulsification process including the steps:
In another embodiment of the invention, the system includes:
In another embodiment of the invention, the system includes:
The invention will be more clearly understood by the following description of some embodiments thereof, given by way of example only, with reference to the accompanying drawings.
Referring to the figures, and initially to
Referring now to
In this embodiment, the apparatus 1A comprises a means for categorizing the emulsion into an emulsion quality category 8. This categorization is based on the measured droplet characteristics data 5 as compared with the desired droplet characteristics specification 7.
It is to be appreciated that the desired droplet characteristics specification 7 is pre-determined prior to use in the system (1A; 1B).
The emulsion quality category 8 determines whether or not the desired droplet characteristics specification 7 is achieved. When the desired droplet characteristics specification 7 is achieved emulsification is terminated.
Referring now to
It is to be appreciated that the droplet characteristics specification as described in
It will be appreciated that the system (1A; 1B) of the invention provides greater accuracy, precision and much higher speed in the evaluation of emulsion quality and optimum process time prediction compared to manual evaluation. Also, advantageously the system (1A; 1B) of the invention does not require any dilution of the emulsion, which enables the system to be automated.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Each box plot, from
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
It is to be appreciated that the supervised classification model built using the selected PCs of
Referring now to
Referring now to
Referring now to
The examples given below present an investigation of two different image segmentation techniques that were performed to extract various droplet characteristics from images obtained by the imaging device, such as optical micrographs, during emulsification.
Methods described below are modelled on typical industrial production of cream-based emulsions.
The emulsion was continuously mixed for 30 minutes using a homogenizer at a tip speed of 25 m/s for the first 15 minutes and at a tip speed of 15 m/s for the last 15 minutes respectively. Microscopic images (micrographs), of an oil in water (o/w) cream emulsion, where acquired at 5-minute intervals from the start to the end of a 30-minute emulsification process. The emulsification process was stopped after 30 minutes, as any further processing was considered as over processing.
It will be appreciated that while a typical o/w emulsion is shown, the apparatus and method would also suitably work with other types of emulsion. While micrographs were obtained as the image of choice, other image types are envisioned to also suitably work in the system herein described. Furthermore, while the method described shows a standard o/w emulsification the system is not to be considered as limited to controlling only o/w emulsions, being suitable for controlling emulsification of other emulsion types.
A Zeiss Microscope Axio imager Atm was employed to obtain bright field micrographs of 40× magnification under standard illumination settings from each sample and were saved as TIFF (tagged file format) files. An example of the images obtained can be seen in
A macro, user-defined program to execute specific tasks, was programmed in Fiji to execute an image segmentation procedure.
The edge and symmetry-based image segmentation steps dynamically in the following order. First the micrographs were calibrated to a micrometer scale such as 4 pixels/μm. The images were then converted to 8-bit in order to facilitate further processing. Edge and Symmetry filter (ESF) was applied to detect the oil droplets in the images. The filtered images were auto-thresholded through the red channel to enhance the detected droplets. The images were then converted into binary and ‘watershed segmentation’ was applied separating the droplets that touch/overlap each other.
Alternatively, an image segmentation procedure based on producing a histogram of the distribution of gray values (pixel intensity) in the region of interest.
Calibration of the micrographs to a scale of 4 pixels/μm and conversion to 8-bit greyscale were then performed which provides 2{circumflex over ( )}8=256 levels of intensity values for each pixel.
This was followed by histogram-based image segmentation, which was executed. First a histogram was computed from all of the pixels in the images for each image. The mean pixel intensity of the histogram was calculated and stored in a variable. The images were then thresholded using the calculated mean intensity value. Converted to binary and watershed segmentation was applied to separate the droplets that touch/overlap each other. Finally, each micrograph was analysed for a droplet area range ≥1 μm2 and a circularity range of 0.00 to 1.00 to extract the droplet characteristics. The minimum size settings can be changed to detect droplets down to nanometres.
It is to be appreciated that such calibration and conversion may form part of the initial steps of the macro.
The droplets detected using one of the two methods of image segmentation were then analyzed using the ‘Analyze Particles’ functionality in Fiji. The purpose of this image analysis was to extract the characteristics of the detected droplets for a user-specified range of droplet area (μm2) and circularity. A diverse set of characteristics was extracted for each droplet such as size, shape, orientation, centroid, solidity etc.
Thirteen characteristics were extracted for each droplet from the 60 micrographs.
These are the following:
The droplet characteristics were automatically exported, by the macro, into a CSV file in the user-specified directory.
It is to be appreciated that such analysis may form part of the final steps of the macro.
Statistical analysis of the droplet characteristics was performed in RStudio, version 1.1.383 for all the micrographs, which is an integrated development environment for programming using the R language. R 3.4 is the version used in this study. The mean values of the droplet characteristics were calculated and box plots were generated to identify the characteristics that varied significantly over the emulsification period. The R data visualization package ggplot2 was used to create the plots. The variation in the mean and median of each characteristic and also the correlation between them were also investigated. A reduced set of droplet characteristics, deemed suitable, were finalized as the input feature space for the classification of micrographs. The evolution of average droplet size in terms of area (μm2), perimeter (μm), minimum Feret diameter (MinFeret in μm) and maximum Feret diameter (Feret in μm) and the evolution of droplet count at each 5-minute processing interval were studied in detail. The oil concentration, i.e., the % of area occupied by the droplets, was also determined for each micrograph.
The total set of 60 micrographs were categorised manually into four groups named Unacceptable (U), Marginal (M), Acceptable (A) and Target (T) by micrograph analysts from industry. An unsupervised cluster analysis of the droplet characteristics was performed using PCA to observe the patterns in the data. The PCA technique also helped to reduce the dimensionality of the data and to obtain a set of uncorrelated components.
The variation in the droplet characteristics throughout the process was statistically analysed using the box plots. Among the 13 characteristics obtained from each droplet, only the size features such as area, perimeter, Feret and MinFeret as well as droplet count showed significant variation during the emulsification process. The remaining droplet characteristics such as orientation, shape and centroid were not considered relevant, as they showed no variation throughout the emulsification process.
The overall variation in the mean droplet characteristics during the whole 30-minute process can be summarised as follows:
On the basis of the variation presented by the mean droplet size characteristics as well as the droplet count, including the knowledge of process experts, the 60 micrographs were manually grouped into four quality-based categories referred to as TAMU. The processing intervals to which the categories belonged to are shown in Table 1.
The 10 sample micrographs obtained from the first five minutes were categorised as ‘Unacceptable’ (U), another 10 micrographs obtained after 10 minutes of emulsification were labelled as ‘Marginal’ (M), the next 10 obtained after 15 minutes, in total, were categorised as ‘Acceptable’ (A) and the 30 micrographs acquired after 20 minutes until the end of the process were grouped as ‘Target’ (T).
The five droplet characteristics, area, perimeter, Feret, MinFeret and count, were selected as the input features for building the classification model. A correlation matrix of the droplet characteristics was obtained and is presented in Table 2.
The droplet size characteristics were found to be highly correlated (r=0.84 to 1.00). The next major step in the analysis was to covert the correlated feature space into a reduced set of uncorrelated components. This was followed by an unsupervised cluster analysis of the components to distinguish the TAMU patterns identified by the industrial experts.
PCA was performed to transform the five variable feature space into an equal set of uncorrelated principal components (PCs). An unsupervised clustering pattern of the four response categories (TAMU) was also explored using the score plots of the first three PCs.
A PC-LDA model was developed as a linear combination of the first three principal components, PC1, PC2 and PC3. The number of predictive discriminant functions derived from the model is calculated as the minimum of G−1 and p, where G is the number of response categories (i.e., four) and p is the number of predictor variables (i.e., three). In the current case, the value of p and G−1 are equal and therefore, the PC-LDA model resulted in three discriminant functions, LD1, LD2 and LD3, as given by Equations 1, 2 and 3. The model formula is represented by Equation 4.
LD1=1.03PC1−0.31PC2−0.35PC3 Equation (1)
LD2=−4.68PC1−0.29PC2−0.35PC3 Equation (2)
LD3=0.13PC1+1.70PC2−1.46PC3 Equation (3)
Category˜LD1+LD2+LD3 Equation (4)
The discriminant functions, as given by the above-mentioned equations, were then used to separate the four TAMU categories, the results of which are presented and discussed under
A stratified 5-fold cross-validation was performed to evaluate the classification accuracy of the PC-LDA model. 10 micrographs from each category were selected for the cross-validation. In each fold, the micrographs were randomly split into 70% for training and the remaining 30% for testing. Five models were created in such a way that each model consisted of a training set of 28 micrographs (seven from each category) and a test set of 12 micrographs (three from each category) respectively. For each model, the classification score was recorded and a confusion matrix was created.
The models were further validated using a set of six micrographs (two from ‘U’ category, two from ‘M’, one from ‘A’ and one from ‘T’ respectively). These micrographs were obtained from a laboratory, in a different country, under varying illumination settings. All the five models successfully classified five out of the six micrographs into the correct category except the Target (T) one, which was classified as Acceptable (A). This gives an overall accuracy of 83%.
The droplets detected from a sample micrograph acquired after the initial five minutes of processing. Similarly, the droplet characteristics of all the 60 micrographs were extracted automatically in an iterative loop by the macro.
The edge and symmetry detection technique highly depends on well-defined droplet borders and also on the radius parameter of the algorithm specified by the user. As the droplet size decreases with emulsification, it is difficult for the user to identify an accurate value for the droplet radius. This limits the use of this technique. Multiple detections will be required for a single micrograph, using a set range of radii, to detect droplets from a varying size distribution, which is not feasible in terms of time and accuracy.
Such edge detection techniques have proved less successful in the image analysis of highly concentrated emulsions with dispersed phase fraction greater than 10 to 15%.
The existing image segmentation techniques, as described in the literature, for droplet detection from emulsion micrographs are mainly border-based edge detection methods. Such techniques have demonstrated potential only in the case of high-quality images, droplets with pronounced borders, less overlap and also in emulsions with low dispersed phase fraction ≤15%. By contrast, the histogram-based technique (HBT) as used in the present invention is capable of detecting both bigger and smaller droplets thus providing a smooth evolution of droplet size and count. The droplet size decreases significantly during the last minutes of processing and they appear as texture rather than discrete droplets. The histogram-based approach demonstrates proficiency in detecting those droplets.
The HBT method presented good detection of both bigger and smaller droplets. An R function was written to read the droplet characteristics of each micrograph from the CSV file (saved by the macro) in a sequential loop and create box plots showing their evolution over emulsion processing time. A plugin was developed in Java language, as a soft sensor, to integrate the Fiji macro with the R function. For these reasons, HBT is the preferred image segmentation technique for use in the present invention.
The machine vision techniques ESF and HBT were compared by analyzing the oil concentration obtained using the image analysis.
The HBT demonstrates potential in assessing the evolution of droplet characteristics during emulsification until the desired characteristics are achieved. Therefore, the technique can be implemented as an automated tool in chemical industries to predict and identify the optimum process time. This can contribute to eliminating over-processing and associated resources that can be regarded as surplus to production requirements.
The system of the invention can entirely automate, in real-time, the quality assessment of the emulsion. It is possible to implement the system of the invention by integrating the machine vision technique with real-time imaging (an endoscope coupled with a CCD camera). The system of the invention can be applied as a soft sensor, for in-situ process monitoring, to provide real-time feedback on emulsion quality.
The competence of edge & symmetry and histogram-based machine vision techniques, applied on optical micrographs, in the valuation of droplet characteristics during emulsification is hereinabove shown. Droplet area, Feret diameter and count are the characteristics identified as the product quality indicators.
The edge & symmetry technique is less advantageous, as it does not lend as well to full automation. This is due to the need to recalibrate the radius parameter as droplet sizes change. On the other hand, the histogram-based approach of the present invention is fully automated. The histogram-based image segmentation has demonstrated significant potential in the detection of droplets and their corresponding characteristics. The technique has provided a progressive evolution of decreasing droplet size and increasing droplet count. The oil concentration results obtained using the histogram-based approach is in good agreement with the studied emulsion. The technique is capable of identifying the equilibrium point at the end of the industrial process. The system of the invention avoids the over-processing of emulsions, leading to smart and sustainable manufacturing practices.
In the specification the terms “comprise, comprises, comprised and comprising” or any variation thereof and the terms “include, includes, included and including” or any variation thereof are considered to be totally interchangeable and they should all be afforded the widest possible interpretation and vice versa.
The invention is not limited to the embodiments hereinbefore described which may be varied in both construction and detail within the scope of the appended claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 1817089.4 | Oct 2018 | GB | national |
This application is a national phase to PCT Application No. PCT/EP2019/078599 filed Oct. 21, 2019, which in turn claims priority to UK Patent Application No. GB1817089.4 filed Oct. 19, 2018, all said applications incorporated in their entirety herein by reference thereto.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2019/078599 | 10/21/2019 | WO | 00 |
