On-site analysis system with central processor and method of analyzing

Description

STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH OR DEVELOPMENT

[0002] Not applicable.

FIELD OF THE INVENTION

[0003] The present invention is generally directed to a process for analyzing materials, preferably at multiple locations. More particularly, the present invention relates to a method of rapid analysis which utilizes analytical instrumentation, for example spectroscopic sensor units, on site to transmit and receive information to and from a central processor. The invention also relates to a system for acquiring data from a sample at remote site locations concerning a property of interest of that sample, transmitting the data to the central processor for data analysis, and receiving and optionally displaying processed information at those locations. Further, the invention relates to a method of providing analysis services to customers from a central processor.

BACKGROUND OF THE INVENTION

[0004] There are numerous instances where one or more properties of a material are preferably analyzed at one or more locations removed from an analytical laboratory where testing would normally be conducted. For example, agricultural products may be analyzed for the presence and concentration of certain components during the crop growing stage, at harvesting, during transportation, or after the product has been stored, as at a grain silo. Other non-limiting instances where this type of analysis would be useful include applications in the processed food industry, the mining industry, the chemical industry, the finished hard goods industries, and a variety of service, retail sales, and medical industries.

[0005] In the absence of equipment or skilled personnel for conducting sample analysis at the location of the sample, substantial time delays can result in initiating and completing an analysis. Thus in the case of the analysis of an agricultural product such as an oil seed which would be harvested within a narrow window of time, traditionally specific characteristics of the seed are determined by a laboratory. This is due to the fact that the equipment and skilled personnel generally required to conduct such analyses are not normally available to the farmer or even to the silo operator. Thus, if an oil seed is to be analyzed, a sample is taken from the farmer's truck or from the silo operator and sent to an independent laboratory for analysis. It is not uncommon in this situation for the sample to require one day for forwarding to the laboratory, two days for the laboratory to conduct the analysis, and an additional day for the results to be returned to the silo operator. Thus, a particular lot of oil seed may require four days to be analyzed. Where the value of the oil seed is dependent on the analysis, utilizing laboratory analysis results in substantial delays to the farmer in obtaining a value for his crop, to the silo operator in valuing the crop and determining the market into which the seed will be sold, and to the ultimate purchaser of the oil seed. When one considers that in the case of oil seed harvesting the oil seed crop typically is processed within a narrow window of time over a wide geographic area, the individual delays described above become multiplied at each silo within the crop growing region.

[0006] Alternatively, it is known to analyze certain components of a particular agricultural product at the location where the material is either grown, harvested, transported, or stored. Nevertheless, this resultant analysis of the product may not be directly comparable to an analysis of the same agricultural product in a different location, even though using the same methodology. Even when the same sample is analyzed at different locations, differences in analytical results may arise, for example, because of a difference in environmental conditions at one analysis location relative to the other or because of a difference in the performance of the analyzers. Results may also differ because of variations in the procedure of presenting the sample to different analyzers.

[0007] It may be convenient or necessary for on-site analyzers to be able to be easily transported from one location to another. A portable sensor unit or spectrometer is one that is sufficiently compact and robust to permit it to be transported to alternate testing locations as needed. These units are able to be removed from service and returned to service quickly for transportation to and use at a desired site for analysis. The analytical instruments for such analysis must be rugged and capable of making repetitive analyses without extensive recalibration by a skilled operator and with little or no variation over the course of use of the unit.

[0008] Because the analysis of, for example, a particular agricultural product may need to be determined at locations over a wide geographic area within a narrow time frame, it may become impractical to conduct the analyses using only one instrument. Generally it then becomes necessary to test these products at multiple sites with multiple analyzers. Under these circumstances, each of the analyzers must be calibrated so that the output results from the various analyzers can be properly compared. Depending on the type of analysis, with traditional techniques the analytical results of a particular agricultural product using multiple analyzers may vary because of different effects resulting from the environmental, instrument, or sample presentation variations discussed above. To address this, either each analysis should be conducted under the same environmental and sample presentation test conditions, or each analysis should be completed with the ability to compensate for differences in temperature, humidity and other relevant environmental variations in generating data by the individual analyzers. Further, results generated by different analyzers may differ because of inherent manufacturing differences between the highly sensitive instrument components and differences in the precise assembly of the components, the differences becoming more pronounced over time producing instrument drift. As a result, no two analytical instruments are precisely identical to one another, so accommodation must be made in considering the results generated by an analyzer when comparing the results with those from other units. While work has been done to develop practical methods for transferring multivariate calibrations between instruments, for example as discussed in U.S. Pat. No. 5,459,677, these methods require that some instrument, sometimes called a reference instrument or a master instrument, be maintained in some known or reproducible state or be capable of being brought into a reproducible and well defined state to achieve instrument standardization. Then, a master calibration model developed on one instrument can be transferred to a number of target instruments. However, all calibration transfer and instrument standardization methods require additional steps to be taken at various times potentially over a range of time intervals after the initial calibration transfer. For example, the analyst may have to evaluate a set of calibration transfer samples on the target instruments after the initial calibration transfer, usually by a skilled operator, and adjust either the models or the instruments so that the response from the target instruments agrees with the response from the master or reference instrument. Further, the measurement conditions of the material samples being analyzed, for example the sample temperature, may also be different at the various sites. Again, accommodations must be made in considering the results generated from those material samples.

[0009] Another matter to be considered in conducting remote analyses of materials such as agricultural products is the amount and quality of information desired from the analysis, and the demands placed on the analyzer. Generally, as the analyzer is able to perform more sophisticated analysis, the analyzer itself becomes more complex, a higher level of training is required to operate the analyzer and interpret the results, and the weight and size of the instrument may increase as a result. An analyzer capable of undertaking more complex analyses is generally more susceptible to damage and to generating inaccurate results by the process of moving the analyzer from site to site, utilizing the analyzer under varying conditions, and the like. Consequently, the results from such an instrument are more likely to change and thus render comparison between various remote analyzers more difficult or even impossible.

[0010] The need to be able to generate comparable, statistically equivalent analyses of materials at remote site locations can extend to a wide range of materials in addition to agricultural products such as, but not limited to, manufactured products, natural phenomena, ores, renewable raw materials, fuels, and living tissue.

[0011] The combination of a calibration model with an analytical instrument to generate a predicted result has been practiced. It is known to use, for example, calibration models associated with near-infrared, mid-infrared, and Raman spectrometers in commercial processes to monitor the status of chemical reactions. This monitoring capability can involve the generation of results from an analytical method with the application of statistical analysis and calibration models to interpret and quantify the data. For example, in the manufacture of carboxylic acids and derivatives from fats and oils, it is known to use near-infrared spectrometers loaded with the appropriate chemometric software to measure a number of properties of the carboxylic acids and their derivatives. This monitoring can be done during the manufacturing process on intermediate product, as well as on the finished product. The spectrometer can be operated in a stand-alone mode with the operator bringing samples to the spectrometer for at-line analysis. Alternatively, the spectrometer can be connected in-line to enable monitoring of the process stream as the manufacturing operation proceeds. Thus, two commercially available near-infrared spectrometers such as the Bomem MB-160 FT-NIR spectrometer loaded with HOVAL software (such as Version 1.6, 1992) and AIRS software (such as Version 1.54, 1999) from Bomem Inc., Canada, and the Bruker Vector 22/N spectrometer loaded with OPUS-NT Quant-2 software (such as Version 2.6,1999) from Bruker Optik GmbH, Germany have been used to analyze intermediate and finished carboxylic acid products for acid value, iodine value, titer, viscosity, hydroxyl value, saponification value, composition of fatty materials and derivatives, and for the presence of carboxylic acid methyl ester contaminants in a specific carboxylic acid.

[0012] The calibration models for evaluating the above properties were derived from the Grams-PLS plus (Version 3.01B, 1994, Galactic Industries Corporation) and Bruker OPUS Quant-2 software. In those instances where more than one data acquisition device was used to generate predicted results for a particular property of interest, individual calibration models were developed for corresponding individual instruments or a master calibration model was developed on a particular master instrument, transferred to one or more other instruments, and adjusted with instrument-specific correction factors to standardize the predicted results across multiple instruments.

[0013] In determining the chemical properties of incoming raw materials such as tallow, coconut oil and palm kernel oil for the production of carboxylic acids, near-infrared spectrometry with appropriate chemometric techniques such as the partial least squares (PLS) method has been used to evaluate the free carboxylic acid content of the starting materials, as well as iodine value and moisture content. The near-infrared monitoring can also be used to monitor the progress of the transesterification process utilizing fatty triglycerides and methanol as reactants. A near-infrared spectrometer connected to transesterification process equipment can also monitor free glycerine content, bound/combined glycerine content and methyl ester concentration. Alternatively, samples can be taken during the progress of the reaction to a stand-alone near-infrared spectrometer loaded with appropriate calibration models for off-line analysis. In connection with the monitoring of the progress of a reaction, the near-infrared spectrometer can utilize a fiber optic probe connected to the spectrometer by fiber optic cable. The use of the near-infrared spectrometer in combination with the application of modeling software permits analysis of particular chemical species during the progress of chemical processing, as well as at the conclusion of the chemical process. Spectrometers such as near-infrared operating in the in-line mode are capable of providing data substantially on a real time basis. Data generation in these instances occurs under tightly controlled test and environmental conditions and involves one or more probes connected to a single instrument connected to a single data processing unit.

[0014] There is presently a high interest in the analysis of agricultural products. Genetically modified materials are of particular interest. The grain and food distribution segments in agriculture have expressed significant need for analytical technology to meet market requirements to identify and quantitate genetically modified crops, especially corn and soybean, in world markets. This need has developed rapidly. U.S. farmers have increasingly accepted crops derived from genetic engineering after the success they experienced in the 1996 growing season. The U.S. Department of Agriculture estimated that approximately 25% of U.S. corn and 54% of U.S. soybeans produced in 2000 were grown from genetically engineered seed with input traits to provide resistance to herbicides, insects, or both. The composition of such input trait crops is generally macroscopically indistinguishable from similar crops without the corresponding input traits.

[0015] In contrast, the foods of the future which will incorporate improvements of direct benefit to the consumer likely will be based at least in part on crops having enhanced output traits. The composition of these enhanced crops is different from the corresponding conventional crops. Examples include high oil corn, high sucrose soybeans, and low linolenic canola. Genetically-enhanced crops can be produced either by genetic engineering, as enabled by recent advances in biotechnology, or by specially designed traditional breeding programs. Even traditional crop improvement practices can result in plants with changed genetics and enhanced properties.

[0016] The growth and the need for analytical technology for agricultural products has been the promulgation of labeling regulations adopted in many regions of the world including the two largest agricultural commodity trading communities, the European Union and Japan. These labeling requirements have required or are expected to require food processors to label finished food products as to the genetically modified content of the ingredients used to produce these products. The initiation of labeling and the growing number of food processors electing to use raw materials which have not been genetically modified are driving the need for identity preservation.

[0017] Labeling specifications are nearing completion in both Europe and Japan. Identifying the genetic composition of grain in commercial crops and maintaining that identity throughout the agricultural complex to support labeling has become a high priority for seed companies, commercial growers, distribution and process companies, as well as food processors and is expected to increase as labeling is further implemented in the future. Consequently there is a need to provide an economical and efficient way to analyze seeds and crops at various locations along the supply chain, to identify and quantify the chemical composition and potentially other measurable properties of one or more output traits in genetically enhanced as well as conventional crops.

[0018] The interest in obtaining detailed analysis of agricultural products extends also into areas involving analysis of other materials. There remains a need as to other materials in providing an economical and efficient way of analyzing materials on site at remote locations to identify and quantify their chemical compositions and other properties of interest.

BRIEF SUMMARY OF THE INVENTION

[0019] The present invention is directed to a process for identifying and quantifying one or more properties of interest of a material, the process involving providing a material to be analyzed; providing one or more data acquisition devices capable of acquiring data for prediction of one or more properties of the material; providing a central processor capable of computing one or more predicted results using multivariate calibration models and storing a database of multivariate calibration models; providing a communication link between data acquisition devices and the central processor; and analyzing the material using the data acquisition devices and the central processor in order to obtain results on one or more properties of interest. Preferably the central processor stores at least a portion of either measurement data, measurement results, or both. Preferably the data acquisition devices are capable of being transported from site to site. The calibration model is multivariate, and compensates for an effectively comprehensive set of measurement conditions and secondary material characteristics. Preferably, the communication link is capable of providing resultant information from the central processor to a user interface in the vicinity of the sensor. However, the resultant information may be conveyed by other means, such as by telephone communication, or may be conveyed by the same type of communication link as available between data acquisition device and central processor but to a location removed from the data acquisition device.

[0020] The invention is also directed to an analysis system comprising a central processor loaded with one or more calibration models, at least one data acquisition device connected to the central processor to supply input information, and a user interface to initiate data acquisition and to optionally provide an indication of the results generated by the central processor. A user interface may receive user input and present results to users using any known user-interface technologies, where user input can be provided by a keyboard, mouse, touchscreen, voice recognition, dedicated buttons, and the like, and presentation of results by a visual display, speech synthesis, printed pages and the like. A user interface may comprise multiple units or may be incorporated into a single device, including into the same device as a data acquisition device. Furthermore, user input may be received by multiple devices, and/or multiple devices may provide output to a user. Generally, the central processor includes a data repository to store at least resultant information, but the repository may also include at least a portion of the information acquired at the data acquisition device. The invention also encompasses a method for providing analysis of value to a customer.

[0021] Understanding of this subject matter is facilitated through the defining of certain key terms. The property of interest of a sample of material being analyzed is referred to herein as the primary variable. This variable is distinguished from other variables which influence the instrument response, which are identified as secondary variables.

[0022] Analytical instruments do not generate the values of a property of interest directly. Rather, measurement signals such as voltage or current are generated by the instruments. These signals may be pre-processed by transforming the raw data using instrumental transduction and computation steps performed by internal electronics, computer circuit boards, and the like to a form more readily assimilated by subsequent processing steps. These optionally pre-processed measurement signals are called the instrument output or the instrument response. An instrument is unable to analyze a sample of a material without the creation of a statistically valid relationship between the instrument response and known values for the property of interest of the material. The process of developing such a relationship is known as calibration.

[0023] The collection of all variations in secondary variables resulting from variations in the performance of one or more instruments is the instrument variance. Instrument variance encompasses variations both in the instrument components and in the assembly of the components, and encompasses variations which develop over time, but does not encompass operator variation in the use of the instrument, or other factors distinct from the instrument hardware which affect the observed value.

[0024] The actual mathematical relationship between the instrument response and the property of interest of a material is called the calibration model. To develop the model, the particular analytical instrument, employing a particular analytical method, is trained to measure a particular property of interest through development of a mathematical relationship between the instrument response and known values of the property. Experimental data related to the property of interest are generated by recording values of the property of interest determined by a reliable independent method on a particular group of samples. These recorded values are called known values. It is recognized that the experimentally determined values of the known data are characterized by experimental uncertainties, so the known values are not known exactly.

[0025] The group, or set, of samples of a material with known values of the property of interest used to develop this calibration model is called a calibration set. Variations in the material characteristics that are expected to be present in the population of samples that will be analyzed in the future should be represented by samples in the calibration set. This calibration set, or more typically a subset thereof, is then used to generate a collection of instrument responses over a range of measurement conditions for evaluating the property of interest. The collection of known values and instrument responses generated from the calibration set over a range of measurement conditions is a data set called the training set. Because each sample in the calibration set may be subjected to a range of conditions involving a number of secondary variables, as well as repeated measurements under the same conditions, the training set may contain more values than the number of samples comprising the calibration set. A training set generally encompasses both variations in a range of material characteristics and variations in a range of measurement conditions that are expected to be present during actual on-site analyses in the future.

[0026] As used herein, secondary material characteristics are material characteristics other than the property of interest that may influence the instrument response. The primary material characteristic is the property of interest. The collection of all variations in secondary variables resulting from variations in the secondary material characteristics of one or more samples of a material is the sample variance. Sample variance encompasses variations both within samples and between samples of a material. Variations in the primary material characteristic are encompassed by particular values in the training set spanning the expected range of the property of interest.

[0027] A validation set is another data set of known values and instrument responses generated from the calibration set, or more typically a subset thereof, or from a new set of samples of the same material with known values over a range of measurement conditions, such that this data set is usually distinct from the training set. The validation set is used to test the predictability of the calibration model that was developed using the training set. As used herein, an instrument-response set is defined as a data set that can be used as either a training set or a validation set.

[0028] The known values corresponding to the property of interest of the samples used to generate an instrument-response set may be determined by measurements using a validated analytical technique herein referred to as a primary analytical method. These known values are considered to be observed values that are suitable for use as reference data in developing a calibration model for a secondary analytical method. As used herein, the known or observed values corresponding to the property of interest for a set of calibration samples will be treated in the same manner regardless of whether the values were those of reference standards or the values were measured by a primary analytical method.

[0029] A remediation update is a new calibration model that is developed by generating new instrument responses for a previous training set without necessarily modifying the calibration set or the particular levels of measurement conditions of the previous training set in order to re-establish previously attained calibration statistics. Remediation updates are often needed to compensate for changes in instrument variance over time. Since the usable lifetime of calibration samples may be shorter than the time interval between recalibrations, a remediation update may be generated from newly prepared samples of the same material covering a similar range of material properties as those of a previous calibration set.

[0030] In comparison, an enhancement update is a new calibration model that is developed from a training set which has been altered from a previous training set for the purpose of improving some prediction capability of a model. An enhancement update may be developed from combinations of an extended training set that includes additional calibration data to span a wider range of either sample characteristics or measurement conditions, or both; a corrected training set that includes modified calibration data or excludes erroneous calibration data to correct errors discovered in a previous training set; or an improved training set that includes new calibration data, some of which may replace corresponding values in a training set, where the new values may result from improvements in a primary analytical method used to generate better estimates of known values or from an improved representative sampling of the material. An improved training set may also be developed by selecting a different set of calibration samples or measurement conditions for the new training set, where the number of observations may be greater than, the same as, or less than that used in a previous training set.

[0031] A global training set is a training set configured to compensate for an effectively comprehensive range of variation in the secondary variables in connection with predicting the property of interest from a calibration model.

[0032] The term compensation as used herein is defined as the reduction or elimination of the impact of variation in one or more factors on the predicted result.

[0033] If the output of an analytical instrument depends only on the property of interest, a univariate calibration model would be created. Generally, a univariate calibration model is rarely developed because a number of additional variables are usually encountered which affect the instrument response. Such other variables may include impurities in the sample and the temperature of the sample. Where more than one variable affects the instrument response, the calibration model generated to interpret the instrument response is considered multivariate. Thus, a multivariate model having n variables of the instrument response R is defined mathematically as

R=f
(x1, x2, . . . , xn),

[0034] where x1 is the property of interest and x2 to xn are the additional n−1 variables that affect the instrument response R.

[0035] The objective of calibration model development is to predict the instrument response from known values of the property of interest in the presence of variations of the secondary variables. A training set for the calibration model is developed by generating instrument responses at different levels spanning the expected ranges, or a portion of these ranges, of the primary and secondary variables. In the development of a calibration model, the instrument response is a dependent variable, and the primary and secondary variables are identified as independent. The dependent variable is the presumed effect or response to a change in the independent variables. Independent variables are also known as predictor variables.

[0036] After the calibration model is developed, it is rearranged to express the property of interest as a function of the instrument response and the secondary variables. The measurement process predicts, or measures, the property of interest by using the analytical instrument response generated for an unknown sample as input to the calibration model. The primary objective of calibration model use is to predict the property of interest from the instrument response in the presence of variations of the secondary variables. In actual situations where the calibration model is multivariate, the relationship of the property of interest to the instrument response is affected by the presence of secondary variables, which include all other factors that will significantly influence the instrument response. These secondary variables may be described alternatively as influential factors, interfering factors, or contaminating factors in the measurement process.

[0037] The quality of a calibration model can be described by calculating the degree of correlation between the known values and the corresponding predicted results using a training set or a validation set. One such process is called cross-validation, wherein the same observations of a training set are used for two different purposes, model building and validation. As used herein, an observation is the data corresponding to a single measurement process. One version of cross-validation is known as the leave-one-out method, involving a training set of M observations with repetition of building the calibration model M times. Each time, one observation (the ith) is excluded and the remaining M−1 observations are used to build the ith calibration model, where i ranges from 1 to M. The excluded observation is then used to validate the ith model, whereby the ith residual is computed as the difference between the ith predicted value and ith known value.

[0038] The statistical measure of the degree of correlation between the known values and the corresponding predicted results of the data set is the square of the correlation coefficient, known as r2. The square of the correlation coefficient is also known as the coefficient of determination, which measures the fraction of the variation in the dependent variable about its mean that is explained by variation in the independent or predictor variables. The total variation in a set of predicted values is the sum of two parts, the part that can be explained by the model and the part that cannot be explained by the model. The ratio of the explained variation to the total variation, or r2, is a measure of how good the model is. The r2 statistic is therefore a measure of the strength of the relationship between the observed and predicted values. The values of r2 range from 0 to 1, and cover the range of no correlation up to perfect correlation. In practice, these two extremes are rarely if ever encountered. High values of r2 indicate that the model tends to determine its predictions with small errors. R2 is defined as 100 times r2, so the values of R2 range from 0 to 100. Additional information on these and related statistical terms can be found in “Multivariate Calibration”, 1989, John Wiley & Sons, Ltd. by Harold Martens and Tormod Naes; and “Chemometric Techniques for Quantitative Analysis”, 1998, Marcel Dekker, Inc. by Richard Kramer, both texts of which are incorporated herein by reference.

[0039] Samples with known values of the property of interest are called known samples. These samples are distinct from other samples called unknown samples, for which values of the property of interest may not be known.

[0040] It is possible that the measurement of an unknown sample may generate a value which is not a valid prediction by the existing calibration model. This invalid predicted value is called an outlier. An outlier may be an observed measurement which deviates so much from other observations as to arouse a suspicion that the sample was taken from a population distinct from that used to create the calibration model. Alternatively, an outlier may be a false positive observation which does not deviate noticeably from valid observations but belongs to an overlapping, contaminating population. An outlier is always a value that is distinct from a basic population of valid predictions of a property of interest. Generally it is expected that large gaps would be noted between observations of outliers relative to observations of acceptable values which fall within the basic population. However, the distinction between acceptable and outlier observations is not always clear-cut because contaminating distributions can overlap the basic distribution. In practice then, it is expected that not all outliers will be identified. The determination of an outlier is done with a statistical probability rather than with certainty.

[0041] Some outliers may be caused by invalid measurements, such as when an instrument malfunctions to produce an abnormal spectrum or when an incorrect type or insufficient quantity of sample is measured. Other outliers may result from the inadvertent use of erroneous values of known data in a training set or a validation set. Outliers caused by invalid measurements or erroneous calibration data are called bad outliers and the corresponding predicted values should be considered to be invalid results. In other situations, outliers may be valid measurements of samples or measurement conditions which fall outside the range of primary or secondary variables spanned in the training set. In still other situations, outliers may result from valid measurements in which some previously unidentified secondary variable has become an influential factor. In these latter two situations, such outliers are called good outliers since they identify opportunities to extend the training set to cover a wider range of samples and measurement conditions. While the predicted values of good outliers should also be considered to be invalid results, if a new calibration model is developed by including a range of such good outliers in the training set, future measurements will be capable of generating valid results over a wider range of samples and measurement conditions.

[0042] There is no unequivocal test to determine whether an observation is an outlier. The possibility of an observation being an outlier is determined by the type of test used to evaluate that observation. In a situation where the distance of an observed data point from a central measure of a population can be an indication of an outlier, the operator can employ the Mahalanobis distance to determine the existence of a possible outlier. The Mahalanobis distance is the scalar distance between a multivariate observation and the centroid of a multivariate distribution that takes into account the actual spatial distribution of values in multidimensional space. A typical cutoff point or threshold value for determining if an observation is considered to be a probable outlier is a Mahalanobis distance that is often found to be in the range from 0.1 to 1. At values lower than the threshold, there is typically insufficient reason to exclude the observation. Such observations are presumably valid. Depending on the desired probability of detection, the threshold value of the Mahalanobis distance value which would be an indicator of a probable outlier may be adjusted downwardly or upwardly to increase or decrease the probability of detection. However, increasing the probability of detecting an outlier also increases the probability of rejecting valid observations. Alternatively, a recommended threshold value for detecting a probable outlier can be computed by chemometric software such as OPUS Quant-2 from the observed distribution of Mahalanobis distances in the training set. The reference method used for determining Mahalanobis distance (MAH) was ASTM E 1790-96.

[0043] It has been found that different threshold values of the Mahalanobis distance can be used to classify probable outliers into a small number of different categories according to the probable reason that the observed value is a probable outlier. Thus, for example, Mahalanobis distances in the range from about 0.4 to about 1.0 often correspond to good outliers, while Mahalanobis distances greater than about 1 often correspond to bad outliers. Furthermore, when the Mahalanobis distance is considerably greater than 1, for example greater than 100, the outlier is extremely bad and often corresponds to a major instrument malfunction such as a non-emitting excitation source or the complete absence of a sample during data acquisition. When the Mahalanobis distance is in the range from about 1 to about 100, the bad outlier often corresponds to smaller problems with the sample, the instrument, the environment or the sample presentation such as when a sample is present during data acquisition but is of a different material from that used to develop the calibration model or when an inadequate amount of sample is detected. Generally, the threshold value for bad outliers is generally at least twice the magnitude of the threshold value for detecting a probable outlier which herein is called the threshold value of good outliers. The threshold value for extremely bad outliers is generally about 100 times the magnitude of the threshold value for bad outliers.

[0044] At a basic level, the on-site analysis system comprises one or more data acquisition devices, a central processor, and a communication link. The data acquisition devices are used primarily, though not necessarily exclusively, for data acquisition, and the central processor for data analysis. A packet of information transmitted from an individual data acquisition device to the central processor is called measurement data, and the packet of information transmitted from the central processor to an output device of a user interface in the vicinity of a data acquisition device or to a third party is called measurement results. The data acquisition devices typically are physically separated from the central processor, though two or more data acquisition devices may be at a single location. The data acquisition devices may be geographically separated from each other and the central processor by great distances, but this is not required. As used herein, the data acquisition devices broadly identify the group of devices which acquire information about a sample of a material. Preferably, the data acquisition devices acquire spectroscopic data, though other analytic mechanisms may be employed, such as via chromatography, mass spectrometry or emission detection. Preferably, the data acquisition devices are transportable and are capable of generating an instrument response from data acquired on samples of material at a number of remote locations. The data acquisition device may also include in a single unit the sample presentation device for providing a sample for data acquisition by a detector, and a local processor (such as a laptop computer) with a user interface for executing the steps necessary to generate a measurement result, with an optional output device to read out the measurement result.

[0045] The central processor as used herein is a computer system which stores one or more calibration models and manipulates data transmitted from one or more data acquisition devices using the calibration models to predict values for the property of interest of a material. The central processor is not necessarily a single entity, however, since it may reside on multiple computer servers or clustered servers, where some duplication may be provided for redundancy and other duplication may be provided to mirror servers in multiple geographic locations. The use of multiple servers also increases the processing capacity, i.e., the number of transactions which can be completed within a period of time. In practice, the central processor behaves as if it were a single entity at a central location. The group of redundant and mirrored processors is known herein as the central processor. Further, the database of calibration models stored in the central processor is preferably constructed to compensate for expected variations in an effectively comprehensive set of secondary variables to provide statistically equivalent results from different remote instruments over time without instrument-specific calibration transfers or remediation updates.

[0046] An advantage of the analysis system is the relationship of the multiple data acquisition devices to the central processor. The system can function with many data acquisition devices located at sites which may be far removed from each other geographically for measuring the properties of a single material, such as an agricultural or mineral ore product. However, the system can also encompass multiple data acquisition devices in the same room or building. The use of a central processor means that all data from each data acquisition device are being manipulated in the same way for predicting a property from a particular model.

[0047] The database or library of calibration models stored in the central processor can be modified as desired to provide enhancement updates, add models to expand the capabilities for analyzing new properties, or to delete models that are no longer needed. All modifications to the database of calibration models can be done without making any changes to the hardware or software of individual data acquisition devices. Thus, enhancement updates and new models installed in the central processor can be used to analyze measurement data from all data acquisition devices immediately after installation without the need to separately undertake any remedial action at each of the remote sites, and the measurement results from all data acquisition devices typically are stored in the central processor for subsequent reporting and data analysis. As discussed in more detail below, the influential variations which exist in the test environment, the sample presentation to the data acquisition device, the physical and chemical characteristics of the sample, or the individual data acquisition devices themselves are able to be taken into account in quantifying the particular property or identifying the particular substance being measured. Thus, for example, the quantified measurement of the concentration of a triglyceride in an oilseed in one part of the world can be directly compared with a triglyceride concentration of an oilseed at a different location. Even where only one data acquisition device is connected to the central processor, the improved calibration model permits analysis to be conducted over time without the need for remedially updating the model.

[0048] The central processor stores a database of calibration models; receives a plurality of data values from a single measurement process, these data acquired by data acquisition devices, typically spectroscopic, about the particular material of interest; computes values for one or more properties of interest of the material using algorithms or computational procedures to manipulate the data and generate predicted values from the calibration models; and forwards the resultant information which it has generated. Optionally, the central processor sends results back to a user interface of the individual data acquisition devices. The data acquisition devices and central processor preferably transmit information to each other over a communication link, though it is possible for information to be transmitted from the data acquisition device to the central processor using a different communication link than that used for transmitting results. Presently, it is preferred that the information be transmitted in digital form. The communication link broadly is one or more communication pathways, often in a communication network, and may include various combinations of, for example, hard-wired telephone lines, cables, optical fibers, a system of towers or satellites for wireless communication, radio equipment, or combinations sufficient to transmit a signal carrying the desired information between any location and a central processor.

[0049] The calibration model based on chemometric methods of multivariate analysis provides the capability for generating useful measurement information even where more than one secondary variable encountered during the data acquisition step of a single measurement process may vary simultaneously, independently, or both.

[0050] Traditionally, calibration modeling has focused on variations in the sample and sometimes on one or more secondary variables, but not an effectively comprehensive set of factors in constructing the model. The instant invention not only recognizes the effect of sample variation in connection with developing a calibration model, but also evaluates and compensates for effects due to variation in the environment, the instrumentation, and the sample presentation.

[0051] After the potential universe of factors which may affect the characterization of the sample has been identified, it is determined whether one or more factors can be eliminated from consideration by mathematical methods of data pretreatment, whether certain factors must necessarily be considered in connection with developing a training set for the calibration model, or whether certain factors must be controlled in a manner as might be done in a traditional methodology to reduce or eliminate variation during the measurement process. The invention thus takes into account all factors known to a reasonable analyst for characterizing the sample as well as other factors that have not been recognized previously, and then proceeds to minimize the effect of all influential factors to develop an acceptable revision of the calibration model. The inventive method thus initially takes into account a number of factors and variations within each such factor sufficiently wide to span the expected range of variations, and then attempts to compensate for these factors in the process of ultimately creating an acceptable calibration model which contains substantial improvements in predictability and which eliminates or substantially reduces effects such as from instrument drift over time relative to existing calibration modeling techniques.

[0052] One method for developing the calibration model involves proceeding in a stepwise fashion, initially compensating for a single secondary variable, comparing the data generated by the thus-created calibration model with data from a reference method, followed thereafter by compensating for a second secondary variable and determining if the correlation improves or if the predictability is acceptable according to some statistical criteria, and repeating this process until an acceptable prediction level is achieved in the presence of variations of all known influential factors. This stepwise process is useful in the feasibility stage to identify influential factors and develop appropriate compensation techniques. Using the extended training set developed in this manner, it becomes more efficient to develop subsequent model revisions by including all factors at one time. Alternatively, the calibration model is initially developed by compensating for, or by otherwise taking into account, a number of known relevant factors at one time. Under these circumstances it is possible that the calibration model will still need to be refined one or more times before an acceptable model exists. However, the number of refinement operations will typically be lower than the number encountered when only one variable is evaluated at a time. In both cases, the goal is to identify all statistically significant factors which might arise from variations in the sample, the environment, instrumentation, and sample presentation; eliminate those factors from consideration which respond to data pretreatment, or satisfactorily reduce their effect; control those factors which may produce greater instrument responses than those from variations in the property of interest; and incorporate into the training set of the calibration model representative data spanning a range of variations for those remaining factors which affect the property of interest.

[0053] An example of an environmental variation is temperature change which results in samples having different temperatures at different times of measurement. This variation may affect the predictability of a calibration model generated where there has been no compensation for temperature, or if the measurement occurred at a temperature outside the range of temperatures represented by data in the training set. A number of other variations which may affect the calibration model can be introduced via the measuring instrument. The nature and number of these variations are a function of the type of instrument employed. For example, the incident radiation directed toward a sample using a near-infrared spectrometer has been found to vary as a function of the orientation of the source filament relative to the sample. Also, an older lamp generally does not provide the same light intensity as a new lamp. Concerning the sample presentation variations, the rate at which a sample passes through an incident light beam during a measurement process, or differences in the amount of sample in different measurement processes, may affect the predictability of the calibration model.

[0054] It should be recognized that the multivariate calibration models of the invention can be applied to any secondary measurement technique in which a data acquisition device generates multiple data values rather than a single numeric value for each measurement. The plurality of data values may be acquired as discrete values in a digital device, or they may be acquired in continuous fashion from an analog device and then converted into digital format. For example, a spectrum consists of a multitude of intensity values over a range of wavelengths. The spectral data may be acquired continuously or digitally. If the storage location of a single data value is referred to as a data channel, then a multichannel instrument is one for which acquisition of multiple data values may be considered to occur by storing the individual data values in a multitude of individual data channels. Thus, as used herein, a spectrometer that generates a continuous spectrum is a multichannel instrument, since the spectrum could be digitized and the multitude of discrete data values resulting from digitization could be stored in a multitude of individual data channels and could not be stored as a single data value in a single data channel. Hence, the invention applies generally to secondary measurements from multichannel instruments, where a secondary measurement is the prediction of a result from a multivariate calibration model of an analytical method. As discussed below, though the invention will be described in the context of NIR spectroscopy, it should be recognized that the invention can be practiced for any analytical method based on a multichannel instrument.

[0055] The samples that will be analyzed at remote locations may not be pure single molecular species, and thus will typically contain several components of which some may be contaminants and others may be distributions of molecular species as occur in synthetic polymers and natural products. Analysis of multichannel data on impure or multi-component samples can contain peaks showing considerable overlap. Chemometric methods of multivariate analysis such as partial least squares (PLS), principal component regression (PCR), artificial neural networks (ANN), and the like allow for determining the properties of interest of multiple components in each sample simultaneously. While the present invention will be described in terms of PLS, it should be understood that other chemometric methods of multivariate analysis can be used to construct calibration models. Multivariate calibrations make use of not just a single data point, but take into account data features over a range of data values, so analysis of overlapping bands or broad peaks becomes feasible.

[0056] PLS is one of a number of factor-based methods of multivariate analysis, where a factor space is an alternative coordinate system for a data set, a factor is an axis of the alternative coordinate system or factor space, and the dimensionality of a factor space is the number of axes or factors in the factor space. In a factor-based method of multivariate analysis, the axes of a factor space are selected to most efficiently span the data values in a manner that will capture as much as possible of the systematic variation in the data along a subset of axes or factors.

[0057] The total variation in a set of multivariate data is composed of two parts, the part caused by systematic variations in the primary and secondary variables and the part caused by random variation or experimental noise. It is usually found that some factors in a factor space contain only or mostly experimental noise, and such factors are therefore not related significantly to variations in the primary and secondary variables. Thus, it becomes possible to reduce the number of axes in a factor space relative to the number of axes in the original coordinate system of the data by omitting factors or axes that contain only or mostly experimental noise. The dimensionality of the factor space that is sufficient for predicting results to an acceptable level of precision is therefore generally smaller than that of the original coordinate system. The number of factors remaining after such a reduction in dimensionality is called the rank of the factor space. The rank characterizes the dimensionality of the fit of a calibration model to the data. It is generally preferred to avoid overfitting a model by selecting a smaller rank when possible.

[0058] The invention is also directed to a method which can support the analysis needs of customers, particularly where the materials to be analyzed are in locations geographically removed from each other and the operators are not skilled in analytical methodology. The method may also support the customer's transactions, the underlying contract agreements, material rating and billing functions, and further data analysis of material. The method utilizes the analysis method and system hardware described herein, encompassing a collection of processing, infrastructure and software components that support multiple application models that involve collection of data, transmittal of that data over a communication link, analysis of that data by appropriate software applications to derive value from the data, storage of the analyzed data in a repository for generating historical statistics, identity preservation and tracking, auditing, forecasting and model improvement, and delivery of results back to the original submitter or an alternate location over the same or different communication link.

[0059] It is therefore an object of the invention to provide a calibration model for a property of interest which can accommodate an effectively comprehensive range of variations in one or more characteristics of the material to be analyzed, the instrument, the environment, and the sample presentation without need for remedial model updating.

[0060] It is a further object of the invention to provide an analysis system which utilizes calibration models capable of compensating for a range of secondary variables.

[0061] It is a further object of the invention to provide an analysis system which provides for at least one data acquisition device and a central processor in combination with a calibration model algorithm to be able to accommodate an effectively comprehensive range of variations of primary and secondary variables, which can generate analysis data on the material to be analyzed.

[0062] It is a further object of the invention to provide an analysis system which provides for multiple remote data acquisition devices and a central processor which can generate analysis data on multiple samples of materials remote from each other but which are each analyzed using the same calibration model algorithm for the particular property being measured.

[0063] It is a further object of the invention to provide an analysis system which incorporates a user interface in combination with the data acquisition device to provide analysis information generated by the central processor for a particular sample being measured at the location where the measurement is taken.

[0064] It is a further object of the invention to provide a method of analysis which permits the measurement of one or more properties of materials utilizing a single calibration model algorithm for each property at a central processor.

[0065] It is a further objective of the invention to provide a method of analysis incorporating a calibration model algorithm that has been constructed to compensate for an effectively comprehensive set of expected variations in sample, sample presentation, environmental conditions and instrument.

[0066] It is a further object of the invention to provide a method of analysis which permits the measurement using multiple instruments of one or more properties of materials located in remote locations utilizing a single calibration model algorithm without incorporating instrument-specific parameters in the algorithm.

[0067] It is a further object of the invention to provide a method for analyzing materials at remote locations, e.g., for multiple customers, individuals, entities, or the like.

[0068] These and other objects and advantages of the invention are provided in the detailed description of the invention and in the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

[0069]
FIG. 1 is a block diagram of an on-site analytical system.

[0070]
FIG. 2 is a block diagram of components and information transmission within a data acquisition device.

[0071]
FIG. 3 is a block diagram of system architecture.

[0072]
FIG. 4 is a flowchart to establish model feasibility.

[0073]
FIG. 5 is a flowchart of model development and use.

[0074]
FIG. 6 is a flowchart of filter refinement.

[0075]
FIG. 7 is superposed NIR spectra of six samples of squalane in squalene at different concentrations.

[0076]
FIG. 8 is predicted versus observed values from the cross-validation of Model 1.0.

[0077]
FIG. 9 is predicted versus observed values from the cross-validation of Model 1.1.

[0078]
FIG. 10 is superposed NIR spectra for different environmental light intensities.

[0079]
FIG. 11 is superposed NIR spectra for different sample cap orientations.

[0080]
FIG. 12 is predicted versus observed values from the validation of Model 1.1 for different sample cap orientations.

[0081]
FIG. 13 is predicted versus observed values from the cross-validation of Model 3.0.

[0082]
FIG. 14 is predicted versus observed values from the validation of Model 3.0 for different sample cap orientations.

[0083]
FIG. 15 is superposed NIR spectra for different sample pathlengths.

[0084]
FIG. 16 is superposed transformed spectra after vector normalization pretreatment of the FIG. 15 NIR spectra.

[0085]
FIG. 17 is predicted versus observed values from the cross-validation of Model 4.0.

[0086]
FIG. 18 is predicted versus observed values from the cross-validation of Model 4.1.

[0087]
FIG. 19 is predicted versus observed values from the cross-validation of Model 5.0.

[0088]
FIG. 20 is background spectra at different humidities.

[0089]
FIG. 21 is superposed NIR spectra of vapor phase water at different generated levels of humidity.

[0090]
FIG. 22 is superposed NIR spectra of 1.00% squalane in squalene at different generated levels of humidity.

[0091]
FIG. 23 is superposed NIR background and absorbance spectra of 1.00% squalane in squalene.

[0092]
FIG. 24 is superposed transformed spectra after first derivative and vector normalization pretreatment of the FIG. 23 NIR spectra.

[0093]
FIG. 25 is predicted versus observed values from the cross-validation of Model 7.0.

[0094]
FIG. 26 is superposed NIR spectra of 2.00% squalane in squalene using different fiber optic probes.

[0095]
FIG. 27 is superposed transformed spectra after first derivative and vector normalization pretreatment of the FIG. 26 NIR spectra.

[0096]
FIG. 28 is predicted versus observed values from the validation of Model 7.0 for different instruments.

[0097]
FIG. 29 is predicted versus observed values from the cross-validation of Model 9.0.

[0098]
FIG. 30 is predicted versus observed values from the validation of Model 9.0 for Instrument B.

[0099]
FIG. 31 is predicted versus observed values from the cross-validation of Model 10.0.

[0100]
FIG. 32 is predicted versus observed values from the validation of Model 10.0 for Instrument B.

[0101]
FIG. 33 is a schematic diagram of a flow-through sample presentation system attached to a FT-NIR spectrometer.

[0102]
FIG. 34 is superposed NIR spectra of representative seed samples of a variety of canola.

[0103]
FIG. 35 is predicted versus observed values from the cross-validation of Model 11.0.

[0104]
FIG. 36 is predicted versus observed values from the cross-validation of Model 11.1.

[0105]
FIG. 37 is predicted versus observed values from the cross-validation of Model 11.2.

[0106]
FIG. 38 is an abnormal NIR spectrum resulting from a failed excitation source.

[0107]
FIG. 39 is superposed NIR spectra of the same sample of canola seed taken using different sampling amounts.

[0108]
FIG. 40 is a NIR spectrum of wheat.

[0109]
FIG. 41 is superposed NIR spectra of different samples of a single variety of canola seed different from the canola variety of FIG. 34.

DETAILED DESCRIPTION OF THE INVENTION

[0110] The invention is directed in part to a method of characterizing a material for at least one property of interest comprising acquiring multichannel data on at least one multichannel signal from a sample of the material using a data acquisition device at a location, transmitting the multichannel data along a communication link to a central processor wherein the central processor includes at least one algorithm for manipulating the multichannel data and evaluating at least one multivariate calibration model which can accommodate a range of variables distinct from the variation in the property of interest of the sample, manipulating the multichannel data by the central processor to generate a result predictive of the property of interest of the sample, and forwarding the result from the central processor. The result is typically forwarded to one or more locations remote from the central processor. Preferably, the calibration model can compensate for at least instrument variance. Preferably, the result is transmitted to a user-interface output device in the vicinity of the data acquisition device along the same communication link used for connecting the data acquisition device with the central processor. The result, however, can be forwarded to a different location utilizing the same or a different communication link as desired. Various user interfaces can be provided at one or more locations to enable a user to communicate with the central processor, a data acquisition device, or both. The method of analyzing can encompass a single data acquisition device in a location physically remote from the central processor, or can alternatively include a plurality of data acquisition devices which communicate with the central processor by a communication link. The calibration model loaded onto the central processor is multivariate, and preferably compensates for variations in an effectively comprehensive set of actual measurement conditions and secondary material characteristics.

[0111] In generating this calibration model the secondary variables or influential factors that can significantly impact an instrument's response are identified experimentally. These factors have the potential to influence the results predicted from the model. Generally, it is not possible to determine a priori if a particular variable will influence the results predicted from the model. Thus, these variables are described as potentially influential factors until their status is validated experimentally. It is not necessary to determine the actual physical or chemical cause of a potentially influential factor to identify such a factor experimentally or to generate calibration data resulting from systematic or random variations in the level of such a factor. The modeling process utilized herein accounts for a fundamentally wider range of potentially influential factors, while accommodating for this range of factors in a manner which substantially decreases the amount of time required to maintain the model since the need to develop remedial updates is avoided.

[0112] The invention also relates to the analytical system comprised of at least one data acquisition device in a remote location, and a central processor loaded with one or more calibration model algorithms and connectable by a communication link to the data acquisition device during the measurement process. Preferably, the system also comprises a user-interface that is connectable to the central processor over a communication link and an output device connected to the user interface to receive a result generated by the central processor. Preferably, the output device is located in the vicinity of the detector. Further, the output device preferably is connected to the central processor by the same communication link connecting the data acquisition device and the central processor.

[0113] It is known to employ multivariate calibration modeling in connection with analytical techniques to predict a result. Generally, these calibration models were generated by first determining the property of interest and focusing on variables relating to variation of material characteristics in the population of samples. Generally, attempts were made to eliminate the effects due to other influential variables by controlling those parameters which might vary. The calibration model generated in this way might maintain an acceptable degree of predictability for a period of time, but eventually would reach a point when the accuracy of the predicted values was unacceptable. At that time, the calibration model would have to be regenerated, or the instrument re-calibrated using a remediation update or adjustments to the instrument hardware.

[0114] The multivariate calibration model disclosed herein considers actual measurement conditions as they are expected to occur during future measurements and identifies an effectively comprehensive set of factors or experimental variables which can significantly impact the instrument response. Broadly, the multivariate calibration model employed in the invention is developed by identifying potentially influential factors which may significantly impact the response of the instrument. Experiments are then run to determine whether the potentially influential factors are indeed influential by recording information generated by the instrument, such as a spectrum, at different levels of individual factors or combinations of factors. If this information at different levels of factors shows no significant differences, then these factors are not influential in connection with predicting a value for the property of interest.

[0115] The on-site analytical system can be broadly depicted as shown in FIG. 1. Sensors 2, 4, 6 are examples of data acquisition devices connectable bi-directionally through communication link 8 which in turn is bi-directionally linked through the central processor 10 to pass information. Information traveling from the sensors 2, 4, 6 through the communication link 8 to the central processor 10 is generally identified as measurement data 12. Information transmitted from the central processor 10 through the communication link 8 to user interfaces (not shown) in the vicinity of one or more sensors 2, 4, 6 is generally identified as measurement results 14. Other information such as announcements, status indicators, user preferences, requests for billing information, system usage information and the like may also be transmitted to and from the central processor 10.

[0116] It has been found that utilization of a subset of acquired data from a multichannel instrument, such as a portion or subregion of the available spectrum for a NIR instrument, may generate predicted results that compensate for variations in a wider range of influential factors compared with the results generated from the entire set of multichannel data acquired from a single measurement process for a particular property of interest. A single measurement process may generate a set of multichannel data from the accumulation of multichannel data from individual repetitive runs, and the accumulated data may also be preprocessed. If one or more factors are determined to be influential by experimentation as described above, the instrument response, such as spectral data, may be pre-treated utilizing one or more of several mathematical operations. Pretreatment as used herein encompasses data transformation prior to model prediction. Pretreatment may be used to simplify the data, reduce experimental noise, and eliminate or reduce effects from some secondary variables by mathematical operations such as, but not limited to, filtering the data to reduce its size to one or more smaller subregions of interest and eliminate data from non-interesting or non-beneficial subregions, and applying one or more mathematical transformations such as, but not limited to, weighting, multiplicative scatter correction (MSC), weighted MSC baseline corrections, derivative operations, vector normalization, and standard normal variate and detrend (SNVD).

[0117] Pretreatment is performed in connection with the process of developing and using a calibration model, and may be performed before or preferably after transmission of acquired data to the central processor 10. In addition, if the instrument generates a spectrum, pretreatment generally includes filtering a spectrum by eliminating unwanted data at selected wavelengths or wave numbers for a particular property model. Each spectrum forwarded from the sensors 2, 4, 6 preferably retains the entire available spectral region. Later manipulation of the spectrum at the central processor 10 can then utilize various subregions as necessary to generate the predicted result for one or more properties of interest. In some cases, pretreatment is observed to eliminate differences between multichannel data sets that had been present in specified spectral subregions before pretreatment while maintaining sufficient information to permit prediction to a desired statistical level. Pretreatment may also be used with acquired data in forms other than as spectra. When elimination or minimization of differences in the transformed data is observed after pretreatment, the potential influence of these factors on the predicted result is eliminated or effectively reduced by the pretreatment operation. In the case of multichannel data such as spectra, filtering combined with other mathematical transformations of the data in appropriate spectral subregions may be able to compensate for variations in the level of one or more factors by eliminating these factors as variables of the calibration model.

[0118] If pretreatment does not eliminate or effectively reduce the spectral differences arising from variations in one or more factors, then the training set is expanded to include spectra corresponding to different levels of those factors, depending on the desired level of precision for the predicted result. In this instance, the properly configured expanded training set will enable a model to be developed that will compensate for variation in the level of the factors by including data in the training set relating to the dependence of the predicted result on the factor. Significantly, it is not necessary to quantify each factor being considered. Rather, a relative or semi-quantitative assessment may be made to ensure that the variation in these factors span the range from hottest to coldest, highest to lowest, and the like. The analyst needs only to include measurements taken at different levels of each factor over an expected range, where the expected range encompasses levels only in that part of the entire possible range of a factor that are expected to occur during actual measurements in the future. For example, if a particular sample was being presented to a detector and a spectrum was generated, the orientation of the sample container at the time of data acquisition by the detector might be considered a potentially influential factor in the final instrument response. Assume that different orientations of the sample container at the time of analysis will create spectral differences which either must be eliminated by pretreatment or must be compensated by extending the training set. To effect compensation, it is not necessary, and occasionally it is not even possible, to determine or establish a quantitative measure of the exact orientation of the sample container at the time when the detector is used to acquire data from the sample. Under the modeling procedure utilized herein, it is only necessary to evaluate the sample container orientation at various random positions which would be expected to span the range of orientations during future measurements.

[0119] Generally, the number of levels of a primary or secondary variable which should be selected to span the expected range of variation should correspond at least to two plus the expected complexity of a polynomial approximation of the assumed relationship between observed and predicted values. Thus an assumed, approximately quadratic relationship should have four levels of the factor at a minimum.

[0120] As used herein, a property model is an algorithm (or computational procedure) for generating the predicted value of a property of interest for pre-processed data developed from a training set. The property model algorithm is the combination of pretreatment of pre-processed data, followed by evaluation of a calibration model to generate a predicted value from the pretreated, pre-processed data. The property model may be evaluated according to a single mathematical relationship between the instrument response and the property of interest spanning the entire region of the multichannel data, or piecewise from two or more mathematical relationships in various subregions of the multichannel data. In all cases, the algorithm generates a single predicted result that does not depend on knowledge of the particular data acquisition device providing the multichannel data. A global property model is a property model developed from a global training set. Note that a global training set of a property model needs to include data spanning the expected range of only those influential factors that have not been compensated by the pretreatment operations of the property model.

[0121] A block diagram showing additional detail of the components and information transmission flow within a data acquisition device is shown in FIG. 2. Sample 20 is deposited into or flows through the sample chamber 24 of the sample presentation device 22. If required by the analytical method, an excitation source 26 is utilized to supply radiation or other form of energy 40 to the sample 20 in the sample chamber 24, the resultant radiation or non-stimulated emission being raw data 42 received by the detector 28. The detector 28 and, if present, the excitation source 26 are located within the analytical instrument 30, which also includes associated components required to generate an instrument response. Information from the analytical instrument 30 is then transferred to the local processor 34 with user interface 32 in the form of digitized data 44 where pre-processing steps may be performed. The user interface 32 passes measurement data 12 to the central processor 10 (not shown) via communication link 8. Measurement results 14 are received from the communication link 8 back to the local processor 34, wherein optionally the resultant information is presented at an output device (not shown) of the user interface 32.

[0122] Pre-processing as used herein encompasses the transformation of raw data prior to communication over a communication link 8 using instrumental transduction and computation steps to a form more readily assimilated by the central processor 10. Some pre-processing steps may be performed by the analytical instrument 30 and other pre-processing steps may be performed by the local processor 34. Pre-processing is independent of the process of developing or using a calibration model.

[0123] Generally, the generation of predicted values is initiated at the sensor 2 as an example of a data acquisition device. The bi-directionality providable through the communication link 8 can permit the central processor 10 to submit a reminder or other trigger signal to the user interface 32 to initiate the acquisition of data by sending an instrument control signal 46 to the analytical instrument 30. For example, detection of a probable outlier in some measurement results 14 at the central processor 10 can cause the central processor 10 to forward a signal to a user interface 32 of the local processor 34 of a need for additional measurement data 12 on the same sample or a new sample. Alternatively, the scan of a sample may desirably be semi-automated so that additional scans are initiated at regular time periods after an initial scan. This prompt can originate at the user interface 32, or it can originate at the central processor 10 which in turn can forward the prompt to the user interface 32.

[0124] Specific combinations of analysis system components have been described. However, it is anticipated that modifications to these combinations will also perform satisfactorily, and are considered to be part of the invention. As an example, the sensor 2 has been described as including a sample presentation device component 22, an analytical instrument component 30 consisting of an excitation source (or other radiation or energy-generating unit) 26 and a detector 28, and a local processor 34 with user interface 32. The sensor 2 initiates data acquisition on the sample 20 in the sample presentation device 22 at the command of the user interface 32, and transmits the acquired measurement data 12 via a communication link 8 to the central processor 10.

[0125] It can be appreciated that the sensor 2 does not need to contain the sample presentation device 22, analytical instrument 30, and local processor 34 in a single enclosure for the system to be able to operate. The sample presentation device 22, analytical instrument 30, and local processor 34 can each be stand-alone units if this becomes necessary or desirable, and combinations of these components can be assembled. The desired method operations occurring in the vicinity of the sensor 2 include: data acquisition of a sample 20 of a material, and transmission of an optionally pre-processed form of the acquired data to the central processor 10. Preferably, the method also includes generation of a predicted result using a property model algorithm stored in the central processor 10 and transmission of the measurement results 14 to one or more user interfaces. The sample may be located in the sample chamber 24 of a sample presentation device 22, but some methods of data acquisition may not require this such as when a probe is used to acquire data from a portion of complete living organisms, such as an ear of corn or intact human skin. The data acquisition may be initiated manually by an operator or automatically from a control signal from instrument control 46 transmitted by the user interface 32. The user interface 32 may reside in the local processor 34, but various input and output devices of a user interface may reside in other locations either internal or external to the data acquisition device 2. The local processor 34 may be a stand-alone unit such as a laptop or desktop personal computer or an assembly of components including one or more computer chips, read-only memory, firmware, and the like located within or external to the data acquisition device. Pre-processing of the acquired data from the measurement process is optional. Further, the measurement results 14 are not necessarily transmitted to the user interface 32 of the local processor 34. Measurement results 14 may be forwarded to one or even several locations removed from the operator at the instruction of the customer.

[0126] For FT-NIR sensors, the raw detected data is in the form of individual scans of interferogram spectra which may be combined by accumulation of data from repeated individual scans. Instrument operation software, such as the Bruker OPUS product, installed on the local processor 34 of the sensor 2 averages the acquired multi-scan spectra in the interferogram mode, converts this average interferogram to a single-channel spectrum by fast Fourier Transform, and converts the single-channel spectrum to an absorbance spectrum according to the equation:

absorbance spectrum = - \log (single - channel spectrum / background spectrum) .

[0127] This calculation removes eccentricities in the sample spectrum attributable to the background spectrum specific to that sensor 2. It is noted that a “single-channel” spectrum is an alternative representation of a spectrum that contains a plurality of data values and, thus, a single-channel spectrum is still a multichannel data set. A background spectrum is acquired by operating the instrument 30 either with no sample in the sample chamber 24 or with a data acquisition probe in direct contact with a reference material such as the reflective surface of a mirror or a spectralon composed of polytetrafluoroethylene. The acquired background spectrum provides a digitized spectrum 44 which is then used to pre-process spectral data according to the above equation. A digitized background spectrum is preferably stored in the local processor 34 of each sensor 2, 4, 6 although other storage locations may be used. Thus, a single background spectrum is generated and stored for each sensor 2, 4, 6 for use in generating a plurality of predicted results from one or more property models in the future. The single background spectrum is generally acquired from a single data acquisition device, but may be generated from an accumulation of background spectra acquired from two or more data acquisition devices. The background spectrum is generally stored in the local processor 34, but may be stored in the central processor 10 or another location remote from the data acquisition device that can be connected to the data acquisition device by a communication link 8.

[0128] A material as used herein encompasses any object for which it is desired to generate a value of a property. The value may be a measurement for which a quantitative result is desired. The value may alternatively be a qualitative one, indicating only the presence or absence of a component. The material may be in any physical form, i.e., gaseous, liquid, solid, or mixed phases, and may encompass both discrete units or components thereof, such as either a whole oilseed or the oil expressed from the seed, and may consist of mixtures of different substances, such as foreign matter in a sample of whole grain. In addition, for certain types of analyses the term may encompass living plant or animal matter, such as human tissue or fluids. The sensor 2 may optionally include an excitation source 26, a sample handling device 22 to present the sample 20 to the detector 28, and associated electronics to convert the detector output into a digitized format 44. A wide range of sensors of various types, herein called sensor-types, can be used to acquire data for subsequent analysis, including but not limited to those for NIR, mid-IR, Raman, UV, visible, and NMR spectroscopy, liquid or gas chromatography, or mass spectrometry. Other spectroscopic and non-spectroscopic types of sensors may also be used.

[0129] If sensors of different types are used for on-site analysis, sensor-type-specific property models are required. A sensor-type is a type of instrumentation that acquires multichannel data based on a specific analytical method, such as Fourier-Transform NIR or gas chromatography. It is preferred that different instruments within a single sensor-type are manufactured according to well defined design specifications as is done for specific models of an instrument, usually by a single manufacturer, such as the MATRIX Model F FT-NIR spectrometer manufactured by Bruker Optics.

[0130] In one embodiment of the present invention, the detector information acquired on a sample 20 may optionally be both pre-processed and converted into a digital format to facilitate rapid communication with the central processor 10 and subsequent data processing. While another embodiment could utilize the transmission of unprocessed multichannel data, the preferred embodiment is advantageous in that digitization and averaging occur prior to transmission of data to the central processor 10.

[0131] The user interface 32 may be installed in an apparatus physically attached to, or integrated with, the sensor 2, but this is not required. Generally, an output device of the user interface 32 is located in the vicinity of the sensor 2 because the resultant information will typically be desired at the location where the data on the particular material is acquired. For example, in the case of the characterization of a property of interest for oilseeds, the measurement data 12 may be acquired at a storage silo, or near a transport truck, and the measurement results 14 are returned to the operator at these locations. Alternatively, the measurement results 14 may be disclosed at an office near the storage silo with optional ancillary equipment such as a printer to generate a written record of the generated result. As a further alternative, however, the measurement results 14 may be disclosed at an administrative or processing location of the customer which is geographically far removed from the storage silo location where the measurement data 12 was acquired. It is possible, though not preferred, to communicate with the operator or other recipient of the result in the vicinity of the sensor 2 by a communication link independent from that of the link 8 connecting the sensor 2 and the central processor 10. Thus, measurement results 14 generated by the central processor 10 may be communicated to the operator or another designated party in an indirect manner such as telephone or facsimile transmission even where the sensor 2 and central processor 10 are linked via the Internet.

[0132] The hardware of the system architecture is characterized by an unusual master-slave relationship established between the one or more sensors 2, 4, 6 and the central processor 10. Since data acquisition is initiated at the sensors 2, 4, 6 the central processor 10 becomes a slave to the sensors 2, 4, 6 in the field. The sensors 2, 4, 6 do not operate as self-contained analyzers, but are dependent on the central processor 10 for data analysis. Thus, the sensors 2, 4, 6 are dumb masters and the central processor 10 is a smart slave in a many-to-one relationship.

[0133] In another embodiment of the system architecture, the central processor 10 sends one or more property models (as, for example, an information packet of parameters sufficient to define one or more property model algorithms) to at least one sensor, e.g., 4, over the communication link 8, at various intervals as desired to enable at least sensor 4 to perform local computations of measurement results. In this mode of operation, the sensor 4 is a self-contained analyzer, although all sensors 2, 4, 6 still use the same property model algorithm for a particular property of interest. Since the algorithm does not contain any instrument-specific parameters, this embodiment is different from calibration transfer and instrument standardization methods which attempt to compensate for instrument variance by developing and transferring instrument-specific calibration models for use by specific instruments. This embodiment is useful as an alternative strategy for situations in which it may not be possible or desirable to use a communication link 8 for real-time model calculations.

[0134] This alternative embodiment can be used as a backup strategy to enable measurements to be performed when interruptions in the communication link 8 have existed or may be expected. This embodiment can also be used for measurements in remote locations where it is impossible, difficult, expensive, or inconvenient to use a communication link 8 to the central processor 10. This mode of operation may enable alternative productive analysis methods as electronic components get smaller and faster, particularly for situations where very rapid response times are desired or when it is desirable to avoid sending data or results over a communication network 8. Presently, this embodiment is less preferred because an update to the property model is not automatically available to the sensor 4. Nonetheless, this embodiment retains the advantage of a single property model algorithm at a point in time from which predictions are made.

[0135] In another embodiment, the current version of the property model algorithm is transmitted to the sensor 4 immediately prior to the data acquisition step of each measurement process to ensure that the measurement results are computed from the most recent update to the property model. In these alternative embodiments, the measurement data and locally computed measurement results may be transmitted to the central processor 10 for storage and distribution at a later time.

[0136] The user interface 32 located near the sensor 2 can provide a selectable menu of properties of interest that are available at the central processor 10. Prior to each measurement, for example, the central processor 10 can transmit the current list of available properties of interest to the sensor 2. Then, the user will always access from the updated selectable menu displayed at sensor 2 the most current list of available properties, and thereby use the most current revision to all property models without needing to manually install software updates to replace, change, add, or delete models as would need to be done if the models were stored in computing devices individually connected to each sensor 2.

[0137] Where multiple sensors 2, 4, 6 may be used, the sensors 2, 4, 6 may be located at a variety of distances from the central processor 10 as needed to provide for on-site analysis. The sensors 2, 4, 6 are typically remote or distant from the central processor 10, where remote refers only to the existence of a physical separation between the sensors 2, 4, 6 and the central processor 10. A remote sensor 2, 4, or 6 does not in any way require that the on-site location is isolated geographically or technologically. Indeed, one or more remote sensors 2, 4, 6 can be located in a central laboratory in a large metropolitan area as well as at isolated sites far removed from population centers.

[0138] One example of a communication link 8 between sensor 2 and central processor 10 which may be utilized is the Internet. In operation using one or more NIR spectrometers as analytical instruments 30, the on-site analysis system utilizes a user interface 32 running in a browser installed in the local processor 34 which performs a security logon function, presents the operator with input fields to identify the sample being analyzed, and prompts the loading of the sample 20 into the sample presentation device 22. The security logon process requires both a password known only to the operator and a sensor 2 identified electronically by a serial number. This logon therefore requires both a specified piece of hardware and an independent password.

[0139] The user interface 32 then presents the operator with input fields to identify the sample 20 being analyzed. Examples of descriptive data collected about the sample 20, herein called sample identification data, include but are not limited to the type of sample being tested, the location where the sample was collected, and a unique identity of the sample. The operator is then prompted to load the sample 20 into the specific instrument sample presentation device 22 and to start the collection of measurement information from the multichannel analytical instrument 30, this information herein called multichannel data. In the case of a near-infrared spectrometer, the multichannel data collected is pre-processed spectral data. The measurement data 12 comprises the sample identification data and the multichannel data, which is sent over the Internet 8 to the central processor 10 for analysis. The information packet of measurement data 12 is processed through routers and firewalls where it is received by a standard web server such as the Microsoft® Internet Information Server.

[0140] The information packet is accepted at the central processor 10 by a web server which initially processes the information and forwards this information to a queuing system. The queuing system accepts near-simultaneous transmissions from multiple sensors and queues the submissions to be processed in FIFO (first in first out) order by the central processor.

[0141] The analysis engine of the central processor 10 accepts the information packet of measurement data 12 sent by the queuing system and opens the packet to find both the multichannel data to be analyzed and sufficient descriptive information entered by the operator in the sample identification data to select the proper property model to be used for analysis. After calculating the predicted results through the proper property model, the measurement results 14 are then passed back to the queuing system for communication over the Internet to the operator at the user interface 32, or optionally to another user interface at a location in the same vicinity, a field office of the customer, or another alternate location. The measurement data 12 and the processed measurement results 14 are preferably stored in a database, called the data repository, where they may be retrieved for later reference if desired.

[0142] The communication link 8 used in transmitting data and results, briefly discussed above, is considered in more detail. The communication link 8 may encompass any device or communication system which can provide for information transfer within acceptable limits for signal degradation and transmission speed. For example as discussed above, the on-site location sensor 2 and central processor 10 may be connected to one another via a global, public network such as the Internet. Alternatively, a local or global private network or combination of public and private communication links may be employed. Communication between the sensor 2 and central processor 10 can be enabled by a user interface 32 consisting of an emulated instrument panel or graphical user interface which runs on a standard Internet browser executing in the local processor 34. The local software component of the instrument panel is a set of HTML-based code that includes additional code such as Java® as an example of client-based software that communicates with the instrument 30. This set of software code is configured to be able to communicate with instruments from multiple manufacturers, or multiple instrument designs from a single manufacturer. Thus, though the sensor 2 may include a near-infrared device, the host-based software component of the instrument panel executing in the central processor 10 is designed to accept input from other spectroscopic and non-spectroscopic devices. The user interface 32, running within a standard Internet browser, is an interactive software application that can be run from any platform that can host a browser, including but not limited to desktop personal computers and laptops, as well as various wireless devices.

[0143] Within a sensor 2, the data acquisition portion of the system is generally the detector 28 of the analytical instrument 30 and the user interface 32. Note that the local processor 34 may be a separate, stand-alone device or may be integrated into the device 2. In either case, the sensor 2 is generally considered to comprise at least the detector component 28 of the analytical instrument 30 and the local processor 34 that runs the user interface 32. The user interface 32 contains the entry point to the system sign-on, input fields for customer authentication and system usage authorization, and other miscellaneous interfaces such as system status, announcements, access to authorized portions of information in a data storage unit to provide additional information reporting capability, and help text to provide users with specific operating instructions. The user interface 32 running on the browser of the local processor 34 of the sensor 2 connects to an analytical instrument 30 such as a NIR detector, in which case the user interface 32 controls data acquisition from instrument 30 and the transmittal of that data over a communication link 8, i.e., the Internet in this instance, to the central processor 10.

[0144] The system architecture based on use of the Internet as the communication link is shown in FIG. 3. This diagram shows software components of the hardware illustrated in FIG. 1. Data acquisition by the analytical instrument 30 is initiated by the user interface 32. Connection between the user interface 32 and the central processor 10 is provided by the ASP (active server page) 50 running on a web server and controlling the communications. The central processor 10 is comprised of the ASP 50, a transaction queuing system 52 that provides scalability for transaction volume, the security controller 54 for authenticating and granting processing rights to incoming transactions, the analysis engine 56 which requests parameters of property models from the data repository 58, performs pretreatment operations, and computes the predicted value for each property requested, and the data repository 58 which stores transaction input information, or measurement data 12, and transaction output information, or measurement results 14, for providing potential added informational value to the customer. The data repository 58 may be a SQL database, although other types of databases may be used. It is preferred that the database be relational. Integrated into the central processor 10 is a security architecture that protects proprietary rights to the various data types stored in the data repository 58 of the central processor 10 and delivers information only to those properly authorized to receive the information.

[0145] The queuing system 52 forwards the measurement data 12 to the analysis engine 56, upon which the analysis engine 56 requests and obtains the parameters defining one or more property models from the data repository 58 as specified by the descriptive information in the measurement data packet 12. The analysis engine 56 performs calculations comprising pretreatment, property prediction, and associated statistical measures of the property prediction such as the Mahalanobis distance. The queuing system 52 requests that the data repository 58 accept the measurement results 14 for storage and requests that the web server 50 accept the returned measurement results 14 for transmission to the user interface 32, to one or more other user interfaces, or both.

[0146] To enable multiple sensors 2 to communicate with the central processor 10, the transaction queuing system 52 of the central processor 10 performs a function of accepting a high volume of near-simultaneous communications from multiple locations over the communication link 8 and controlling the communication flow to the analysis engine 56 and relational database 58. The queuing system 52 is described in detail in a separate patent application, entitled “Extensible Modular Communication Executive With Active Message Queue And Intelligent Message Pre-Validation, by James Thomas Kent, et al., filed on even date herewith, Serial No. 60/307,347, which is incorporated herein by reference.

[0147] In preparing to conduct an analysis of a material at a variety of remote locations, it is necessary to generate a calibration model which addresses all factors that may significantly influence the measured properties. As noted above, such modelable factors occur in the following areas: the material to be analyzed; the instrument; the environment; and the material presentation. Operator-to-operator variations in the measurement process are generally captured by influential factors associated with the material presentation. The desired reliability of the analysis dictates the number of factors within each of these areas which should be anticipated and modeled.

[0148] The development of calibration models is generally done using one or more sensors that are not necessarily connected to the central processor 10. Furthermore, the calibration models are developed and validated using chemometric software such as OPUS Quant-2 and Ident on a computer that may be different from the local processor 34 of a sensor. Validated calibration models are loaded into the data repository 58 of the central processor 10 to enable on-site measurements by remote sensors 2. In the course of generating an instrument response relating to the property of interest of a sample 20 of a material, using one or more sensors not necessarily connected to the central processor, a univariate calibration would be used if the instrument response was dependent only on the property of interest. Unfortunately, it is rarely possible to obtain ideal measurements of a property where the measurement process is selective for just the property of interest. Realistically, particularly to build models suitable for use by non-specialists, additional variables must be taken into account to reflect the realities of generating a calibration model where variations may be expected to occur in the instrument, the environment and in the sample presentation, in addition to variations between and within samples of material. In addition to random measurement noise, the data may be affected by chemical and physical interferences. Chemical interferences alter the instrument response due to the presence of chemical impurities in the material being tested, inhomogeneities in the distribution of chemicals in a mixture, and the like. Physical interferences alter the instrument response due to, for example, light scattering effects and instrument variances.

[0149] A flowchart for establishing the feasibility of developing a property model is provided in FIG. 4. This process is used to determine if a selected measurement method using a specific sensor-type is capable of being used to predict a property of interest over a desired range of that property. Initially, as shown in block 70, the method and objectives are defined. The method of block 70 refers to the analytical method, such as FT-NIR or gas chromatography, and includes specifying the analytical instrument, such as a Bruker MATRIX Model F FT-NIR spectrometer, and the sample presentation, such as 0.5 to 1.5 mL of liquid contained in a metal closure cap with an 18 mm diameter. The objectives of block 70 define expectations for a property model. These objectives identify the property of interest, the desired precision for predictions of the property of interest, the calibration set upon which a feasibility assessment will be made, and the expected ranges of the secondary variables related to the samples and measurement conditions as specified by the customer and as recommended by the analyst responsible for model development. Then, as shown in block 72, the expected range of the primary variable is defined from the objectives of block 70, and a preliminary set of calibration samples is obtained that span the expected range of the primary variable as shown in block 74. The number of samples in this preliminary calibration set typically ranges from about 5 to about 50.

[0150] Next, a preliminary model is built for the primary variable, shown in block 76. At this initial stage of model development, the potentially influential factors are not intentionally varied. Measurements at this stage are taken under ambient conditions with a single instrument as defined by the method and objectives 70. The training set used in block 76 may be generated by taking a single measurement of each sample in the calibration set of block 74 or the training set may be expanded to include repeated measurements of some or all of the calibration set of block 74. Each instrument response in a training set may be generated from the multichannel data acquired during a single multichannel measurement or during two or more multichannel measurements. In some cases, it may be desirable to combine or accumulate the multichannel data acquired from two or more multichannel measurements of different samples, different measurement conditions, or both, into a single set of multichannel data comprising the instrument response.

[0151] Using the symbol “y” to denote the property of interest, the following definitions apply in further considering the calibration process:

1yobs(i)=the ith observed or known value of propertyy. This is also called the true or expectedvalue.ypred(i)=the ith predicted value of property y. This isalso known as the measured value.Resi=ypred(i) − yobs(i), and is known as the ith residualor deviation between predicted andobserved results.M=the number of predicted values in theinstrument response set.SSE=MΣ(Resi)2i = 1=sum of squared errors.RMSECV=square root of SSE/M for the training set=root mean square error of cross-validationwherein the calibration model is generatedwith a training set and predictions from thismodel were made using the same trainingset.RMSEIR=Square root of SSE/M for the instrument-response set=root mean square error of predictions fromthe instrument-response set.RMSEP=square root of SSE/M for the validation set=root mean square error of predictionwherein the calibration model is generatedwith a training set and predictions weremade on a validation set.r2=coefficient of determination; or square ofthe correlation coefficient (r), whichprovides a measure of the degree ofcorrelation between yobs(i) and ypred(i).R2=100 r2

[0152] Using commercially available chemometric modeling software such as HOVAL software (such as Version 1.6, 1992), AIRS software (such as Version 1.54, 1999) or Bruker OPUS-NT Quant-2 software (such as Version 3.01, 2000), the initial version of the calibration model is generated, as indicated in block 76, from a training set in which the primary variable spans the anticipated range over which this variable is expected to vary during actual measurements in the future. The coefficient of determination, r2, is used to determine whether the correlation is adequate to measure the property, as shown in block 78. Generally, if r2 is less than about 0.6, or equivalently if R2 is less than about 60, it will be necessary to consider adjusting the method or objectives as shown in block 80. During this adjustment step, a procedure for sample preparation or impurity reduction may be defined or refined, the preliminary calibration set may be modified, or a different method may be selected. If the method or objectives can be altered, then the new definitions are adopted as shown in block 82 and, according to the expected range of block 72 and using a preliminary set of calibration samples from block 74, another preliminary model is developed in block 76. The coefficient of determination, r2, of the training set of block 76 is used to decide whether the property of interest is measurable according to the method and objectives of block 82. As long as the property is not measurable (block 78) and the method or objectives can be adjusted (block 80), blocks 82, 72, 74, 76, and 78 are repeated. If the method or objectives cannot be adjusted (block 80), then model development is not feasible (block 98) in accordance with the definitions of block 70 as adjusted by block 82.

[0153] If it is determined that the correlation is adequate (block 78), specifically if r2 is greater than about 0.6, although smaller or larger values can be used depending on specific objectives in blocks 70 and 82, it is considered feasible to begin testing for potentially influential factors. The next step is to identify the types and expected ranges of the potentially influential factors as shown in block 84. Next, as shown in block 86, one or more potentially influential factors are selected for experimental investigation. In block 88 experimentation is conducted to determine whether the factors are indeed influential using a small validation set. In block 90 the preliminary model is revised to compensate for secondary variables that have been experimentally demonstrated to be influential factors and appropriate methods of pretreatment are identified when possible. A decision is then made in block 92 to determine if the property of interest is still measurable in the presence of variations in the secondary variables. The measurability of block 92 is determined by comparing the RMSEP of the validation set of block 88 with the desired precision specified in the objectives of blocks 70 and 82. The property is considered measurable in block 88 if the RMSEP is less than or approximately equal to the desired precision value. If not measurable, the method or objectives for developing the calibration model are adjusted as in block 82, if possible (block 80), and the process is repeated beginning again with block 72. If, in block 92, the property of interest is still measurable to the limit of desired precision specified in the objectives of blocks 70 and 82, and if there are more factors in block 94 that have not been investigated experimentally, then the feasibility process returns to block 86 where one or more additional, potentially influential factors are selected, experimentation is conducted in block 88 to determine those additional factors that are influential, the model is revised, methods of data pretreatment are identified in block 90, and a determination is made in block 92 whether the property of interest is still measurable to the limit of desired precision. The process of selecting potential factors (block 86), experimentally determining influential factors (block 88), revising the model and identifying pretreatment methods (block 90), and determining if the property is still measurable to the limit of desired precision (block 92) is repeated until there are no more potentially influential factors to consider in block 94. If the property of interest is ultimately determined to be measurable (block 92) in the presence of variations in the secondary variables identified in the objectives of block 70 as adjusted by block 82, and if there are no more potentially influential factors to consider in block 94, then model feasibility is established as shown in block 96.

[0154] The process for identifying influential factors and revising the calibration model to accommodate secondary variables that are demonstrated to be influential factors experimentally is considered in detail below.

[0155] To determine if a factor has an influence on the predicted result of a model (block 88), a small validation set is developed which includes measurements at some different levels of the potentially influential factor. Specifically, one or more samples with yobs(t) for the property of interest in the presence of a range of values for the secondary variable being considered are measured with the instrument 30 (FIG. 3), and the ith instrument response is used to generate ypred(t) using a previously generated calibration model. The RMSEP of this validation set is then calculated using this previously generated calibration model to serve as an estimate of the level of precision of the property model in the presence of variations in secondary variables. During this feasibility assessment process, it may be sufficient to use a validation set containing a single pair of observed and predicted values, and use the absolute value of the difference between these values as a single-point estimate of RMSEP. It can be appreciated that different factors will have different degrees of influence on the property of interest values being predicted by the property model. In some applications the desired level of precision as specified in the objectives of blocks 70 and 82 will be very high, and factors having even a very minor influence will be considered in generating the model. In other instances where the desired level of precision is lower, the factors having low levels of influence may be ignored, with consequent losses in precision.

[0156] One method of quantifying an acceptable or desired level of precision involves consideration of a confidence interval for predictions of a property of interest. A confidence interval is a range of predicted values that includes the true average value of the property of interest a specified percentage of the time. Thus, for example, if the average value of a property is 3.5±0.6 at the 95% confidence level, then 95% of the time the predicted value of the property is expected be in the interval from 2.9 to 4.1. Since the average value of a number of measurements tends to follow a Gaussian distribution, the confidence interval can be expressed as an average value plus or minus a multiplicative factor times the standard deviation of the average value, where the multiplicative factor can be obtained from statistical tables that relate this factor to the area under a standard Gaussian distribution curve. For example, when the multiplicative factor has a value of 1, 2 or 3, the corresponding area under the standard Gaussian distribution is about 68%, 95% or 97.7%, respectively. So, if it is desired to have about 95% of the future predicted values fall in the interval from +0.6 to −0.6 of the true average value of a property, then the multiplicative factor would be 2 and the standard deviation of the average value should be 0.3, which is one-half of the desired precision of 0.6. The standard deviation of the average value can be estimated, for example, by the RMSEP of a validation set. As a result, the desired precision can be expressed as a multiplicative factor times RMSEP. Thus, a desired precision of ±0.6 at the 95% confidence level is equivalent to specifying that the RMSEP should be less than or equal to 0.3. This is equivalent to specifying the desired precision of ±0.6 as 2 times an RMSEP of 0.3 at the 95% confidence level. This is also equivalent to specifying the limit of desired precision as a RMSEP of 0.3 at the 95% confidence level.

[0157] In general, varying the level of a potentially influential factor produces instrument responses for the validation set that, after pretreatment and evaluation by the calibration model, predict values of the property that differ numerically from the corresponding known values. The numerical differences may or may not be statistically significant. If the RMSEP value computed from these numerical differences is less than or approximately equal to the value used to specify the limit of desired precision, the numerical differences are not statistically significant to that limit of desired precision. Alternatively, if the RMSEP value is greater than the limit of desired precision, the numerical differences are considered to be statistically significant. In those cases when the numerical differences are not statistically significant as measured by RMSEP, the results predicted from the property model are considered to be statistically equivalent. Thus, when varying the level of a potentially influential factor to generate a validation set in block 88 produces statistically equivalent results, the factor is considered to be non-influential for that property model to within the limit of desired precision. Alternatively, if the numerical differences as measured by RMSEP are statistically significant, then the factor is influential and the property model needs to be revised in block 90. If the revised model generated in block 90 then produces results that are not statistically different from the known values, the property is still measurable to the desired precision in block 92.

[0158] Acceptable levels of precision as measured by RMSEP fall within the range of values defined by the particular client acquiring the predicted results as established in blocks 70 and 82. If acceptable precision is obtained in the experimentation of block 88, predictions can be made with that calibration model independent of changes in the secondary variable, the associated factor is considered to be non-influential for that model, and the model does not need to be revised in block 90. The RMSEP is one calculated value which is used to evaluate the level of influence of a factor. A greater increase in RMSEP from a validation set relative to the RMSECV of a training set indicates a greater level of influence. If the RMSEP from the validation set of block 88 does not meet the objectives of blocks 70 and 82, then the model will need to be revised in block 90. Again, the ultimate desired precision of the model will determine if the influence due to a particular factor will require compensation in the development of the model.

[0159] It can be appreciated that a factor which is ultimately determined to be influential in connection with predicting one property of interest of a sample may not influence the prediction of another property of interest. Thus, the determination of whether a factor is influential is dependent on the specific circumstances of the acquisition of data. Furthermore, it is not required to quantify each of the secondary variables. Thus, it is not necessary to determine the actual humidity inside the measuring instrument, or determine the actual age of the excitation source, such as a light used in a near-infrared spectrometer, as long as data points spanning the expected ranges of humidities and light ages are included in the training set or the validation set, unless pretreatment eliminates such factors. Quantifiable values for the secondary variables are not required but may be recorded if the description of a value is desired.

[0160] The model considers a range of variables wherein the range is defined by the type of analysis being made and the expected range of measurement conditions for the particular property of interest. The primary variable is directed to the particular property of interest. The secondary variables may include but are not limited to the following directed to secondary material characteristics of the sample: impurities or other components in the material to be tested; the form of the sample, i.e., solid, liquid, gas or mixed phases; the presence of turbulence in a gaseous or liquid sample; the presence of multiple phases in a sample, such as gas in a liquid, liquid in a solid, hydrophilic and lipophilic liquids combined as an emulsion, or solids dispersed in a liquid or a gas; the particle size distribution of a solid sample; the presence of inhomogeneities in a sample regardless of form; the distribution of shapes of solid particles; the degree of compaction of a sample of solid particles; the tendency of a sample to alter its composition and structure during the measurement process, such as by the formation of hydrates or oxides in a humid air environment; the tendency of a sample or components thereof to evaporate, sublime or decompose; and the tendency of one or more components of a liquid or gaseous sample to settle, or stratify.

[0161] The secondary variables also may include but are not limited to the following directed to the instrument: changes in one or more mechanical, optical or electrical components which would impact signal detection or conversion of the input signal into a format suitable for subsequent processing. These changes include inter-relationships between components such as positional and orientation relationships, and electrical, optical or mechanical interactions in the assembly of the components. For many of the instrument components the effect of age is negligible or unlikely to affect the instrument response, such as regarding the instrument housing or the overload protection circuitry. Other components will create a more dramatic and definite effect on the sample output signal over time, such as a light source in a spectrometer.

[0162] The secondary variables further may include but are not limited to the following directed to the environment and the sample presentation: temperature of the sample; temperature, humidity and atmospheric pressure of the environment in the vicinity of the test instrument; humidity inside the test instrument; airflow in the vicinity of the test instrument; background radiation in the vicinity of the test instrument; the dimensions of the sample container or detector chamber as they affect the pathlength of the excitation beam through the sample; the distance between sample and detector; the presentation speed or flow of the sample relative to the detector and relative to the data acquisition rate of the detector; and the amount of sample presented to the detector, expressed as volume, weight, surface area, pressure, and the like.

[0163] Each of the above variables are considered as potentially influential factors. The actual significance of a secondary variable is determined during the process of developing the calibration model for a particular property of interest.

[0164] In connection with developing and using a calibration model, reference is made to the flowchart in FIG. 5. Initially, the feasibility of establishing a calibration model is determined in block 100, which consists of at least blocks 70, 72, 74, 76, and 78, as well as blocks 80 and 82 as required, of FIG. 4. In addition, after the property is determined to be measurable in block 78, one or more potentially influential factors is investigated experimentally as indicated in blocks 84, 86 and 88. Revisions to the preliminary model in block 90 can be undertaken either by making several stepwise adjustments for one or a small number of factors determined to be influential in block 88, or by adjusting for all experimentally determined influential factors at one time. During these feasibility steps, essential characteristics are identified for the training set along with preliminary methods of pretreatment. The essential characteristics of a training set include the range of material characteristics that must be represented by samples in the calibration set, and the levels of influential factors that must be represented by instrument responses generated from the calibration set. The preliminary methods of pretreatment are those mathematical operations that must be used on optionally pre-processed instrument responses to compensate for influential factors that are not necessarily represented by instrument responses in the training set.

[0165] Considering this information, in block 102 of FIG. 5, a training set is defined and a method of pretreatment is determined in order to begin the process of revising the preliminary model. Then, in block 104 the ith revised model is built and RMSECV(i) is computed, where i equals one for the first revision and i is incremented by one for each subsequent revision. In block 106, if no outliers are found in the training set, then a validation set is defined and the methods of pretreatment are adjusted as shown in block 108. The ith revised model is then validated and RMSEP(i) is calculated in block 110. Next, if outliers are not found in the validation set using this model as indicated at block 112, then a decision is made at block 114 whether to install the new property model in the data repository (block 58, FIG. 3) of the central processor 10. The outcome of block 114 depends on whether the model has been constructed to compensate for an effectively comprehensive set of influential factors. If no, then the process returns to block 108, the validation set is defined to include variations of one or more additional influential factors, the pretreatment is adjusted, and the process continues at block 110. If yes, then the new property model is installed, and on-site measurements can be taken using the new property model as indicated at block 116.

[0166] If probable outliers are found in the training set at block 106 or the validation set at block 112, then the outliers are classified at block 123 and a determination is made if the probable outliers are good outliers at block 124. If any detected outliers are good, then a decision is made to determine whether to extend the training set at block 126. If yes, then the process returns to defining which good outliers will be added to the training set and an appropriate method of pretreatment is determined for this training set, as indicated in block 102. After building the next, or ith, revised model and computing RMSECV(i) in block 104, the process continues forward with block 106. If the decision is made not to extend the training set in block 126, or if the probable outlier is not good at block 124, then a decision is made whether to correct or improve the training set in block 128. If erroneous data are found in the training set or the validation set, or if improved known values become available, then the training set is re-defined with the improved or corrected data and the process resumes at block 102. If the decision is made not to extend the training set in block 126, and if there are no corrections or improvements to be made in block 128, then if the current, ith revised model has not been validated in block 130, the process resumes at block 108. The validation steps in blocks 108, 110, and 112 are repeated until no outliers are found in block 112, and a decision is made at block 114 whether to install the new property model. Alternatively, if the decision is made not to extend the training set in block 126, and if there are no corrections or improvements to be made in block 128, and if the current, ith revised model has been validated in blocks 108, 110, and 112, and if it is decided to install the new property model in block 114, the next step is to proceed with taking on-site measurements at block 116.

[0167] While taking on-site measurements, block 116, as well as while building at block 104 and validating at block 110 a property model, the Mahalanobis distance, MAH, is computed for each predicted value. If the MAH is greater than the threshold value for good outliers of that property model, the predicted value is considered to be a probable outlier. Thus, the determination of whether a predicted value of an on-site measurement at block 116 is a probable outlier at block 118 leads to two possible outcomes. If yes, then the predicted result is probably invalid and MAH is used to classify the prediction as a particular type of probable outlier at block 122. If no, then the prediction is presumably valid at block 120. In either case, the results are reported. If a probable outlier is detected, the results include a descriptive interpretation of the type of probable outlier at block 120. The analysis system is then ready to process the next on-site measurement at block 116.

[0168] The descriptive text categorizing the predicted results at block 120 depends on the MAH of the predicted result and on a set of previously determined threshold values of a property model. If, for a particular property model, the threshold value for good outliers is 0.4, the threshold value for bad outliers is 1.0, and the threshold value for extremely abnormal multichannel data is 100, then the descriptive interpretations could be “Possible new type of sample” if MAH is greater than or equal to 0.4 and less than 1.0, “Unexpected result” if MAH is greater than or equal to 1.0 and less than 100, and “Sample not detected” if MAH is greater than or equal to 100. If MAH is less than 0.4, the predicted result is presumably valid and no descriptive interpretation is required. Help text is also provided at an output device of the user interface 32 in the vicinity of the sensor 2 that gives a recommended course of action as specified, for example, by an administrator of the operator's company. Thus, for example, if a predicted result is accompanied with the message “Possible new type of sample,” the help text can instruct the operator to forward the sample to a laboratory for further characterization. If a predicted result is labeled as “Unexpected result,” the operator can be instructed to verify that a sufficient quantity of sample is available, to verify that the type of material is identified correctly in the input fields of the user interface 32, and to take another measurement of the sample. In this case, if a second measurement of the sample gives an “Unexpected result,” then the operator can be instructed to consider the material to be unacceptable. If MAH is greater than 100, the invalid prediction value can be omitted from the measurement results 14, and the message “Sample not detected” can be accompanied by help text that instructs the operator to contact the provider of the on-site measurement service to help investigate and remedy the problem.

[0169] When outliers are detected during on-site measurements in block 118 and a classification is made of the type of probable outlier in block 122, two events occur. First, the results are reported in block 120 and second, an investigation of the outliers is initiated in block 124. If the outliers are good in block 124, then there is an opportunity to extend the property model to compensate for a wider range of variations in the sample and the measurement conditions. If the outliers are not good in block 124, and if the training set does not need to be corrected or improved in block 128, then there is no opportunity to enhance the model and, since the current property model has not been altered, the current model is considered to be validated in block 130 and on-site measurements continue to be taken in block 116 with no intervention from these two events. If the outliers are good in block 124, then the customer can consult with the provider of the on-site measurement service to decide whether to extend the training set as indicated in block 126. This decision can be based on requirements or preferences of the customer, time and personnel resources of the provider, and economic considerations from both parties.

[0170] The ability of the on-site analysis system to detect probable outliers during on-site measurements at block 118, classify the type of probable outlier at block 112, and identify good outliers at block 124 provides the continual opportunity to adapt the property model to compensate for an effectively comprehensive set of influential factors which may change at some unpredictable time in the future. The outlier detection and classification occur on a real-time basis, so the customer is notified of probable outliers at the earliest opportunity. Furthermore, since the measurement results are stored in the data repository 58 of the central processor 10, the occurrence of probable outliers can trigger an automatic notification to the responsible parties, which can be the provider of the on-site analysis system, one or more administrative personnel at the customer's company, or both. This notification can begin the process of investigating the cause of the outliers and, in cases when one or more detected outliers are good, a decision can be made to extend the training set at block 126, and revise the property model (beginning with block 102 and continuing forward) so it will compensate for an effectively comprehensive set of a wider or different range of influential factors that occur during actual measurements.

[0171] Alternatively, it can be decided that one or more good outliers are not appropriate for inclusion in an extended training set. This decision can result from a situation in which the occurrence of the good outlier will probably be a rare occurrence or a situation in which the good outlier is caused by a type of material or an unusual measurement condition that the customer wants to avoid classifying as a valid measurement. Hence, the “effectiveness” of an effectively comprehensive set of influential factors can be defined or refined for a particular property model. The set of influential factors of a property model is considered to be effectively comprehensive until one or more good outliers is detected during on-site measurements, whereupon either the model is revised to accommodate a wider or different range of influential factors or a decision is made to exclude one or more types of good outliers as valid measurements. The on-site measurement system is adaptable both in its ability to accommodate a potentially changing set of influential factors based on real-time detection of probable outliers and to refine the definition of valid measurements according to established criteria.

[0172] Generation of a global calibration model involves the development of a global training set, which is then validated. The process of validating the calibration model is not only important in establishing the calibration model initially; the validation process attains even greater importance in maintaining the predictability of the calibration model over time to implement enhancement updates.

[0173] In generating global training sets for developing a property model, it is preferred to exercise discretion in selecting the data used in the training set. Data indiscriminately incorporated into the training set may introduce bad outliers, unwanted good outliers, and essentially duplicative information which, while increasing the size of the training set, does not necessarily improve its quality of prediction.

[0174] Desirably, relative to the total number of observed values of a property of interest available for development of a training set, a first subset of values is selected and used to generate instrument responses such that the property of interest is spanned over its expected range and the values typically span the range at approximately regular intervals. After this subset of values is used to generate a preliminary or a revised version of the calibration model, these observed values and the predicted values from the training set are used to compute the RMSECV. A separate subset of observed and predicted values from a validation set is then used to validate this version of the model and compute the RMSEP. The subset used for validation is generally less than the remainder of the original set of observed values less the first subset. If bad outliers are identified in an instrument-response set, those values are discarded. If a number of good outliers are identified, then consistent with the practice of incorporating minimal numbers of additional calibration data to avoid substantially duplicative or incrementally indistinct data, only a portion of the good outliers may be used to develop an extended training set.

[0175] The process of developing a calibration model that will compensate for an effectively comprehensive set of influential factors includes an investigation of pretreatment operations. Pretreatment consists of filtering a data set from a multichannel instrument to one or more subregions within a data set, operating on the data set with one or more mathematical transformations, or both. The combination of filtering and transforming multichannel data is observed to compensate for variations in some types of influential factors. Filtering may be performed before or preferably after the other data transformations. For other types of influential factors, an effective method of pretreatment may not be identified. In many such cases, compensation for those influential factors is possible by extending the training set with observations that span the expected ranges of the corresponding secondary variables. In some other cases, a combination of extended or partially extended training sets and pretreatment is found to be effective, where a partially extended training set includes observations that span some but not all of the expected range of variations of one or more influential factors. In the remaining cases, pretreatment and enhancement training sets will not provide effective compensation, and it will be necessary to control variations in one or more influential factors in order for the property to be measurable to the level of precision specified in the method and objectives (blocks 70 and 82 of FIG. 4) for the model. In those cases where either pretreatment or extension of a training set is possible, consistent with the practice of incorporating minimal numbers of additional calibration data, pretreatment is preferred.

[0176] The procedure for filtering an instrument response by selecting one or more subregions from the entire region of the instrument response as part of a pretreatment method to improve the predictive capabilities of a calibration model is defined as filter refinement. A flowchart of the filter refinement process is shown in FIG. 6. First, a preliminary property model, labeled the ith property model in block 140, is obtained or developed for the property of interest, and RMSECV(i) is calculated. The ith property model in block 140 can be the preliminary model of block 76 in FIG. 4, a revised preliminary model of block 86 in FIG.4, the ith revised model of block 104 in FIG. 5, or the ith validated revised model of block 110 in FIG. 5. Filter refinement may be used to adjust the pretreatment in block 108 of FIG. 5.

[0177] Next, in block 142, statistical criteria for acceptable filters are defined in terms of the RMSECV of a preliminary property model that will satisfy the objectives of blocks 70 and 82 in FIG. 4, specifically the maximum value of RMSEP and the maximum absolute value of the offset of the validation curve of predicted versus observed results. For example, it may be determined that the objectives of blocks 70 and 82 for a property of interest will be satisfied if the RMSEP is not greater than 1.5 times the RMSECV of the preliminary model and the maximum absolute offset between predicted and observed values can be allowed to be as great as 50% of this RMSECV in order to compensate for the influential factors, where the offset can be calculated as the absolute value of the difference between the average predicted value and the average observed value.

[0178] Next, after such statistical criteria are defined, an instrument-response set is defined in block 144 by recording instrument responses acquired as the level of one or more influential factors is varied over a range. As used herein, a global instrument-response set is one for which the range spans the expected range of variations in each of the experimentally determined influential factors, and a partial instrument-response set is one for which the range spans part but not all of the expected range of variations of one or more factors. In the feasibility stage of model development, it is acceptable to use partial instrument-response sets. In the development of a model that is suitable for on-site measurements, it is preferred to use global instrument-response sets.

[0179] Next, in block 146, a procedure involving a search algorithm is used to compute RMSEIR for a number of trial filters containing one or more subregions, where the number of trials j is typically from 50 to 300 for a particular property model, although greater or lesser numbers can be used. Thus, from a number of subregions in a plurality of trial filters comprising discrete combinations of subregions, the multichannel data from applying each trial filter to an instrument response contains at least one subregion of data within the entire available region of data. The instrument-response set for this procedure can be partial or global, although it is preferred that the set be global. The specific subregions evaluated by the search algorithm can be selected by commercial software such as the Bruker OPUS Quant-2 product. These subregions can also be selected manually. The output from the search algorithm can be summarized in a table of trial filters that lists the corresponding RMSEIR and rank of the PLS model for each trial filter, where a trial filter comprises one or more subregions selected from the available multichannel region of a sensor-type. It is convenient to order the trial filters according to RMSEIR(k), where k ranges from 1 to j, as shown in block 148, such that RMSEIR(k) is less than RMSEIR(k+1) for each k from 1 to j−1, but this step is optional.

[0180] Next, in block 150, a decision is made whether one or more trial filters from block 148 satisfies the statistical criteria defined in block 142. If no such trial filter can be identified, then in block 164 the customer can be consulted to determine if less precise predictions will be acceptable. If the customer will change the criteria of block 142 such that at least one trial filter exists that satisfies these criteria and thereby adjusts the objectives defined in blocks 70 or 82 of FIG. 4, or if the outcome of block 150 is yes, then any of these trial filters may be selected as an acceptable filter in block 151. A preferred filter may be selected in block 152 from a group of acceptable filters by the following decision criteria. Criterion A: The acceptable filter that produces the smallest value of RMSEP is most preferred. Criterion B: If an acceptable filter is found that is composed of a smaller number of sub-regions than that identified by Criterion A, then the acceptable filter composed of a smaller number of sub-regions is preferred. If two or more such acceptable filters are identified, Criterion A is applied for those filters. Criterion C: If two or more acceptable filters composed of the same number of subregions are found from applying Criterion B, then acceptable filters corresponding to the smallest PLS rank are preferred. If two or more such acceptable filters are identified, Criterion A is applied for those filters.

[0181] In an alternative embodiment of the procedure for selecting the preferred filter, Criterion C may be used before Criterion B. It is also possible to select a preferred filter by using either Criterion B or Criterion C alone.

[0182] Next, in block 154 the (i+1)th property model is built from the preliminary (i)th property model by using the preferred filter of block 152. RMSEIR(i+1) is computed in block 156. In block 158, the RMSEIR of the (i+1)th property model must also meet the criteria of block 142, specifically the RMSEIR of the instrument-response set for the (i+1)th revised model must meet the same criteria as the RMSEIR of the instrument-response set for the (i)th property model for the preferred filter. If the RMSEIR of the revised model fails these criteria, and therefore is not acceptable in block 158, then a decision is made in block 162 whether some other acceptable filters of block 151 have not yet been considered. If yes, then the next most preferred filter is selected in step 152, and steps 154, 156, and 158 are repeated. If the RMSEIR of a revised property model satisfies these criteria, then RMSEIR(i+1) is acceptable in block 158, filter refinement is complete in block 160, and the identified preferred filter is called the refined filter.

[0183] Returning to block 164, if the customer criteria cannot be altered, then the filter cannot be refined for the instrument-response set of block 144, and it becomes necessary to extend the global training set in block 167 by including calibration data generated over a range of variations in the influential factors that are expected to occur during future measurements using the property model. The (i+1)th property model is built from this training set in block 168, and RMSECV(i+1) is computed in block 170. The decision in block 172 of whether RMSECV(i+1) is acceptable is based on the same criteria as used for block 158. If RMSECV(i+1) is acceptable in block 172, then the calibration model developed from the extended training set is validated in block 174. If RMSECV(i+1) is not acceptable in block 172, then it is necessary to evaluate alternatives in block 176. These alternatives include searching over additional trial filters in block 146 and continuing forward, using the (i+1)th property model of block 168 as the ith property model in block 140 and continuing forward, or selectively omitting calibration data in the training set of block 167 to define a partially extended training set for developing the ith property model in block 140 and continuing forward. This latter case corresponds to controlling one or more influential factors by deciding to redefine the objectives of the on-site analysis (blocks 70 and 82 of FIG. 4) to exclude measurements under conditions where the property is not measurable to the desired precision or to hold those measurement conditions constant. If none of the above alternatives is acceptable, it may be decided to use a different analytical method in blocks 70 and 82 for on-site analysis.

[0184] The calibration model described herein is generally capable of predicting values for the property of interest by compensating for variations in an effectively comprehensive set of measurement conditions and secondary material characteristics. Secondary variables which potentially influence instrument response can each be evaluated to generate one of three outcomes: the secondary variable has no effect or a minimal effect on instrument response; the secondary variable has an influential effect on instrument response, which can be entirely or substantially compensated by pretreatment; or the secondary variable has an influential effect on instrument response which cannot be compensated adequately by pretreatment, but can be compensated by extension of the training set. Note that pretreatment can include filter refinement. There is also a fourth outcome which does not involve prediction by the calibration model. This fourth outcome may result from determining if a property is measurable in the presence of variations of a secondary variable. In this determination, the variation in the property due to variations in the secondary variable over its expected range is compared with the limit of desired precision for predictions of the property of interest. If the range of this variation is a substantial percentage, or greater, of the limit of desired precision of a predicted result, the ability of the calibration model to predict values for the property of interest is hampered, and may prevent the prediction of usable values. The ability for the model to predict values is a function of the rate of change of the property of interest with respect to each secondary variable. For a particular secondary variable, if this rate of change is relatively small, variations in the secondary variable can be compensated by the property model. If this rate of change is relatively large compared with the limit of desired precision, then the secondary variable must be controlled by restricting the possible range of variation during data acquisition and the objectives of blocks 70 and 82 redefined accordingly, or an acceptable property model cannot be generated.

[0185] The generation of a calibration model according to the invention involves the consideration of experimental factors over a wide range in connection with predicting a property of interest, with testing for potential influence on the predicted result and, when effective methods of pretreatment can be identified, no longer requiring that the training set be expanded by including observations taken at different levels of influential factors. The result of this procedure is a calibration model which accounts for an effectively comprehensive range of influential factors. It is possible that a combination of pre-processing, pretreatment and calibration model revision may be employed in connection with a single variable, either primary or secondary.

[0186] An advantage of the calibration model developed as described herein is the ability to compensate for secondary variables previously considered too significant to overcome in a single calibration model across a group of two or more instruments of a particular sensor-type. One such set of secondary variables is the characteristic influential factors attributable to each measuring instrument in a group of similar instruments, where the collection of variations in these variables is described herein as instrument variance. Because of the complexities of manufacture, and the tolerances which necessarily exist in connection with the manufacture of the component parts, measuring instruments constructed from these component parts will not provide identical output in response to the same sample. Further, the response of each such measuring instrument will differ one from the other over time, as age will have a varying effect on the instrument collectively and the component parts individually. For example, component parts within manufacturing tolerances but produced in different batches may demonstrate different output properties over time. Two different instruments, even though manufactured at the same time, may be used to different degrees, effectively wearing out one instrument faster than the other.

[0187] Instead of generating a calibration model on one instrument, then necessarily transferring that model for use by another and developing instrument-specific correction algorithms, the calibration model generated herein compensates for variations of the characteristics of each instrument within a sensor-type and between such instruments by use of a sensor-type-specific property model. Thus, a single property model is generated for all instruments of a particular sensor-type, not multiple property models which must be replicated and corrected or adjusted individually for each instrument within a sensor-type and which must take into account the individual characteristics of each instrument. Significantly, for a collection of instruments of a particular sensor-type, the property model of the invention does not require any individual identification of the specific sensor in use for the purpose of building or using the model. Identification of the on-site location, and thus the instrument, may be important for billing, forecasting or archival purposes, among others, but the model operates without the need for actually identifying the particular instrument and thus the instrument's characteristics. In this sense, the instrument variance is considered in the same way as variation in temperature or sample presentation.

[0188] Instruments of a particular sensor-type must be sufficiently similar. To determine if discrete instruments of some sensor-type are sufficiently similar, the calibration set or a subset of the calibration set that had been used to build a multivariate model for one instrument is used to validate a second instrument over a range of measurement conditions. The set of predictions from the second instrument using this validation set is obtained from the model developed for the first instrument. If the RMSEP of the validation set is within an acceptable tolerance of RMSECV, for example, less than 1.5 times RMSECV of the training set used to develop the calibration model for the first instrument, the second instrument is determined to be sufficiently similar to the first instrument, and both instruments can use the same property model, that model being the model as developed for the first instrument without any modifications. Therefore, a group of instruments of a particular sensor-type are determined to be sufficiently similar by validating each instrument with one or more global property models. The group of such validated instruments produce results from each instrument that are statistically equivalent using a single property model for each property of interest. An acceptable tolerance of RMSECV of the training set for the property model can be specified as the desired precision value in the objectives of block 70 and/or 82 of FIG. 4.

[0189] The filter refinement procedure of FIG. 6 can be used to develop a property model that will compensate for instrument variance. First, in step 140, a global property model is obtained or developed to compensate for all influential factors except for instrument variance, and RMSECV(i) is calculated. This model is called a single-instrument global property model. After defining statistical criteria in block 142 as previously described, an instrument-response set is defined by taking a number of measurements on one or more instruments different from that used to develop the single-instrument property model. Then, additional steps are performed as indicated in blocks 146, 148, 150, 152, 154, 156, 158, as well as those in block 162 and 164 if required, until one of two outcomes occurs. In the first outcome, the filter is refined (160) and the model of step 154 is a multi-instrument global property model, while in the second outcome, it is found that the filter cannot be refined (166). As an alternative to the first outcome, an acceptable multi-instrument global property model can be built by using any acceptable filter. If the filter cannot be refined to at least the level of an acceptable filter, but the extended training set of block 168 leads to a successful validation in step 174, the model of block 168 is an acceptable multi-instrument global property model.

[0190] In reference to the alternatives previously described for block 176, in the case of instrument variance it is possible to selectively omit calibration data in the training set of block 168 acquired from one or more instruments to define a partially extended training set based on two or more instruments, develop the ith property model in block 140 based on this partially extended training set, and continue forward. If this procedure leads to the successful development of a multi-sensor global property model, it is likely that some aspect of the hardware of the omitted analytical instrument is flawed and that instrument should be rejected for use in an on-site sensor. It is also possible to test and accept or reject specific components of an instrument using this procedure.

[0191] Predicted values from a multi-instrument global property model can be used in a quality control procedure to accept or reject analytical instruments or components for use in the sensor devices of an on-site analytical system. In such a quality control procedure, predicted values are generated from one or more untested instruments or components using a previously established multi-instrument global property model. The quality-control data set is the instrument-response set formed from these new predictions. If the RMSEP of this instrument-response set satisfies the criteria of block 158 in FIG. 6, the new instrument or component is acceptable. If not, the training set can be extended in block 167 and a new multi-instrument global property model can be built in block 168. If the resulting RMSECV computed in block 170 is acceptable according to the criteria of block 172, then the property model developed from this extended training set (block 168) is adopted as the multi-instrument global property model. If the RMSECV is not acceptable according to the criteria of block 172, then the instrument is rejected and the new property model built in block 168 is not adopted. Generally, this quality control procedure is performed for all new or untested instruments that will be installed for on-site measurements. In addition, this quality control procedure can also be performed for new or untested components used in generating measurements, such as but not limited to probes, interferometers, and detectors.

[0192] The multi-instrument global property model is able to predict values of properties using single algorithms that do not contain any instrument-specific parameters. The multichannel data acquired from different instruments produce statistically equivalent results without using instrument-specific correction factors. Generally, only one instrument-specific computational data transformation is undertaken during the computation of measurement results 14 from measurement data 12, specifically during pre-processing in the local processor 34 of an individual sensor 2, at which time eccentricities in the sample spectrum attributable to the background spectrum unique to the sensor 2 are removed. No instrument-specific information for use by the modeling algorithms is transmitted to or stored at the central processor 10.

[0193] Though individual instrument characteristics can be compensated in practicing the invention, there are limits to the extent of compensation which can be performed. For example, it is not presently possible to generate a calibration model which can accept instrument responses from different types of spectrometers, such as NIR and Raman, and compensate for the different characteristics of each. The instruments should preferably be of the same sensor-type, usually from the same manufacturer, and be the same model. The instruments within a sensor-type should be sufficiently similar to generate statistically equivalent results, as described earlier. Most preferably, the instrument should exhibit narrow manufacturing tolerances as to those components which affect the instrument's data acquisition performance and its performance over time. Thus, in the case of a NIR spectrometer, it is important for the instruments to exhibit good interferometer alignment. Over time, it is important for the instruments to exhibit good light source reproducibility and data acquisition probe reproducibility. In the case of instruments utilizing other portions of the electromagnetic spectrum or instruments which generate a response non-spectroscopically, reproducibility of the component or components which interact with the sample, and the component or components which register the response, is desired.

[0194] As noted above, the invention also encompasses a method of generating measurement results for a customer and supplying information of value to the customer based on these results. The method incorporates a hardware infrastructure, software and data processing to create a material analysis service which encompasses the collection, transmission and manipulation of data, with delivery of information of value to the customer, to the original submitter of data or to an alternate location. The data and information are transmitted along a communication link.

EXAMPLES

[0195] The following detailed examples describe various aspects of the invention in greater detail. The examples are intended to enable one skilled in the art to practice the invention, not to limit the scope thereof. Numerous variations are possible without deviating from the spirit and scope of the invention.

Example 1

[0196] A feasibility study was done to determine if a property model could be developed to measure the concentration of squalane in squalene. In accordance with block 70 of FIG. 4, the method was defined as FT-NIR using the MATRIX Model F instrument manufactured by Bruker Optics, with sample presentation for liquid samples provided by closure caps with dimensions of 18-mm diameter×10-mm high manufactured by Cincinnati Container Corporation. The objectives for the property model included measurements of squalane in squalene having concentrations ranging from trace amounts to about 10 weight percent with a limit of desired precision of 0.10% or smaller as measured by RMSEP. The objectives further indicated that the measurements will be taken by non-skilled operators who will dispense about 1 mL of liquid samples into separate, disposable caps, and the sample temperature may vary from about 0° C. to 60° C. In accordance with the block 72 of FIG. 4, the expected range of the concentration of squalane in squalene was defined as 0 to 10%, where the % symbol indicates a percentage calculated from the weight of solute and the weight of the solution.

[0197] A set of six samples with 0.00%, 2.00%, 4.00%, 6.00%, 8.00% and 10.00% of squalane (99%, Aldrich Chemical Company) in squalene (97%, Aldrich Chemical Company) were prepared to serve as calibration samples to generate a training set, shown as block 74 (FIG. 4), to assess model feasibility. The known values were determined by calculation of the concentrations expressed as weight percentages using the weights measured by an analytical balance. 1.0 mL of each sample was transferred to a separate, disposable cap. For each sample, one FT-NIR spectrum was acquired in the transflectance mode at ambient temperature (20° C.).

[0198] The type of FT-NIR spectrometer used for all of the examples was MATRIX Model F instrument manufactured by Bruker Optics equipped with an InGaAs detector and a fiber optic probe approximately 91 cm long. The fiber optic probe was bundled with 200 optical fibers, 100 for illumination and 100 for collection. Each fiber was 100 μm in diameter, and the total illuminating area was about 3-mm in diameter. The spectral resolution of the spectrometer was 8 cm−1 and the available [Question about use of “available” region. See Alan's notes] spectral region of the spectrometer was from 4500 cm−1 to 10,000 cm−1. The spectra were acquired within a relatively brief period of time. Individual averaged spectra for a single sample were generated each within typically less than about one minute.

[0199] A button on the fiber optic probe, serving as an input device of the user interface, was depressed to initiate data collection of 20 spectra at a scanning speed of 20 kHz and about 2 scans per second. The spectra were averaged in interferogram mode, converted to single-scan mode by fast Fourier Transform, and then converted to an absorbance spectrum for spectroscopic analysis according to

absorbance spectrum = - \log (single - channel spectrum / background spectrum)

[0200] wherein the reference or background spectrum for transflectance measurement was the average of 20 scans measured by direct contact of the fiber optic probe on a mirror surface. The six observed NIR spectra are shown in FIG. 7.

[0201] As indicated in block 76 of FIG. 4, these 6 spectra were the pre-processed instrument responses in the training set used to build an initial calibration model, herein called Model 1.0, according to the PLS method using Bruker OPUS Quant-2 software. In Examples 1 through 15, the calibration models were generated using a computer operating independently of the central processor, though this was not required. Model 1.0 was built using the entire available [Again question on use of “available”] spectral region from 4500 to 10,000 cm−1 and with no data pretreatment. The feasibility of predictive measurements was assessed by cross-validation of the training set, which produced values of 86.95 for R2 and 1.23% for RMSECV with a rank of 3. No outliers were detected in the training set. The observed and predicted values from the training set for Model 1.0 are given in Table 1, and the corresponding calibration curve is shown in FIG. 8. These results, specifically since the coefficient of determination was greater than 60, demonstrated that the property was measurable (block 78) and it was feasible to develop a model (block 100, FIG. 5) to predict the concentration of squalane in squalene.

2TABLE 1ResidualSample No.Observed (%)Predicted (%)(Obs − Pred)10.002.71−2.7122.001.920.0834.003.500.5046.005.330.6758.008.54−0.54610.0010.89−0.89

[0202] To increase the sensitivity of the method for the identification of influential factors (block 84) in the following examples, it was preferred to use a refined filter (block 160, FIG. 6) as the pretreatment of block 86. Filter refinement, using the OPUS Quant-2 software to select trial filters, yielded Model 1.1, which was a property model with a refined filter of 4597.5 to 5025.6 cm−1. Model 1.1 gave 99.1 for R2 and 0.325% for RMSECV with an optimal rank of 3. The resulting calibration curve and the table of observed and predicted values from the training set for Model 1.1 are shown in FIG. 9 and Table 2. No outliers were detected in the training set.

3TABLE 2ResidualSample No.Observed (%)Predicted (%)(Obs − Pred)10.000.49−0.4922.002.19−0.1934.003.940.0646.005.800.2058.008.39−0.39610.0010.40−0.40

Example 2

[0203] The light intensity of the environment is a potentially influential factor (block 84). The lights are expected to be either on or off. To determine if a difference in the intensity of background light in the room will affect the concentrations predicted by Model 1.1 (block 88), the spectrum of a single sample with an observed value of 2.00% of squalane in squalene was acquired four times with the overhead fluorescent room lights on and four times with the lights off without changing any other measurement conditions. FIG. 10 shows the superposition of the resulting eight spectra. Since no measurable differences are observed between the spectra, variations in the light intensity of this particular environment will not affect the predicted concentrations.

[0204] To further demonstrate that the light intensity is not an influential factor, Model 1.1 was used to compute predicted values of the concentrations corresponding to each of the eight spectra for the 2.00% squalane samples. As shown in Table 3, the residuals between the observed and predicted values from the validation set each expressed in percent are each less than the RMSECV of Model 1.1. Specifically, since each residual value in percent is much less than the RMSECV of 0.325%, and since each residual value is less than the desired precision value of 0.10%, the results were statistically equivalent. Thus, variation in the light intensity of the environment was not an influential factor (block 88) in the prediction of concentrations of squalane in squalene by FT-NIR measurements to a precision within the limit of desired precision, and no revision to the preliminary model (block 90) was required.

4TABLE 3ResidualSpectrum No.LightObserved (%)Predicted (%)(Obs − Pred)1Off2.002.010.012Off2.001.99−0.013Off2.002.020.024Off2.002.010.015On2.002.010.016On2.002.010.017On2.002.010.018On2.002.010.01

Example 3

[0205] The orientation of the sample cap is a potentially influential factor (block 84). The orientations are expected to be random. To determine if variation in the orientation of the sample cap (block 86) will affect the predicted concentrations (block 88), a sample of 1.00% of squalane in squalene was prepared and measured with four different cap orientations. The initial orientation of the cap was selected at random, and additional orientations were attained by successively rotating the cap by approximately 90 degrees about an axis perpendicular to the bottom of the cap between measurements. As shown in FIG. 11, measurable differences were observed in these spectra, indicating that orientational variance is probably an influential factor.

[0206] The six calibration samples in the training set of Example 1, each with a sample cap orientation labeled as orientation 1, were then measured with three additional orientations selected at random, labeled as orientations 2, 3, and 4, each rotationally differing by about 90°. It should be understood that a particular numbered orientation, such as orientation 3, indicates only the order in which a particular random orientation was generated in a sequence of random orientations for a sample measurement, so orientation 3, for example, indicates only that this was the third random orientation measured. The resulting additional 18 spectra were used as a validation set to predict squalane concentrations according to Model 1.1 with the refined filter of Model 1.1 for pretreatment (block 90). RMSEP was 0.234%, R2 was 99.53, and the validation curve is shown in FIG. 12. The observed and predicted values from the validation set are listed in Table 4.

[0207] Since the RMSEP of the preliminary model in the presence of variations in sample cap orientation exceeded the limit of desired precision, the residuals were statistically significant, and orientational variance was determined to be an influential factor (block 88). It was therefore necessary to build a revised model (block 90) to compensate for orientational variance.

5TABLE 4SampleResidualNo.OrientationObserved (%)Predicted (%)(Obs − Pred)120.00−0.190.19130.00−0.360.36140.00−0.380.38222.001.870.13232.001.800.20242.001.770.23324.004.02−0.02334.003.700.30344.003.900.10426.005.760.24436.005.670.33446.005.660.34528.007.650.35538.007.840.16548.007.810.196210.0010.08−0.086310.0010.06−0.066410.0010.02−0.02

[0208] In order to revise the preliminary model so it would compensate for orientational variance, the twenty-four spectra obtained from measurements of the four different orientations at each concentration, which were the spectra used to generate the predicted values listed in Tables 1 and 4, were then used as the training set (block 102, FIG. 5) to build Model 3.0. Assuming there was an approximately quadratic relationship in the observed versus predicted values in FIGS. 9 and 12, four different levels were used for the orientations in the training set. Using the same refined filter identified for Model 1.1, namely 4597.5 to 5025.6 cm−1, Model 3.0 was a property model that yielded values of 99.92 for R2 and 0.098% for RMSECV with a rank of 5 (block 104). No outliers were detected (block 106). It is noted that to compensate for orientational variance, Model 3.0 required two additional PLS factors compared with Model 1.1. The calibration curve for Model 3.0 is shown in FIG. 13.

[0209] Model 3.0 was validated using the same calibration set but with validation measurements taken at new random orientations, labeled as orientation 5 for each sample (block 108). The validation gave RMSEP of 0.046% (block 110) which showed a significant improvement in the predicted results compared with the RMSEP of 0.234% obtained using Model 1.1. No outliers were detected (block 112). The validation curve is shown in FIG. 14. The observed and predicted values from the validation set are in Table 5. Since RMSEP was less than the desired precision value of 0.10%, the residuals were statistically insignificant and the revised preliminary model demonstrated that the property was still measurable (block 92) to the limit of desired precision in the presence of variations in orientation.

6TABLE 5SampleResidualNo.OrientationObserved (%)Predicted (%)(Obs − Pred)150.00−0.070.07252.002.05−0.05354.003.980.02456.005.940.06558.008.02−0.026510.009.980.02

Example 4

[0210] The sample pathlength for the squalane-squalene mixture, which is twice the distance from the air-liquid interface at the top of the sample volume to the reflective surface of the sample cap at the bottom of the sample, is a potentially influential factor (block 84) since the intensity of the NIR absorbance by the sample is proportional to the sample pathlength. The pathlength for a particular measurement is determined by the dimensions of the cap and the volume of sample dispensed into the cap.

[0211] Additional aspects of the method and objectives of block 70 are now defined. Suppose that the disposable caps of Examples 1 to 3 are to be used as economical sample holders for remote measurements. Since these caps are not manufactured identically, variations in the dimensions of the caps, and hence of the sample pathlength for a uniform volume of material, are inevitable. Further suppose that disposable pipettes with 0.25 mL graduations will be used as economical sample dispensers for remote measurements. Variations in sample volume are expected to occur under actual measurement conditions due to variations in operator technique in measuring and dispensing sample volumes. Therefore, pathlength variance is expected to occur during future measurements due to variations in at least two experimental factors involved in sample presentation consisting of dispensing material into different disposable caps using a disposable pipette with 0.25 mL graduations.

[0212] To determine whether dispensing material into different caps can affect the predicted concentrations (block 88), 1.0 mL of 1% squalane in squalene was dispensed into two different caps using a pipette with 0.25 mL graduations. The FT-NIR spectra obtained on these two subsamples, each in a different random orientation, showed observable differences in intensities at various wave numbers as shown in FIG. 15, so pathlength is probably an influential factor.

[0213] Mathematical transformations were next considered for the definition of pretreatment as indicated in block 102. Normalization techniques, such as min-max normalization, vector normalization (VN) and multiplicative scattering correction (MSC), can be used to compensate for at least some of the variation in signal intensity. For example, vector normalization transformed the spectra of FIG. 15 into the spectra of FIG. 16.

[0214] To compensate for differences in spectral intensities which remain after pretreatment, additional calibration data can be included in the training set 102 which intentionally produce a range of spectral intensities if further improvement in the precision of the model is desired. Variations in pathlength can be intentionally introduced by using a selection of different sample volumes to span the range of pathlengths that are expected to occur during future measurements.

[0215] To compensate for shorter pathlengths, for example, a sample volume of 0.5 mL was tested. Model 3.0 predicted that a validation set generated from the instrument responses from 0.5 mL of a 1.00% squalane validation sample in two random orientations contained 2.93% and 2.94% squalane. These predictions were poor because Model 3.0 did not include training measurements or data pretreatment that would compensate for variations in pathlength. Since the residuals of the validation results, 1.93% and 1.94%, were greater than the desired limit of precision, the residuals were statistically significant, pathlength variance was determined to be an influential factor, and it was necessary to build a revised model (block 86).

[0216] Model 4.0 was built (block 104) as a property model by pretreatment of the calibration spectra with vector normalization and a refined filter (block 160, FIG. 6) of 5195.3 cm−1 to 6398.7 cm−1. Model 4.0 produced values of 99.99 for R2 and 0.0313% for RMSECV with a rank of 6. The calibration curve from the training set of Model 4.0 is shown in FIG. 17. No outliers were detected (block 106).

[0217] The predicted values for the 0.5 mL 1.00% squalane validation sample in two random orientations (block 108) using Model 4.0 became 1.35% and 1.44%, compared with the corresponding values of 2.93% and 2.94% that had been predicted from Model 3.0 with no adjustments in pretreatment. Since the residuals of the validation measurements using Model 4.0, 0.35% and 0.44%, were greater than the desired precision value, the defined pretreatment was not adequate to compensate for pathlength variance and it was necessary to extend the training set.

[0218] To make further improvements in the accuracy and precision of predicted results for 0.5 mL sample volumes, three additional calibration samples were prepared using low, middle, and high levels of concentrations to span the range of concentrations in the training set. Assuming that the calibration curve for FIG. 17 was linear, three levels were sufficient to span the expected range. Specifically, 0.00%, 6.00% and 10.00% samples were prepared with 0.5 mL sample volumes to build an extended training set (block 102) for Model 4.1. The data pretreatment was the same as that used for Model 4.0, but the training set was extended to include 12 additional spectra of the samples with 0.00%, 6.00% and 10.00% concentrations, taking spectra for each sample in four different random orientations. Model 4.1 (block 104) was a property model that produced values of 99.98 for R2 and 0.052% for RMSECV with a rank of 7. No outliers were detected (block 106). The calibration curve of Model 4.1 is shown in FIG. 18.

[0219] The predicted values of concentration for the 1.00% validation sample then became 1.01% and 1.04% for two different random orientations (block 108) using Model 4.1. Since point estimates of RMSEP, taken as the residuals 0.01% and 0.04%, were each less than the desired precision value, the residuals were statistically insignificant and the revised preliminary model demonstrated that the property was still measurable (block 92) in the presence of pathlength and orientational variations. This process can be continued to extend the training set to span the particular range of sample volumes and, hence, pathlengths that are anticipated to occur during actual measurements in the future.

Example 5

[0220] Sample temperature is a potentially influential factor (block 84). A validation sample was prepared with 1.00% of squalane in squalene. FT-NIR spectra were obtained on two subsamples each at sample temperatures of 0° C., 20° C. and 60° C. Predicted values of the squalane concentration in the two validation samples were computed for each spectrum using Model 4.1 and listed in Table 6. Since the residuals at 60° C. are greater than the limit of desired precision (block 88), temperature was determined to be an influential factor (block 88), and it was necessary to revise the preliminary model (block 90).

7TABLE 61.00%squalaneSubsample20° C.0° C.60° C.NumberPredicted (%)ResidualPredicted (%)ResidualPredicted (%)Residual11.03−0.030.940.061.28−0.2821.04−0.040.960.041.14−0.14

[0221] To compensate for variations in sample temperature, the training set for Model 4.1 was extended (block 102). The additional calibration spectra were generated by measuring three 1.0 mL calibration samples, with 0.00%, 6.00% and 10.00% squalane concentrations in squalene, each at a low temperature (0° C.) and a high temperature (60° C.), generating spectra using four random cap orientations at each temperature and concentration. Model 5.0 was a property model built from this extended training set (block 104), which was constructed to predict squalane concentration in the range from 0 to 10% with compensation for variations in temperature, cap orientation and pathlength. The refined filter (block 160, FIG. 6) for Model 5.0 was identified as the subregion from 5449.9 cm−1 to 7501.8 cm−1 using vector normalization as the pretreatment transformation (block 102). Model 5.0 produced values of 99.99 for R2 and 0.042% for RMSECV (block 104) with a rank of 9. The calibration curve from the training set of Model 5.0 is shown in FIG. 19. No outliers were detected in the training set (block 106). The observed and predicted values are given in Table 7.

8TABLE 7SampleTemper-ObservedPredictedResidualNo.Volumeature(%)(%)(Obs-Pred)11.0 mLAmbient0.000.01−0.0121.0 mLAmbient0.000.01−0.0131.0 mLAmbient0.00−0.030.0341.0 mlAmbient0.000.000.0051.0 mlAmbient2.001.980.0261.0 mlAmbient2.002.01−0.0171.0 mlAmbient2.001.990.0181.0 mLAmbient2.001.960.0491.0 mLAmbient4.004.08−0.08101.0 mLAmbient4.004.02−0.02111.0 mLAmbient4.004.01−0.01121.0 mLAmbient4.004.04−0.04131.0 mLAmbient6.005.940.06141.0 mLAmbient6.005.930.07151.0 mLAmbient6.005.980.02161.0 mLAmbient6.005.940.06171.0 mLAmbient8.007.900.10181.0 mLAmbient8.008.03−0.03191.0 mLAmbient8.007.990.01201.0 mLAmbient8.008.05−0.05211.0 mLAmbient10.0010.01−0.01221.0 mLAmbient10.0010.02−0.02231.0 mLAmbient10.0010.01−0.01241.0 mLAmbient10.0010.06−0.06250.5 mLAmbient0.000.000.00260.5 mLAmbient0.00−0.010.01270.5 mLAmbient0.000.01−0.01280.5 mLAmbient0.000.02−0.02290.5 mLAmbient6.006.04−0.04300.5 mLAmbient6.006.05−0.05310.5 mLAmbient6.005.970.03320.5 mLAmbient6.005.910.09330.5 mLAmbient10.0010.05−0.05340.5 mLAmbient10.009.980.02350.5 mLAmbient10.0010.01−0.01360.5 mLAmbient10.009.980.02371.0 mLLow0.00−0.020.02381.0 mLLow0.00−0.030.03391.0 mLLow0.00−0.010.01401.0 mLLow0.000.03−0.03411.0 mLLow6.006.10−0.10421.0 mLLow6.006.01−0.01431.0 mLLow6.005.990.01441.0 mLLow6.006.03−0.03451.0 mLLow10.009.990.01461.0 mLLow10.009.930.07471.0 mLLow10.0010.000.00481.0 mLLow10.009.980.02491.0 mLHigh0.000.12−0.12501.0 mLHigh0.00−0.050.05511.0 mLHigh0.000.000.00521.0 mLHigh0.000.02−0.02531.0 mLHigh6.005.980.02541.0 mLHigh6.006.03−0.03551.0 mLHigh6.006.000.00561.0 mLHigh6.006.04−0.04571.0 mLHigh10.009.960.04581.0 mLHigh10.0010.04−0.04591.0 mLHigh10.009.960.04601.0 mLHigh10.009.960.04

[0222] As shown in Table 8A, the results predicted from Model 5.0 for the two original validation samples showed no significant differences from the known value at each measured temperature and a considerable improvement in predictability compared with Table 6.

9TABLE 8A1.00%squalane20° C.0° C.60° C.SubsamplePredictedResidualPredictedResidualPredictedResidualNumber(%)(Obs-Pred)(%)(Obs-Pred)(%)(Obs-Pred)10.990.010.990.011.02−0.0221.01−0.011.02−0.020.980.02

[0223] A 2.00% squalane in squalene sample was then measured at two other temperatures within the anticipated 0°-60° C. range to create a small validation set (block 108). Spectra were acquired for this sample at 5° C. and 40° C., and concentrations were predicted based on Model 4.1 without temperature compensation and Model 5.0 with temperature compensation. The predicted values from the validation set are shown below in Table 8B. Since the residuals of Model 5.0 were each less than the desired limit of precision, the revised model was able to compensate for variations in sample temperature, pathlength, and orientation measured within the expected range, and the property was still measurable (block 92).

10TABLE 8BModel 5.02.00% SqualaneModel 4.1ResidualSubsample NumberPredicted (%)Residual (Obs-Pred)Predicted (%)(Obs − Pred)1 (5° C.)1.870.132.02−0.022 (5° C.)1.920.082.04−0.041 (40° C.) 2.13−0.132.03−0.042 (40° C.) 2.12−0.122.03−0.03

Example 6

[0224] Humidity of the atmosphere is a potentially influential factor (block 84). Although a NIR instrument may be tightly sealed, moisture may still get into the interior of the instrument over an extended time. Furthermore, part of the light path between the probe and the sample may be open to the environment. Humidity in the air either inside or outside the instrument may affect the obtained NIR spectrum of a sample.

[0225] In general, there are two approaches to overcoming potential variations in environmental humidity. A traditional approach, one that would typically be practiced in a laboratory by trained scientists, would be to measure a background spectrum under the actual environmental conditions immediately before each sample measurement. Then, pre-processing the acquired spectrum of the sample with the background spectrum would eliminate environmental factors such as moisture interference automatically. However, this approach is not convenient or reliable for on-site measurements by non-skilled operators. This traditional approach may also be inadequate in compensating for unexpected short term changes in ambient humidity that could occur, for example, if an operator were to breathe moist air into the environment near the light path to the detector of the instrument.

[0226] The second approach is to include a small number of spectra in the training set that are generated with a range of humidities that would be expected to occur under actual conditions during remote testing. Since the NIR spectral features of water are much sharper than most other types of NIR features from condensed phase samples, variations in humidity can be readily discriminated and compensated by the PLS calibration model. Therefore, the potential interference from variations in humidity can be avoided by extending the training set to include some spectra that span a range of humidities.

[0227]
FIG. 20 shows two background spectra, the upper spectrum being taken under conditions of relatively low humidity and the lower spectrum under relatively high humidity. For the purpose of practicing the present invention, it is not necessary to know or quantitate the magnitudes of these humidities, but only to ensure that the range of humidities included in the training set spans the range that is expected to be encountered in the environment under future measurement conditions. Since measurable differences were observed in these spectra, humidity was probably an influential factor.

[0228] To generate spectra of samples for the training set at various levels of humidity, an initial moisture spectrum was required, typically at a relatively low humidity value. First, a background measurement was taken under very dry conditions after desiccant had remained in the tightly sealed instrument for a period of time. Next, the desiccant inside the instrument was removed to allow the internal humidity to increase to a stable value, and an absorbance spectrum at a higher humidity was measured. As shown in FIG. 21, the acquired moisture spectrum (bottom spectrum) was then used to generate two higher humidity spectra by multiplying the acquired spectrum by factors of 2 and 3.

[0229] The spectra used to expand the training set (block 102) were generated mathematically by adding these three moisture spectra to low humidity spectra of samples (taken with the desiccant inside the spectrometer) at 0.00%, 6.00% and 10.00% concentrations in the training set. Model 6.0 was a property model built from this expanded training set using the pretreatment of Model 5.0 (block 104). No outliers were detected (block 106).

[0230]
FIG. 22 shows the superposition of the four spectra of a validation sample prepared with 1.00% squalane before and after the mathematical addition of the moisture spectra at multiplicative scaling factors of 1, 2 and 3. When the spectrum with the highest moisture content of FIG. 22 (block 108) was used as a validation set to predict squalane concentration using Model 5.0, the predicted value was 0.67%. Since the absolute value of the residual, 0.33%, was greater than the desired precision value, this residual was statistically significant, and humidity was determined to be an influential factor. It was therefore necessary to revise Model 5.0 (block 90) to compensate for humidity variance. The predicted value from the validation spectrum using Model 6.0 was 1.02%. Since the residual of 0.02% was less than the limit of desired precision, the property was still measurable (block 92) in the presence of variations in humidity.

Example 7

[0231] The intensity of the excitation light source of the spectrometer is a potentially influential factor (block 84).

[0232] It was found possible to extend a calibration model to compensate for possible variations that can arise over time as a spectroscopic sensor unit ages or, equivalently for the purpose of developing calibration models, for differences in the performance between different spectroscopic sensor units at an arbitrary time. Specifically, it was found that variations in the performance of a small number of components in FT-NIR spectrometers account for most of the variations in the spectra that occur over time or that exist between different spectrometers. Some of these components are the excitation source and the mechanical alignment of the internal optics. Degradation of intensity of the light source or replacement of a light source after failure as well as a shift in the alignment of optical components may cause changes in the instrument responses and, therefore, of the predicted values. Traditionally, correction of such instrument variation would be achieved by re-calibration of each instrument using a remediation update or by adjusting the instrument hardware. The present invention uses a new approach to eliminate the need for frequent remediation updates or to reduce significantly the frequency of re-calibrations and to avoid the need for individual or custom adjustments of instrument-specific calibration transfers on particular equipment.

[0233]
FIG. 23 shows the background spectra and the absorbance spectra of a 1.00% squalane validation sample measured by the same FT-NIR instrument but using three different light sources covering a range of performance from a strong, new source to weaker, older sources. The predicted values from these three validation spectra of squalane using Model 6.0 were 1.10%, 1.23% and 1.30%, and RMSEP was 0.226%. Since RMSEP was greater than the limit of desired precision, it was necessary to revise the model to compensate for variance of the excitation source.

[0234] The uppermost spectrum in FIG. 23 was obtained using the same light source as that used in Examples 1 through 6, and the two lower spectra were obtained using weaker light sources. Since there were offset, ramp and non-linear relationships between these spectra, pretreatment transformations (block 102) could be used to reduce significantly the corresponding spectral differences. In the present example, a first derivative transformation effectively eliminated the effects from offset and ramp, and vector normalization or multiplicative scattering correction suppressed the intensity variances due to the non-linear effects from different light intensities. For example, first derivative and vector normalization pretreatment of the three squalane FT-NIR spectra of FIG. 23 effectively reduced the differences between spectra as shown in FIG. 24.

[0235] Calibration Model 7.0 was a property model built with a refined filter (block 160, FIG. 6) from 5199.2 cm−1 to 8797.7 cm−1 using pretreatment transformations (block 102) consisting of a first derivative transformation with 17 smoothing points followed by vector normalization. Cross-validation of this calibration model gave values of 99.99 for R2 and 0.042% for RMSECV (block 104) with a rank of 8. The calibration curve from the training set of Model 7.0 is shown in FIG. 25. No outliers were detected (block 106).

[0236] The predicted values of concentration from the three validation spectra in FIG. 23 using Model 7.0 were 1.02%, 1.02% and 1.03%. The residuals of the predicted values from these spectra, 0.02%, 0.02% and 0.03%, were each less than the limit of desired precision of 0.10%, so the property was still measurable (block 92) in the presence of variation in the excitation source. Furthermore, this example demonstrates that it was possible to compensate for variations in the intensity of the excitation light source by data pretreatment.

[0237] The training set used to develop Model 7.0 is now considered to be a global training set for a single instrument, and Model 7.0 is a single-instrument global property model.

Example 8

[0238] Replacement of a fiber optic probe is a potentially influential factor (block 84).

[0239] It has been found possible to extend a calibration model to compensate for effects from changing certain hardware components as could occur during instrument maintenance. The most likely hardware components that could be replaced in a FT-NIR system over time include the desiccant, the excitation light source, the laser source, and the fiber optic probe. The method for compensating for the variances from aging desiccant and from decay of the excitation light source was described in Examples 6 and 7. The laser is used to track the wavelength accuracy and will be re-calibrated after replacement of a laser source, so the spectra will not be affected significantly if the laser source is replaced. The current example demonstrates how to compensate for future replacement of a fiber optic probe, which may be needed if it becomes accidentally damaged.

[0240]
FIG. 26 shows the spectra of a 2.00% squalane validation sample measured by the FT-NIR instrument described in Example 1 using three different fiber optic probes selected to cover a range of fiber optic performances (block 86). The lower spectrum was obtained using the same fiber optic probe as used in Examples 1 through 7, while the other two spectra were obtained using two different fiber optic probes. Since measurable differences were observed among these spectra, probe variance is probably an influential factor. The nature of these spectral differences were similar to those in FIG. 23, so data pretreatment as done to correct for light source decay in Example 7, namely first derivative followed by either vector normalization or multiplicative scattering correction (block 102), would also compensate for transmission differences from different fiber optic probes. The effectiveness of first derivative and vector normalization pretreatment on the spectra of FIG. 26 is shown in FIG. 27.

[0241] The predicted values of the three validation spectra shown in FIG. 26 using Model 6.0, which was a property model that did not include a first derivative transformation, were 2.02%, 4.22% and 4.39%. The residuals, i.e., 2.22% and 2.39%, from the two additional probes were each greater than the limit of desired precision, and probe variance was determined to be an influential factor (block 88). The corresponding values predicted from Model 7.0, which was a property model that included the first derivative and vector normalization data pretreatment (block 90), were 1.99%, 2.03% and 2.02%. Since the absolute values of the residuals, 0.01%, 0.03% and 0.02%, were each less than the desired precision value, the property was still measurable (block 92). Furthermore, the identified pretreatment effectively eliminated the impact on the predicted results from changing fiber optic probes. Thus, Model 7.0 was confirmed to be a single-instrument global property model and validated for a wider range of instrument variance.

Example 9

[0242] The use of different analytical instruments is a potentially influential factor (block 84).

[0243] It has been found possible to share a calibration model among two or more NIR instruments without having to develop individual calibrations for each instrument or having to use instrument standardization or calibration transfer methods. To build models that would compensate for variance between instruments, it was necessary to use a set of instruments that are sufficiently similar.

[0244] A real spectrum can be considered as the end result obtained from the combination of a hypothetical equipment-independent spectrum with spectral features that arise from equipment-dependent optical parts and alignment, which include the light source, interferometer, mirrors, lens, windows, fiber optics and detectors. Since it is impossible for a manufacturer to produce identical instruments, differences will arise, for example, in the light source intensity, the quality of the optical parts, the alignment of the optical paths, and the response of the detectors.

[0245] One method of extending a single-instrument global property model to multiple instruments is to extend the training set with spectra acquired from several calibration samples that span the expected range of the property of interest over a range of measurement conditions as measured by other instrument systems. This method directed to two instruments is illustrated below.

[0246] For this example, the particular instrument that was used to take measurements in Examples 1 through 8 was labeled as Instrument A, and a second instrument as Instrument B (block 86). When Instrument B was used to measure the samples from the training set of Example 7 taking measurements at four different, random orientations, and when the concentrations were predicted using the single-instrument global property Model 7.0, the RMSEP of the validation set was 2.19% and the predicted values exhibited a systematic offset of about 2.2% as shown in FIG. 28. The observed and predicted values from the validation set are shown in Table 9. Since the RMSEP was greater than the limit of desired precision, it would be necessary to revise Model 7.0 to compensate for variance between instruments (blocks 88 and 90).

[0247] For illustrative purposes, the procedure of FIG. 6 was used to build a revised property model from an extended training set (block 167) without using a search algorithm (block 146) to select a refined filter (block 160). The property model and RMSECV of block 140 were obtained from Model 7.0. It was determined that the criteria of blocks 70 and 82 of FIG. 4, and specifically that the desired precision for RMSEP of 0.10%, would be satisfied if the RMSEP of the revised model was not greater than 2 times the RMSECV of Model 7.0, which was about 0.042%, and if the maximum absolute offset was less than 50% of this RMSECV, or less than about 0.021%. These defined the statistical criteria of block 142, which would be used in block 172.

11TABLE 9ResidualSample No.Observed (%)InstrumentOrientationPredicted (%)(Obs-Pred)10B1−2.212.2110B2−2.242.2410B3−2.252.2510B4−2.202.2022B1−0.202.2022B2−0.182.1822B3−0.162.1622B4−0.192.1934B11.832.1734B21.822.1834B31.812.1934B41.792.2146B13.792.2146B23.852.1546B33.862.1446B43.792.2158B15.822.1858B25.832.1758B35.812.1958B45.792.21610B17.872.13610B27.892.11610B37.822.18610B47.792.21

[0248] Calibration samples with concentrations of 0.00%, 6.00% and 10.00% were then measured by Instrument B and the spectra were appended to the training set that had been used to build Model 7.0 (block 102). Model 9.0 was a property model built from this extended training set (block 104). R2 became 99.99 and RMSECV became 0.039% with a rank of 9. No outliers were detected (block 106). Since the RMSECV was acceptable in block 172 of FIG. 6, the extension was validated in block 174 and instruments A and B were sufficiently similar using the extended training set. The multi-instrument calibration curve for the extended training set of Model 9.0 is shown in FIG. 29.

[0249] Validation of Model 9.0 using the remaining samples, 1.00%, 2.00%, 4.00% and 8.00%, taking measurements with four different, random orientations using Instrument B to generate the validation set (block 108), gave an RMSEP of 0.040% with a prediction offset of essentially zero as shown in FIG. 30 (block 110). The observed and predicted values from the validation set are given in Table 10. No outliers were detected (block 112). Model 9.0 is a multi-instrument global property model that has been validated for Instruments A and B.

12TABLE 10ResidualObserved (%)InstrumentOrientationPredicted (%)(Obs − Pred)1.00B11.01−0.011.00B21.000.001.00B31.04−0.041.00B40.960.042.00B11.980.022.00B22.03−0.032.00B32.05−0.052.00B42.02−0.024.00B14.07−0.074.00B24.04−0.044.00B34.04−0.044.00B44.02−0.028.00B18.07−0.078.00B28.05−0.058.00B38.04−0.048.00B48.04−0.04

Example 10

[0250] Filter refinement can compensate for variations between instruments. This example demonstrates that predicted values from different instruments can be rendered statistically equivalent using a single property model although the training set for that model does not include instrument-responses from all instruments.

[0251] The selection of spectral subregions that may not necessarily minimize RMSECV or RMSEP using the filter refinement procedure of FIG. 6 is sometimes useful in compensating for instrumental variance and can avoid having to take calibration measurements on specific instruments. This technique to compensate for instrument-to-instrument variance is based on two observations. First, a FT-NIR spectrum has a very broad spectral region (4000 cm−1 to 12,000 cm−1 or 2.5 μm to 0.83 μm), and some narrower subregions within the entire available NIR region are found to be more sensitive to instrumental variance than others. The particular subregions of higher sensitivity are determined, at least in part, by the instrument design and by the properties of specific comprising the spectrometer. As a result, these more sensitive subregions often differ between instrument manufacturers and even between different models from the same manufacturer. Second, there is sometimes an option to choose among different spectral subregions that can be used to build a property model. It has been found that if one or more acceptable spectral subregions is chosen to build the property model using filter refinement, then compensating spectra from other instruments may do need to be added to the training set of the model.

[0252] Model 10.0 was a revised property model built by refining the filter for Model 7.0 using OPUS Quant-2 to search for acceptable filters, and changing the spectral region to the refined filter was the pretreatment adjustment of block 108. This chemometric software provided a routine based on three proprietary search algorithms called NIR, General A, and General B. Table 11A summarizes the best trial regions found by OPUS Quant-2 that contained one or two subregions. It is noted that three trial regions were identified as acceptable filters since they produced RMSEP values less than the desired precision of 0.10%, namely trial regions 3, 4, and 6. According to previously described Criterion A, trial region 6 was most preferred. According to Criterion B, however, trial region 3 was preferred over trial region 6 because it had a smaller number of subregions. Since application of Criterion B resulted in a single trial region, Criterion C was not used. Hence, the refined filter (block 160, FIG. 6) for Model 10.0 was selected according to Criterion B to be trial region 3, which was the single subregion from 4597.5 cm−1 to 9395.6 cm−1.

[0253] For illustrative purposes, an alternative development of Model 10.0 was then considered. Since the RMSEP of trial region 6 was much less than the limit of desired precision, there was an opportunity to improve significantly the level of predictability of the model. Suppose that during consultation with the customer, it was decided to define an improved level of predictability such that RMSEP would be less than 0.05%, thereby redefining the objectives of block 70 in FIG. 4. Then, according to the previously described procedure for filter refinement, and using first derivative (21 smoothing points) and vector normalization pretreatment (block 102), the refined filter (block 160, FIG. 6) for Model 10.0 was selected to be trial region 6, which was the composite subregion from 4597.5 cm−1 to 6398.7 cm−1 and 7594.4 cm−1 to 8797.7 cm−1. Cross-validation of Model 10.0 gave 99.99 for R2 and 0.046% for RMSECV (block 104) with a rank of 8. The prediction offset was essentially zero, and no outliers were detected (block 106). The cross-validation curve for Model 10.0 is reproduced in FIG. 31.

13TABLE 11ATrialOPUSNo. Sub-RegionProcedureRegionsSubregion 1Subregion 2RankRMSEPRMSECV1NIR15349.6-6101.770.1180.0582NIR24597.5-6101.77497.9-9993.470.2980.0523General A14597.5-9395.690.07340.0414General A24597.5-6996.58793.9-9993.470.05270.0555General B14597.5-6398.7100.1430.0516General B24597.5-6398.77594.4-8797.780.03960.046

[0254] Validation of Model 10.0 using calibration samples and measurement conditions of the training set of Model 7.0 and using instrument responses measured by Instrument B generated a validation set (block 108) that produced RMSEP of 0.0396% (block 110) without a significant prediction offset as shown in FIG. 32. No outliers were detected (block 112). The observed and predicted values from the validation set are shown in Table 11B. These results demonstrated an acceptable level of precision based on the model generated for Instrument A but used with Instrument B. Therefore, under these measurement conditions it was possible to share a single property model with multiple FT-NIR instruments. Model 10.0 was a global property model for Instruments A and B, and this model was preferred over Model 9.0 since the training set for Model 10.0 did not require training measurements from Instrument B. Model 10.0 was ready for installation in block 114 of FIG. 5.

14TABLE 11BResidualSample No.Observed (%)InstrumentOrientationPredicted (%)(Obs-Pred)10.00B10.03−0.0310.00B2−0.030.0310.00B3−0.050.0510.00B4−0.030.0322.00B11.950.0522.00B22.000.0022.00B32.02−0.0222.00B41.990.0134.00B14.04−0.0434.00B24.000.0034.00B33.990.0134.00B43.970.0346.00B15.940.0646.00B25.960.0446.00B35.950.0546.00B45.910.0958.00B18.02−0.0258.00B27.990.0158.00B37.960.0458.00B47.960.04610.00B19.990.01610.00B29.980.02610.00B39.940.06610.00B49.940.06

Example 11

[0255] This example illustrates the method of developing several property models for a material after a feasibility study had been completed (block 100, FIG. 5) and an effectively comprehensive set of influential factors had been identified along with appropriate methods of data pretreatment. The method (block 70, FIG. 4) was defined as FT-NIR using the instrument and sample presentation device shown in FIG. 33. The objectives (block 70, FIG. 4) included measurements by non-skilled operators of the total oil, oleic and linolenic contents in canola seeds with measurement precisions characterized by RMSEP values less than 0.6% for each property as predicted by multi-instrument global property models.

[0256] A variety of canola was selected which had been bred to contain oleic acid with a target specification greater than 70% (present as the triglyceride and relative to the total oil content) and linolenic acid with a target specification less than 3.5% (present as the triglyceride and relative to the total oil content). The expected ranges of block 72 were 63% to 75% for oleic content, 2.5% to 7.8% for linolenic content, and 44% to 51% for total oil content. The observed values of total oil content were determined by extraction using a solvent-based extraction method, and those for oleic and linolenic oil content were obtained by analyzing the extracted oil using gas chromatography. The method for determining oil content was AOCS Official Method Am 2-93 (updated 1995). Oleic and linolenic oil content was determined using AOCS Official Method Ce 1-62 (revised 1990). All component concentrations were adjusted to a dry basis by subtracting the actual moisture content in each sample from the total sample weight.

[0257] The calibration set of block 74 comprised 45 canola samples that had been selected to cover the expected ranges of concentrations for total oil, oleic oil and linolenic oil, and to span the expected range of secondary material characteristics. The canola seeds typically had diameters ranging from about 1.5 to 2.5-mm. The grain had been partially cleaned by sieving as is commonly done as part of visual grading used to assess grain quality for grain transactions. A natural selection of foreign matter, called dockage in the grain industry, remained in the samples in amounts up to about two percent by weight.

[0258] For non-destructive FT-NIR measurements of whole grain, FIG. 33 shows the flow-through sample presentation device, which was comprised of the funnel 202 for presenting grain samples, the flow rate controller 206 for the grain sample, the funnel gate 208 for initiating sample flow, and the grain collector 210, and attached to the FT-NIR instrument 200 equipped with fiber optic probe 204. This device was designed to provide a significantly larger sampling area for data acquisition than would be obtained by using a similarly configured fiber optic probe to measure a stationary sample of grain.

[0259] FT-NIR measurements were done by first pouring about 250 grams of canola into the funnel 202, the funnel having an inner cross-section of 120 mm2, and then opening the funnel gate 208 half-way to permit the grain to start flowing into grain collector 210. The flow rate was set at the flow rate controller 206 to pass about 10 grams of canola per second. The fiber optic probe 204 was engaged by pressing a button on the probe trigger about one to two seconds after the grain had started to flow to initiate data collection of 40 spectra at a scanning speed of 20 kHz and about 2 scans per second. The spectra were pre-processed by averaging in interferogram mode, converting to single-scan mode by fast Fourier Transform, and then converting to an averaged absorbance spectrum for spectroscopic analysis. The averaged absorbance spectrum was evaluated by OPUS to predict values of total oil content, oleic oil content, linolenic oil content and the Mahalanobis distances from each property model.

[0260] To develop the training sets for the property models (block 102, FIG. 5), each of the 45 samples of the calibration set was first measured 3 times at ambient temperature to generate repeated measures of the instrument response. Table 12 lists five samples for the calibration set that were then used to generate additional calibration spectra for an extended training set which, in combination with data pretreatment, would compensate for an effectively comprehensive range of influential factors, including temperature, humidity, light source, fiber probe and instrument. These five calibration samples spanned the expected ranges of concentrations of total oil, oleic oil and linolenic oil contents as determined by solvent extraction and gas chromatography. It was demonstrated that the use of different sample presentation devices of the type shown in FIG. 33 was not an influential factor for the properties of interest.

15TABLE 12Sample Label% Oleic% Linolenic% OilS163.007.8246.30S273.893.2251.20S375.722.5848.10S474.252.6743.80S569.623.5738.68

[0261] Temperature compensation was included in the calibration models to accommodate sample measurements over a wide range of temperatures, specifically from about −60° C. to about 50° C. This was accomplished by first cooling samples S1 to S5 in a freezer at −70° C., bringing the samples to the spectrometer area in contact with dry ice, and then measuring the samples as they warmed slightly during the flow-through sample presentation. These five samples were then heated in an oven at 60° C. and measured as they cooled slightly during flow-through sample presentation. It is conceptually important to note that it is not necessary to know the precise values of sample temperatures while they were warming or cooling during data acquisition in order to build an acceptable multivariate calibration model. Since the NIR spectra of the calibration training set included measurements at various temperatures that spanned the expected range, the PLS procedure generated multivariate models that compensated for non-quantified temperature variance within the range of temperatures used in the training set.

[0262] Humidity compensation was included in the models using the technique of Example 6 by generating spectra with a range of humidities for three canola samples, S1, S3, and S5. The models thus compensated for non-quantified humidity variance.

[0263] To optimize the filter and validate the calibration models, these samples were measured by 4 additional Bruker MATRIX Model F FT-NIR spectrometers 200. Two spectra of each of the five samples of Table 12 measured by two of the spectrometers 200 were included in the extended training set, with the remaining spectra used to create a validation set for the models. The models thus compensated for non-quantified instrument variance. Some representative spectra of the canola measured by one spectrometer are shown in FIG. 34.

[0264] Model 11.0 was multi-instrument global a property model (block 104, FIG. 5) constructed to predict total oil content using a refined filter (block 160, FIG. 6) from 4597.5 cm−1 to 7501.8 cm−1. Data pretreatment also included a first-derivative transformation with 13 smoothing points followed by vector normalization. Cross-validation of Model 11.0 gave 97.89 for R2 and 0.527% for RMSECV with a rank of 9 for the concentration range from 38.68% to 51.20% total oil. The calibration curve of Model 11.0 is shown in FIG. 35. The observed and predicted values from the extended training set of Model 11.0 for total oil are in Table 13.

16TABLE 13ResidualSpectrumSampleObserved (%)Predicted (%)(Obs − Pred)1143.7142.551.162143.7142.461.253143.7142.870.844243.8044.66−0.865243.8044.51−0.716243.8044.89−1.097439.7838.781.008439.7838.960.829439.7839.670.1110538.6838.500.1811538.6840.06−1.3812538.6839.60−0.9213744.2544.70−0.4514744.2544.86−0.6115744.2544.89−0.6416843.3541.891.4617843.3542.041.3118843.3542.360.9919943.2343.70−0.4720943.2343.58−0.3521943.2343.55−0.32221045.8645.790.07231045.8645.91−0.05241045.8645.750.11251150.3051.26−0.96261150.3051.15−0.85271150.3051.08−0.78281248.6948.91−0.22291248.6949.44−0.75301248.6949.78−1.09311344.1244.75−0.63321344.1244.42−0.30331344.1244.57−0.45341442.2742.64−0.37351442.2742.87−0.60361442.2742.55−0.28371544.6645.27−0.61381544.6644.88−0.22391544.6645.24−0.58401647.2247.130.09411647.2247.65−0.43421647.2247.42−0.20431743.0642.430.63441743.0642.470.59451743.0642.860.20461847.2747.270.00471847.2747.230.04481847.2747.130.14491944.4543.710.74501944.4543.760.69511944.4543.810.64522045.9945.530.46532045.9945.490.50542045.9945.890.10552150.1949.660.53562150.1949.590.60572150.1950.080.11582241.8041.550.25592241.8041.87−0.07602241.8041.86−0.06612343.1843.61−0.43622343.1843.52−0.34632343.1843.81−0.63642446.4046.130.27652446.4046.350.05662446.4046.030.37672546.3046.71−0.41682546.3047.03−0.73692546.3046.58−0.28702651.0051.70−0.70712651.0051.25−0.25722651.0050.700.30732750.1050.030.07742750.1050.54−0.44752750.1049.960.14762851.2050.970.23772851.2051.140.06782851.2050.700.50792948.1047.650.45802948.1047.880.22812948.1047.470.63823045.5045.59−0.09833045.5045.260.24843045.5045.500.00853146.9046.620.28863146.9046.700.20873146.9046.350.55883242.8043.83−1.03893242.8043.25−0.45903242.8043.34−0.54913342.2042.37−0.17923342.2042.080.12933342.2042.160.04943447.2047.28−0.08953447.2047.75−0.55963447.2047.190.01973547.8047.800.00983547.8047.130.67993547.8047.610.191003643.8043.180.621013643.8043.550.251023643.8043.84−0.041033743.8043.520.281043743.8043.700.101053743.8043.92−0.121063844.5045.05−0.551073844.5044.460.041083949.4049.350.051093949.4048.750.651103949.4048.960.441114140.0040.67−0.671124140.0040.70−0.701134140.0040.60−0.601144241.1040.750.351154241.1040.940.161164241.1041.090.011174350.1050.45−0.351184350.1050.29−0.191194350.1050.43−0.331204445.5045.010.491214445.5045.380.121224445.5045.75−0.251232851.2050.700.501242851.2050.890.311252851.2050.860.341262948.1047.540.561272948.1047.310.791282948.1048.010.091292750.1050.010.091302750.1049.990.111312750.1049.320.78132243.8044.26−0.46133243.8044.70−0.90134243.8044.40−0.60135538.6839.43−0.75136538.6839.31−0.63137538.6839.36−0.681382546.3046.110.191392546.3046.170.131402546.3046.110.191412851.2051.64−0.441422851.2051.51−0.311432851.2051.60−0.401442948.1047.510.591452948.1047.910.191462948.1047.940.161472750.1049.750.351482750.1050.000.101492750.1049.930.17150243.8042.850.95151243.8043.500.30152243.8043.440.36153538.6839.09−0.41154538.6839.06−0.38155538.6838.73−0.051562546.3046.33−0.031572546.3046.76−0.461582546.3046.130.17159243.8043.570.23160243.8044.06−0.26161538.6839.34−0.66162538.6839.72−1.041632546.3046.53−0.231642546.3045.750.551652851.2051.25−0.051662851.2050.860.341672948.1047.091.01168243.8043.640.16169243.8043.070.73170538.6837.930.75171538.6837.770.911722546.3046.060.241732546.3045.690.611742851.2052.25−1.051752851.2050.870.331762948.1047.310.791772948.1047.730.37178243.8044.04−0.24179243.8043.780.02180538.6839.36−0.68181538.6839.18−0.501822546.3045.870.431832546.3046.47−0.171842851.2051.50−0.301852851.2051.30−0.101862948.1047.150.951872948.1047.880.22188243.8043.720.08189243.8043.91−0.11190538.6839.00−0.32191538.6838.470.211922546.3046.180.121932546.3046.58−0.281942851.2051.080.121952851.2051.68−0.481962948.1047.380.721972948.1047.670.43

[0265] Model 11.1 was a multi-instrument global property model (block 104, FIG. 5) constructed to predict the oleic oil content using a refined filter (block 160, FIG. 6) of two spectral subregions, 4246.5 cm−1 to 4601.4 cm−1 and 5449.9 cm−1 to 7501.8 cm−1. Data pretreatment also included a first-derivative transformation with 17 smoothing points followed by vector normalization. Cross-validation of Model 11.1 gave 98.01 for R2 and 0.525% for RMSECV with a rank of 14 within the concentration range from 63.00% to 75.72%. The calibration curve for Model 11.1 is shown in FIG. 36. The observed and predicted values from the extended training set of Model 11.1 for oleic oil are in Table 14.

17TABLE 14ResidualSpectrumSampleObserved (%)Predicted (%)(Obs − Pred)1172.8272.590.232172.8272.330.493172.8272.720.104274.2574.140.115274.2574.55−0.306274.2574.41−0.167569.6269.150.478569.6270.44−0.829569.6270.46−0.8410674.2573.860.3911674.2574.020.2312674.2574.170.0813773.9173.290.6214773.9173.520.3915773.9173.190.7216873.2272.490.7317873.2272.450.7718873.2272.710.5119971.1871.45−0.2720971.1871.22−0.0421971.1871.83−0.65221074.6474.350.29231074.6474.67−0.03241074.6474.630.01251174.5974.470.12261174.5974.65−0.06271174.5974.77−0.18281274.3974.48−0.09291274.3974.47−0.08301274.3974.270.12311373.4873.310.17321373.4873.58−0.10331373.4873.54−0.06341473.9873.870.11351473.9873.350.63361473.9873.530.45371573.4973.58−0.09381573.4973.200.29391573.4973.160.33401675.0174.370.64411675.0174.001.01421675.0174.290.72431771.7172.41−0.70441771.7172.57−0.86451771.7172.99−1.28461873.8774.35−0.48471873.8774.10−0.23481873.8773.470.40491968.0568.20−0.15501968.0568.34−0.29511968.0568.42−0.37522071.8771.590.28532071.8771.92−0.05542071.8771.820.05552174.9674.670.29562174.9674.910.05572174.9674.630.33582270.7470.79−0.05592270.7470.660.08602270.7470.310.43612371.6871.500.18622371.6871.530.15632371.6871.97−0.29642470.5470.320.22652470.5470.310.23662470.5470.040.50672563.0062.450.55682563.0062.830.17692563.0062.650.35702675.1876.22−1.04712675.1876.31−1.13722873.8973.360.53732873.8973.91−0.02742873.8974.07−0.18752975.7276.04−0.32762975.7276.62−0.90772975.7275.260.46783073.6774.35−0.68793073.6773.460.21803073.6773.96−0.29813173.8873.220.66823173.8873.810.07833173.8874.13−0.25843271.8971.060.83853271.8971.280.61863271.8971.290.60873367.1367.48−0.35883367.1367.29−0.16893367.1367.29−0.16903466.8966.99−0.10913466.8967.17−0.28923466.8967.31−0.42933572.7273.84−1.12943572.7273.52−0.80953572.7273.50−0.78963671.1271.030.09973671.1271.040.08983671.1270.850.27993771.3970.960.431003771.3970.770.621013771.3971.250.141024171.4371.260.171034171.4371.240.191044272.3671.800.561054272.3671.790.571064373.6773.91−0.241074373.6773.69−0.021084373.6773.89−0.221092873.8973.200.691102873.8973.360.531112975.7275.530.191122975.7275.350.371132975.7275.490.23114274.2574.85−0.60115274.2574.71−0.46116274.2574.73−0.48117569.6270.04−0.42118569.6269.400.22119569.6269.98−0.361202563.0063.24−0.241212563.0063.15−0.151222563.0063.61−0.611232873.8974.44−0.551242873.8974.10−0.211252873.8973.94−0.051262975.7275.020.701272975.7275.580.141282975.7275.250.47129274.2574.50−0.25130274.2574.100.15131274.2574.58−0.33132569.6270.21−0.59133569.6270.60−0.98134569.6270.32−0.701352563.0063.22−0.221362563.0063.22−0.221372563.0062.800.20138569.7870.79−1.01139569.7870.81−1.03140569.7870.13−0.35141569.7870.18−0.40142569.7870.20−0.42143569.7870.25−0.47144569.7870.29−0.51145569.7870.23−0.45146274.2573.690.56147274.2573.650.601482563.0063.30−0.301492563.0063.74−0.741502873.8974.01−0.121512873.8974.34−0.451522975.7275.88−0.16153274.2573.820.43154569.6270.12−0.50155569.6269.560.061562563.0062.940.061572563.0061.931.071582873.8974.11−0.221592873.8972.761.131602975.7274.641.081612975.7276.00−0.28162274.2573.840.41163274.2575.07−0.82164569.6269.87−0.25165569.6269.87−0.251662563.0062.910.091672563.0063.000.001682873.8973.150.741692873.8973.220.671702975.7275.710.011712975.7275.85−0.13172569.6268.271.35173569.6268.231.391742873.8972.561.331752873.8974.08−0.19176274.2575.83−1.58177274.2574.79−0.541782563.0062.740.261792563.0063.94−0.941802975.7276.12−0.401812975.7275.330.39182274.2574.240.01183274.2574.63−0.38184569.6269.570.05185569.6269.560.061862563.0064.02−1.021872563.0062.890.111882873.8974.12−0.231892873.8974.57−0.681902975.7275.450.271912975.7275.650.071922563.0062.230.771932563.0063.23−0.231942975.7275.660.061952975.7275.90−0.18

[0266] Model 11.2 was a multi-instrument global property model (block 104, FIG. 5) constructed to predict linolenic oil content using a refined filter (block 160, FIG. 6) of one spectral subregion, 4616.8 cm−1 to 6067.0 cm−1. Data pretreatment also included a first-derivative transformation with 17 smoothing points followed by vector normalization. Cross-validation of Model 11.2 gave 96.79 for R2 and 0.262% for RMSECV with a rank of 13 within the concentration range from 1.88% to 7.82%. The calibration curve for Model 11.2 is shown in FIG. 37. The observed and predicted values from the extended training set of Model 11.2 for linolenic oil are in Table 15.

18TABLE 15ResidualSpectrumSampleObserved (%)Predicted (%)(Obs − Pred)112.753.27−0.52212.752.95−0.20312.753.00−0.25422.672.280.39522.672.500.17622.672.490.18733.924.28−0.36833.923.94−0.02933.923.710.211044.054.020.031144.054.10−0.051244.053.820.231353.573.470.101453.573.540.031553.573.62−0.051662.932.95−0.021762.932.630.301862.932.930.001972.792.720.072072.792.650.142172.792.680.112282.862.97−0.112382.862.850.012482.863.00−0.142593.193.23−0.042693.193.55−0.362793.193.42−0.2328102.402.41−0.0129102.402.340.0630102.402.250.1531112.682.650.0332112.682.430.2533112.682.74−0.0634122.652.98−0.3335122.652.76−0.1136122.652.66−0.0137132.542.68−0.1438132.542.450.0939132.542.86−0.3240142.702.660.0441142.702.590.1142142.702.590.1143153.002.790.2144153.002.850.1545153.002.850.1546162.482.64−0.1647162.482.54−0.0648162.482.350.1349173.162.900.2650173.163.100.0651173.162.980.1852182.922.380.5453182.922.410.5154182.922.800.1255203.193.82−0.6356203.193.72−0.5357203.193.74−0.5558212.592.61−0.0259212.592.530.0660212.592.80−0.2161223.373.63−0.2662223.373.46−0.0963223.373.87−0.5064233.333.37−0.0465233.333.330.0066233.333.38−0.0567244.314.40−0.0968244.313.990.3269244.314.060.2570257.827.87−0.0571257.827.710.1172257.827.680.1473262.572.150.4274262.572.120.4575262.572.370.2076271.882.09−0.2177271.881.760.1278271.881.99−0.1179283.223.27−0.0580283.223.32−0.1081283.223.40−0.1882292.582.68−0.1083292.582.370.2184292.582.74−0.1685302.772.600.1786302.772.560.2187302.772.540.2388312.792.88−0.0989312.793.12−0.3390312.792.87−0.0891322.853.28−0.4392322.853.26−0.4193322.853.06−0.2194334.174.27−0.1095334.174.33−0.1696334.174.21−0.0497346.175.890.2898346.175.880.2999346.175.780.39100352.952.840.11101352.952.930.02102352.953.01−0.06103363.383.41−0.03104363.383.380.00105363.383.340.04106373.403.61−0.21107373.403.44−0.04108373.403.360.04109382.792.320.47110382.792.220.57111393.043.06−0.02112393.042.970.07113393.043.040.00114413.083.29−0.21115413.083.33−0.25116422.902.830.07117422.902.860.04118422.903.01−0.11119433.313.300.01120433.313.290.02121433.312.910.40122442.642.400.24123442.642.440.20124442.642.500.14125283.223.78−0.56126283.223.61−0.39127283.223.31−0.09128292.582.59−0.01129292.582.70−0.12130292.582.82−0.24131271.882.35−0.47132271.882.27−0.39133271.882.29−0.4113422.672.350.3213522.672.340.3313622.672.650.0213753.573.110.4613853.573.360.2113953.573.210.36140257.827.320.50141257.827.310.51142257.826.990.83143283.223.090.13144283.223.000.22145283.223.160.06146292.582.490.09147292.582.420.16148292.582.330.25149271.882.14−0.26150271.882.13−0.25151271.882.01−0.1315222.672.71−0.0415322.672.74−0.0715422.672.74−0.0715553.573.58−0.0115653.573.530.0415753.573.62−0.05158257.827.610.21159257.828.08−0.26160257.828.22−0.4016122.672.610.0616222.672.520.1516353.573.190.3816453.573.59−0.02165257.828.09−0.27166257.827.790.03167283.223.56−0.34168283.223.55−0.33169292.582.260.32170292.582.510.0717122.672.250.4217222.672.470.2017353.573.83−0.2617453.574.14−0.57175257.827.800.02176257.827.570.25177283.223.170.05178283.222.620.60179292.582.370.21180292.582.68−0.1018122.672.620.0518222.672.99−0.3218353.573.500.0718453.573.74−0.17185257.827.410.41186257.828.11−0.29187283.223.25−0.03188283.223.49−0.27189292.582.480.10190292.582.350.2319122.672.91−0.2419222.672.82−0.1519353.573.85−0.2819453.573.95−0.38195257.827.620.20196257.827.91−0.09197283.223.70−0.48198283.223.200.02199292.582.73−0.15200292.582.470.11

[0267] Occasional missing data in Tables 13 to 15 resulted from omitting observations from the training sets that were identified as bad outliers (block 106, FIG. 5), which occurred when the quality of an acquired spectrum was poor due to unexpected interruptions in the grain flow during data acquisition or if an inaccuracy in the reference data on specific properties of some samples was subsequently identified and could not be corrected.

[0268] This example demonstrates that it is possible to build multivariate calibrations over a wide range of expected temperatures (from about −60 to about +50° C.), but including calibration data over such a wide temperature range tends to decrease the precision of predicted results. If it is desired or necessary to increase the precision of predicted results beyond that used in this example, and if it is possible to precondition or ensure that the samples that will be measured at remote locations to a narrower range of temperatures and thereby adjust the objectives of block 82 in FIG. 4, then other models can be built from training sets that span temperatures over a narrower range.

[0269] The above three property models were validated using 50 validation samples (block 108, FIG. 5) not used in the training sets by taking measurements over a range of measurement conditions with two Bruker MATRIX Model F FT-NIR instruments in Sensors A and B that had not been used previously. The validation results (block 110, FIG. 5) are shown in Table 16. No outliers were detected in the validation set (block 112, FIG. 5).

19TABLE 16RMSEPRMSEPofR2 ofofR2 ofSensorSensorSensorSensorPropertyRangeAABBWt % total40.0-50.190.5296.080.5595.47oilWt % oleic66.85-76.00 0.5395.040.5694.21oilWt %1.88-7.50 0.2596.850.2696.92linolenicoil

[0270] These results showed that the RMSEP values from the validation sets for the three properties were each close to the RMSECV values of the corresponding property models, each were within the desired upper precision limit of 0.6%, and differences in the predicted values from different instruments were not significant. Models 11.0, 11.1 and 11.2 were thus considered to be ready for installation (block 114, FIG. 5) in the central processor 10.

Example 12

[0271] The Mahalanobis distance can be used to identify bad outliers which may arise from invalid measurements. For Examples 12 to 15, a threshold value of the Mahalanobis distance for good outliers was calculated by OPUS Quant-2 to be 0.42. The threshold values for bad outliers and for extremely bad outliers were taken to be 1.0 and 100.0, respectively.

[0272]
FIG. 38 shows an abnormal FT-NIR spectrum acquired after an instrument malfunction which caused the excitation source to fail. Model 11.1 predicted that the oleic oil in the sample was 199.0%. Since the Mahalanobis distance for this spectrum was 390.00, the predicted value was correctly identified as an extremely bad outlier and the corresponding measurement results were considered invalid.

Example 13

[0273] The Mahalanobis distance can be used to identify bad outliers which may arise from invalid sample presentation.

[0274]
FIG. 39 shows two NIR spectra acquired on the same sample of canola seeds using the flow-through sample presentation system described in Example 11. The known value of oleic oil for this sample was 73.1%. The upper spectrum was taken on a 250 gram sample (a valid sample size according to the method of Example 11), while the lower spectrum was taken on a 100 gram sample (an invalid sample size according to the method of Example 11). Model 11.1 predicted that the oleic oil from the upper spectrum was 73.0% while that of the lower spectrum was 66.1%. Since the Mahalanobis distances of the upper spectrum was 0.25 while that of the lower spectrum was 5.00, the upper spectrum was a presumably valid measurement, while the lower spectrum was correctly identified as a bad outlier and the corresponding measurement results were considered invalid.

Example 14

[0275] The Mahalanobis distance can be used to identify bad outliers which may arise from valid measurements on samples taken from populations different from that of the training set.

[0276]
FIG. 40 shows an NIR spectrum of a sample of wheat using the sample presentation system of Example 11. Model 11.1 predicted that the oleic oil in the sample was 33.2%. Since the Mahalanobis distance of this spectrum was 44.00, the predicted value was correctly identified as a bad outlier and the corresponding measurement results were considered invalid.

Example 15

[0277] The Mahalanobis distance can be used to identify good outliers arising from valid measurements on samples with secondary material characteristics that differ in some characteristic way from the secondary material characteristics of samples included in the training set.

[0278]
FIG. 41 shows six NIR spectra from three different samples of Variety B canola seeds, with duplicate spectra taken for each sample using the sample presentation system of Example 11. Variety B was characteristically different from Variety A, which was the variety of canola used to develop Model 11.1. The observed and predicted values of oleic oil in these three samples according to Model 11.1 is summarized in Table 17. Since the Mahalanobis distance of each spectrum was greater than the threshold value for good outliers, all predicted results on Variety B canola were good outliers and the corresponding measurement results were considered invalid.

20TABLE 17SampleObservedPredictedMAHS172.2%73.3%, 73.0%0.72, 0.71S270.6%71.8%, 72.3%0.71, 0.73S371.3%72.6%, 72.2%0.86, 0.69

[0279] Model 15.0 was a property model constructed by including two spectra from sample S1 in the training set of Model 11.1 while maintaining the corresponding refined filter and pretreatment transformations. The results predicted from Model 15.0 using a validation set containing four spectra from samples S2 and S3 are given in Table 18.

21TABLE 16SampleObservedPredictedMAHS270.6%70.9%, 71.5%0.29, 0.33S371.3%71.5%, 71.6%0.33, 0.29

[0280] Since the Mahalanobis distances in Table 18 were each less than the threshold for good outliers, the predicted results on Variety B canola are no longer probable outliers. The measurement results in Table 18 are considered valid, and Model 15.0 compensates for a wider range of influential factors.

Example 16

[0281] A collection of Bruker MATRIX Model F FT-NIR instruments, sample presentation devices of the type shown in FIG. 33, and laptop computers loaded with Microsoft® Windows® 2000 and the Bruker OPUS 3.01 software were transported to several sites in Canada, remote from the central processor residing in Cincinnati, Ohio. Two remote analysis systems were assembled at two separate sites in Manitoba, Canada by separate personnel, with each sensor 2 comprising one each of the NIR instrument 30, equipped with a sample presentation device 22 (shown in FIG. 33) and a laptop computer serving as the local processor 34. Using appropriate communications software on the local processor 34, the systems were connected to their own separate local area networks with Internet connectivity.

[0282] Once Internet connectivity was established, operators at both sites initiated secure connections to the central processor 10, using a graphical user interface on the local processor 34, and entered device-specific unique user identification codes and passwords for approval by the security controller 54. Successful connection to the central processor 10 typically occurred within one minute.

[0283] Upon connectivity to the central processor 10, the user interface 32 prompted for sample identification. This remote user-supplied sample identification data, along with the subsequently acquired multichannel data, was transmitted to the central processor 10 and used by the analysis engine 56 to select the appropriate parameters from the data repository 58 for three multi-instrument global property models: Model 11.0 for total oil, Model 11.1 for oleic content, and Model 11.2 for linolenic content.

[0284] After entering the sample identification data, the operators placed about 250 grams of whole canola seed samples into the funnel 202 of the sample presentation device 22. The funnel gate 208 was opened and the instrument 200 was then activated by the user interface 32 and the flow of the canola was initiated through the sample chamber 24 past the fiber optic probe 204. Within seconds after the sample flow had completed, the measurement data 12 was transmitted over the secure Internet connection 8 to the central processor 10. The analysis engine 56 computed values for the properties of interest based on Models 11.0, 11.1 and 11.2 (block 116, FIG. 5), and after testing for outliers (block 118, FIG. 5) the measurement results 14 were sent back to the individual sensors 2 (block 120, FIG. 5) via Internet connection 8, all without additional remote user action. The measurement data 12 and the measurement results 14 were stored in the data repository 58. The elapsed time from initial user interface prompt to display of predicted values at an output device of the user interface 32 was generally from about 1 to about 2 minutes. The time interval between transmission of the measurement data 12 and transmission of the measurement results 14 was generally less than one minute.

[0285] Table 19 lists the measurement results 14 generated from sensors A and B during a short time interval when both sensors were performing near-simultaneous analyses. Table 19 is an example of a report of historical information including two or more measurement results 14 acquired from at least one data acquisition device 2 that was transmitted to at least one user interface as aggregated results.

22TABLE 19Submission% TotalTotal OilOleicLinoleicTest No.TimeLocationSensorOilMAH% OleicMAH% LinolenicMAH112:29ManitobaA45.30.173.10.23.20.3212:31ManitobaA45.00.172.60.33.20.1312:31ManitobaB50.50.276.90.22.40.6412:32ManitobaA45.20.173.30.34.00.2512:33ManitobaB50.50.377.10.21.90.7

[0286] Two of the predicted results in Table 19, test numbers 3 and 5 for linolenic content, were identified as good outliers (blocks 118,122, and 124 in FIG. 5). Investigation revealed that the corresponding samples were a different variety of canola than that used to develop the calibration models. If during customer consultation it had been learned that the new variety was an experimental crop that would no longer be produced, it might be decided not to extend the training set (block 126, FIG. 5) so any additional samples of the experimental variety would also be considered invalid during future on-site measurements (block 116, FIG. 5). If, however, it had been learned that the new variety was scheduled for continued production, then it could be decided to extend the training set (block 126, FIG. 5) so future measurements using a new property model (developed according to blocks 102, 104, 106, 108, 110, 112 as well as blocks 124, 126, 128, and 130 as required) installed on the central processor 10 (block 114, FIG. 5) would provide valid predictions from on-site measurements (block 116, FIG. 5) for both new and old varieties of canola.

[0287] Additionally, as demonstrated by the data in Table 20, analyses were generated on a different date after transporting sensor A to a third remote site in Saskatchewan, Canada, with measurement results 14 being returned to user interface 32 at the third site which submitted data to the central processor 10 with brief lapses over the course of about 30 minutes.

23TABLE 20Submission% TotalTotal OilOleicLinoleicTest No.TimeLocationSensorOilMAH% OleicMAH% LinolenicMAH112:32SaskatchewanA47.60.276.50.22.10.2212:35SaskatchewanA41.60.272.30.22.80.2312:37SaskatchewanA47.90.275.70.12.30.3412:40SaskatchewanA47.60.275.30.12.10.2512:41SaskatchewanA44.10.176.00.12.20.2612:44SaskatchewanA48.00.373.70.22.70.2712:46SaskatchewanA41.10.170.00.13.10.2812:51SaskatchewanA40.80.271.10.13.10.2912:54SaskatchewanA45.50.274.50.12.30.31012:56SaskatchewanA46.80.275.20.22.10.21112:58SaskatchewanA47.50.176.70.11.70.31213:00SaskatchewanA42.80.174.20.12.50.21313:02SaskatchewanA48.90.276.70.12.00.31413:04SaskatchewanA47.10.276.30.12.10.2

[0288] The on-site analysis system is simple to operate at each location where data is acquired. Because only one property model algorithm is needed for each property of interest as specified in the method and objectives of block 70 in FIG. 4, and each of these is stored on the central processor, the operator need only have the sample 20 ready to be detected at the sample presentation device 22 and initiate the data acquisition by following the prompts which appear at the user interface 32. A semiskilled, or even non-skilled operator is able to perform the steps needed to acquire data on the sample.

[0289] Also, the analysis system and method enables the customer to submit guidelines to be stored in the data repository 58 via an appropriate security code to the security controller 54 of the central processor 10 to provide annotations to the measurement results 14 to provide customer specific interpretations and help text. Thus, the customer could independently specify whether a particular range of predicted values is a “Pass” or “Fail” for the property of interest. The customer can instruct the central processor 10 to transmit this annotation as appropriate for particular predicted results generated by the property model algorithm. It is also possible for the customer to command the central processor 10 from an off-site user interface 158 to generate custom spreadsheets of historical results for forecasting or quality assurance purposes.

[0290] The on-site analysis system and method of analysis can be used in a range of applications for obtaining information on a number of materials. A multi-instrument global property model which can be refined to a high degree of precision, as necessary, and coupled with substantial immunity to instrument, sample, environmental, and sample presentation variance can be used to produce measurement results 14 which are accepted within a particular trade. Thus, it is possible that an on-site analysis system and method of analysis can be devised with proper attention to inclusion of the requisite variables for the creation of a method whereby results can be certified by a sanctioning body. Because the processing of data received from individual sensors 2, 4, 6 is conducted by a single property model algorithm of the central processor 10 for each property of interest without instrument-specific calibration transfer, predicted results from a number of sensors of the same sensor-type can be directly compared and certified.

[0291] It is possible using this analysis system and method of analysis to compensate for individual sensor variation at the respective sampling locations. Thus, a global property model can be generated which not only compensates for changes to components in one sensor over time, such as the output of an excitation source, but also compensates for inherent differences between similarly constructed sensors. For the case of spectroscopic instruments, it is preferred that the instrument manufacturer generate a line of equipment with individual instruments being sufficiently similar, having minimized differences as to the following characteristics or components: light intensity; optical parts; alignment of the optical parts; detector performance; and wavelength accuracy.

[0292] An additional benefit of using a central processor generating intercomparable results lies in the value generated by archival analysis of historical information which is built up over time. In the case of analyzing oilseeds, for example, information can be stored concerning the specific location of a particular oilseed analysis with the predicted results of that analysis. At the time of the data acquisition, input fields of a user interface can include other information which would be uniquely beneficial for the particular material being analyzed, for example, as part of an electronic identity preservation system. Financial and crop output predictive studies may also utilize this information.

[0293] It is anticipated that the analytical system and method of analysis described herein can be utilized in a wide range of applications. These include a number of agriculture-related applications, such as the analysis of oilseed crops, the analysis of grain, electronic grading, farm chemicals blending, soil condition analysis, waste monitoring, plant nutrition analysis, single-seed analysis, determination of harvest readiness, manufacturing of animal feed and forage, dietary supplements, and raw milk and dairy products handling and processing. In the area of healthcare, the system and method can be used in blood analysis, biological sample analysis, skin disease diagnostics, non-invasive human and animal testing, and drug testing. In chemical manufacturing, the system and method can be used in raw material qualification, process control, quality assurance testing, in-process and finished jet, diesel, automotive fuel quality and identity, and effluent monitoring. In textiles, applications include raw materials qualification, fiber properties qualification, blending and application monitoring, and effluent monitoring. In surface treatment, applications include metal treatment analysis, metal wear measurement, coating thickness analysis, and adhesive application measurement. In consumer testing, applications include determination of the fat content of meat, ripeness of fruits and vegetables, automotive fluids check, fuel octane monitoring, exhaust monitoring and personal medical checks, such as for diabetes and cholesterol.

[0294] Thus it is apparent that there has been provided, in accordance with the invention, an analysis system, a method of analysis and a method of supplying analysis services to customers which fully satisfies the objects, aims, and advantages set forth above. While the invention has been described in conjunction with specific embodiments thereof, it is evident that many alternatives, modifications, and variations will be apparent to those skilled in the art in light of the foregoing description. Accordingly, departures may be made from such details without departing from the spirit or scope of the general inventive concept.

Claims

1. A method of predicting a value of a property of interest of a material comprising: using a calibration model to process data obtained from a sample of the material, wherein the calibration model is configured to compensate for instrument variance in predicting the value of the property of interest.
2. The method of claim 1 wherein the calibration model is also configured to compensate for sample variance.
3. The method of claim 1 wherein the calibration model is also configured to compensate for environmental variance.
4. The method of claim 1 wherein the calibration model is also configured to compensate for sample presentation variance.
5. The method of claim 2 wherein the calibration model is also configured to compensate for environmental variance.
6. The method of claim 2 wherein the calibration model is also configured to compensate for sample presentation variance.
7. The method of claim 5 wherein the calibration model is also configured to compensate for sample presentation variance.
8. The method of claim 1 wherein the calibration model is configured to compensate for variations in orientation of an excitation source relative to the sample.
9. The method of claim 1 wherein the calibration model is configured to compensate for variations in age of an excitation source.
10. The method of claim 1 wherein the calibration model is configured to compensate for variations in mechanical alignment of optical components in the data acquisition device.
11. The method of claim 1 wherein the calibration model is configured to compensate for variations in the output of a detector.
12. The method of claim 1 wherein the calibration model is configured to compensate for variations in the output of an excitation source.
13. The method of claim 1 wherein the calibration model is configured to compensate for variations in transmission of a data acquisition probe.
14. The method of claim 1 wherein the data are transmitted over a communication link to a central processor.
15. The method of claim 14 wherein the data is acquired using a data acquisition device at a location remote from the central processor.
16. The method of claim 14 wherein the calibration model is configured to compensate for variance in more than one data acquisition device connectable to the central processor by the communication link.
17. The method of claim 1 further comprising pretreating the data.
18. The method of claim 14 further comprising pretreating the data after transmission over the communication link.
19. The method of claim 1 further comprising pre-processing the data.
20. The method of claim 1 wherein the data are pretreated with a refined filter.
21. The method of claim 14 wherein the data are pretreated with a refined filter after transmission over a communication link.
22. The method of claim 20 wherein the data are pre-processed.
23. The method of claim 21 wherein the data are pre-processed.
24. The method of claim 1 wherein the data are the result from electromagnetic radiation detected from the sample.
25. The method of claim 24 wherein the data are pre-processed.
26. The method of claim 24 wherein the data are pre-processed using Fourier transformation.
27. The method of claim 1 wherein the data are the result of non-stimulated emission radiation detected from the sample.
28. The method of claim 1 wherein the data are the result from the sample being subjected to mass spectrometry.
29. The method of claim 1 wherein the data are the result from the sample being subjected to chromatography.
30. The method of claim 14 further comprising transmitting a predicted value from the central processor to at least one user interface.
31. The method of claim 14 further comprising transmitting a predicted value to a user interface in the vicinity of the data acquisition device.
32. The method of claim 1 wherein the data are an accumulation of individual repetitive runs.
33. The method of claim 32 wherein the data are pre-processed.
34. The method of claim 1 wherein at least one probable outlier is used in connection with updating the calibration model.
35. The method of claim 1 wherein at least one probable outlier detected from one data acquisition device is used in connection with updating the calibration model for all data acquisition devices using the calibration model.
36. The method of claim 34 wherein the outlier is good.
37. The method of claim 35 wherein the outlier is good.
38. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified instrument variance.
39. The method of claim 1 wherein the calibration model is also configured to compensate for non-quantified sample variance.
40. The method of claim 1 wherein the calibration model is also configured to compensate for non-quantified environmental variance.
41. The method of claim 1 wherein the calibration model is also configured to compensate for non-quantified sample presentation variance.
42. The method of claim 39 wherein the calibration model is also configured to compensate for non-quantified environmental variance.
43. The method of claim 39 wherein the calibration model is also configured to compensate for non-quantified sample presentation variance.
44. The method of claim 42 wherein the calibration model is also configured to compensate for non-quantified sample presentation variance.
45. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified variations in orientation of an excitation source relative to the sample.
46. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified variations in age of an excitation source.
47. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified variations in mechanical alignment of optical components in the data acquisition device.
48. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified variations in the output of a detector.
49. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified variations in the output of an excitation source.
50. The method of claim 1 wherein the calibration model is configured to compensate for non-quantified variations in transmission of a data acquisition probe.
51. The method of claim 1 wherein at least a portion of the measurement data is stored in a database of the central processor.
52. The method of claim 1 wherein at least a portion of the measurement results is stored in a database of the central processor.
53. The method of claim 1 wherein two or more measurement results acquired from at least one data acquisition device are transmitted to at least one user interface as aggregated results.
54. The method of claim 19 wherein the preprocessing uses a background spectrum.
55. The method of claim 54 wherein the background spectrum is acquired from a single data acquisition device.
56. The method of claim 55 wherein the background spectrum is stored at a location, and is used to generate more than one predicted value of a property of interest using the data acquisition device.
57. The method of claim 54 wherein the background spectrum is generated from an accumulation of background spectra acquired from one or more data acquisition devices.
58. The method of claim 57 wherein the background spectrum is stored at a location, and is used to generate more than one predicted value of a property of interest using the data acquisition device.
59. The method of claim 7 wherein the data are transmitted over a communication link to a central processor.
60. The method of claim 59 wherein the data is acquired using a data acquisition device at a location remote from the central processor.
61. The method of claim 59 wherein the calibration model is configured to compensate for variance in more than one data acquisition device connectable to the central processor by the communication link.
62. The method of claim 7 further comprising pretreating the data.
63. The method of claim 59 further comprising pretreating the data after transmission over the communication link.
64. The method of claim 7 further comprising pre-processing the data.
65. The method of claim 7 wherein the data are pretreated with a refined filter.
66. The method of claim 59 wherein the data are pretreated with a refined filter after transmission over a communication link.
67. The method of claim 65 wherein the data are pre-processed.
68. The method of claim 66 wherein the data are pre-processed.
69. The method of claim 7 wherein the data are the result from electromagnetic radiation detected from the sample.
70. The method of claim 56 wherein the background spectrum is stored in a local processor connected to the data acquisition device.
71. The method of claim 56 wherein the background spectrum is stored in the central processor.
72. The method of claim 56 wherein the background spectrum is stored at a location remote from the data acquisition device, and connected to the data acquisition device by a communication link.
73. The method of claim 1 wherein parameters defining a calibration model are transmitted from a central processor along a communication link to a local processor.
74. The method of claim 1 wherein parameters defining a property model are transmitted from a central processor along a communication link to a local processor.
75. The method of claim 73 wherein the parameters defining the calibration model are transmitted to the local processor of at least one data acquisition device remote from the central processor.
76. The method of claim 74 wherein the parameters defining the calibration model are transmitted to the local processor of at least one data acquisition device remote from the central processor.
77. A system for analyzing a material by predicting a value of a property of interest of the material comprising: at least one data acquisition device for obtaining data on a sample of the material; and a central processor connectable to the data acquisition device over a communication link, the central processor loaded with a calibration model configured to compensate instrument variance in predicting the value of the property of interest.
78. The system of claim 77 further comprising a local processor to receive the predicted values for the property of interest.
79. The system of claim 78 wherein the local processor outputs the predicted values for the property of interest to a user interface.
80. The system of claim 77 wherein a user interface is located in the vicinity of the data acquisition device.
81. The system of claim 77 wherein the data acquisition device is in a location geographically removed from the central processor.
82. The system of claim 77 wherein the data acquisition device includes a local processor for pre-processing the data prior to transmitting the data to the central processor.
83. The system of claim 77 wherein the communication link is an Internet connection.
84. The system of claim 77 wherein the communication link is a private link.
85. The system of claim 78 wherein the local processor is configured to pre-process the data.
86. The system of claim 77 wherein the central processor is configured to pretreat the data.
87. The system of claim 77 wherein the central processor is configured to pretreat the data with a refined filter.
88. The system of claim 78 wherein the local processor is configured to pretreat the data with a refined filter.
89. The system of claim 78 wherein a user interface is located in the vicinity of the data acquisition device.
90. The system of claim 78 wherein the data acquisition device is in a location geographically removed from the central processor.
91. The system of claim 78 wherein the data acquisition device includes a local processor for pre-processing the data prior to transmitting the data to the central processor.
92. The system of claim 78 wherein the communication link is an Internet connection.
93. The system of claim 78 wherein the communication link is a private link.
94. The system of claim 78 wherein the central processor is configured to pretreat the data.
95. The system of claim 78 wherein the central processor is configured to pretreat the data with a refined filter.
96. The system of claim 77 wherein the communication link is a combination of public and private links.
97. The system of claim 78 wherein the communication link is a combination of public and private links.
98. A method of generating a calibration model for predicting a value of a property of interest from data acquired on an unknown sample of material comprising: obtaining a preliminary model for predicting the property of interest, the model developed from a training set using at least one instrument; identifying at least one factor which may influence the predictive ability of the preliminary model for the property of interest; determining the at least one factor which influences the predictive ability of the preliminary model outside a limit of defined precision; and revising the preliminary model to compensate for variation in the at least one factor which influences the property of interest to generate the calibration model, the models predicting the value of the property of interest within the limits of defined precision.
99. The method of claim 98 further comprising pretreating at least a portion of data in the training set.
100. The method of claim 99 further wherein pretreating at least a portion of the training set precedes creating the preliminary model.
101. The method of claim 99 further wherein pretreating at least a portion of the training set follows creating the preliminary model.
102. The method of claim 99 wherein pretreating at least a portion of the training set comprises mathematically transforming the training set.
103. The method of claim 99 wherein pretreating at least a portion of the training set comprises filtering the training set.
104. A method of updating a calibration model to compensate for a new influential factor for predicting a property of interest, comprising: obtaining a calibration model for predicting the property of interest, the model developed from a training set using at least one instrument; identifying a new factor which may influence the predictive ability of the generated calibration model for the property of interest using a validation set spanning at least a portion of a range of the new factor; calculating the RMSEP of the validation set to determine whether the new factor is influential; determining whether the new influential factor causes the predicted ability of the generated calibration model to fall outside a limit of precision defined in terms of RMSEP; and updating the generated calibration model to compensate for variance in the new influential factor to generate a revised calibration model having a predictive ability within the limit of precision.
105. A method of updating a calibration model to compensate for a modified range of an existing factor for predicting a property of interest, comprising: obtaining a calibration model for predicting the property of interest, the model developed from a training set using at least one instrument; identifying the modified range of the existing factor which may influence the predictive ability of the generated calibration model for the property of interest using a validation set spanning at least a portion of the modified range; calculating the RMSEP of the validation set to determine whether the new factor is influential; determining whether the modified range causes the predicted ability of the generated calibration model to fall outside a limit of precision defined in terms of RMSEP; and updating the generated calibration model to compensate for variance in the new influential factor to generate a revised calibration model having a predictive ability within the limit of precision.
106. A method of updating a calibration model to compensate for a new influential factor for predicting a property of interest, comprising: obtaining a calibration model for predicting the property of interest, the model developed from a first training set using at least one instrument; identifying a new factor which may influence the predictive ability of the generated calibration model for the property of interest using a second training set spanning at least a portion of a range of the new factor; calculating the RMSECV of the validation set to determine whether the new factor is influential; determining whether the new influential factor causes the predicted ability of the generated calibration model to fall outside a limit of precision defined in terms of RMSECV; and updating the generated calibration model to compensate for variance in the new influential factor to generate a revised calibration model having a predictive ability within the limit of precision.
107. A method of updating a calibration model to compensate for a modified range of an existing factor for predicting a property of interest, comprising: obtaining a calibration model for predicting the property of interest, the model developed from a first training set using at least one instrument; identifying the modified range of the existing factor which may influence the predictive ability of the generated calibration model for the property of interest using a second training set spanning at least a portion of the modified range; calculating the RMSECV of the validation set to determine whether the new factor is influential; determining whether the modified range causes the predicted ability of the generated calibration model to fall outside a limit of precision defined in terms of RMSECV; and updating the generated calibration model to compensate for variance in the new influential factor to generate a revised calibration model having a predictive ability within the limit of precision.
108. A method of defining at least one acceptable region of data wherein the data are generated from an instrument response prior to evaluation of the data by a calibration model to determine if pretreatment can compensate for an influential factor in predicting a property of interest using the calibration model, comprising: identifying at least one trial region containing at least one subregion of data within an entire region of data from a plurality of trial regions comprising discrete combinations of subregions; evaluating the calibration model for each identified trial region using a validation set spanning at least a portion of a range of the influential factor; calculating a RMSEP for each identified trial region; and selecting at least one acceptable region from the at least one identified trial region having the RMSEP within a limit of precision identified in terms of RMSEP.
109. A method of defining a refined filter using a validation set, comprising: identifying at least one acceptable filter for the validation set; selecting a set of at least one acceptable filter wherein each filter within the set has the lowest number of subregions; selecting a subset of the set wherein each filter within the subset has the lowest rank; and defining the refined filter as the filter within the subset which further has the lowest RMSEP calculated from the validation set.
110. A method of defining a refined filter using a validation set, comprising: identifying at least one acceptable filter for the validation set; selecting a set of at least one acceptable filter wherein each filter within the set has the lowest rank; selecting a subset of the set wherein each filter within the subset has the lowest number of subregions; and defining the refined filter as the filter within the subset which further has the lowest RMSEP calculated from the validation set.
111. A method of defining a refined filter using a validation set, comprising: identifying at least one acceptable filter for the validation set; selecting a set of at least one acceptable filter wherein each filter within the set has the lowest rank; defining the refined filter as the filter within the set which further has the lowest RMSEP calculated from the validation set.
112. A method of defining at least one acceptable region of data wherein the data are generated from an instrument response prior to evaluation of the data by a calibration model to determine if pretreatment can compensate for an influential factor in predicting a property of interest using the calibration model, comprising: identifying at least one trial region containing at least one subregion of data within an entire region of data from a plurality of trial regions comprising discrete combinations of subregions; evaluating the calibration model for each identified trial region using a training set spanning at least a portion of a range of the influential factor; calculating a RMSECV for each identified trial region; and selecting at least one acceptable region from the at least one identified trial region having the RMSECV within a limit of precision identified in terms of RMSECV.
113. A method of defining a refined filter using a training set, comprising: identifying at least one acceptable filter for the training set; selecting a set of at least one acceptable filter wherein each filter within the set has the lowest number of subregions; selecting a subset of the set wherein each filter within the subset has the lowest rank; and defining the refined filter as the filter within the subset which further has the lowest RMSECV calculated from the training set.
114. A method of defining a refined filter using a training set, comprising: identifying at least one acceptable filter for the training set; selecting a set of at least one acceptable filter wherein each filter within the set has the lowest rank; selecting a subset of the set wherein each filter within the subset has the lowest number of subregions; and defining the refined filter as the filter within the subset which further has the lowest RMSECV calculated from the training set.
115. A method of defining a refined filter using a training set, comprising: identifying at least one acceptable filter for the training set; selecting a set of at least one acceptable filter wherein each filter within the set has the lowest rank; defining the refined filter as the filter within the set which further has the lowest RMSECV calculated from the training set.
116. A method of revising a calibration model developed from a training set for a calibration model development for predicting a value of a property of interest comprising: detecting at least one probable outlier in the training set; identifying at least one good probable outlier from a group of at least one detected probable outlier; and extending the training set using the good probable outlier to develop a revised calibration model.
117. The method of claim 116 further comprising detecting at least one probable outlier using Mahalanobis distance and a first threshold value as a lower limit for identifying the probable outlier.
118. The method of claim 116 further comprising identifying at least one good probable outlier using Mahalanobis distance, a first threshold value as a lower limit for identifying the probable outlier, and a second threshold value as an upper limit for identifying the good probable outlier.
119. A method of revising a calibration model development from a training set for calibration model development for predicting a value of a property of interest comprising: detecting at least one probable outlier in a validation set; identifying at least one good probable outlier from a group of at least one detected probable outlier; and extending the training set using the good probable outlier to develop a revised calibration model.
120. The method of claim 119 further comprising detecting at least one probable outlier using Mahalanobis distance and a first threshold value as a lower limit for identifying the probable outlier.
121. The method of claim 119 further comprising identifying at least one good probable outlier using Mahalanobis distance, a first threshold value as a lower limit for identifying the probable outlier, and a second threshold value as an upper limit for identifying the good probable outlier.
122. A method of revising a calibration model developed from a training set for use in predicting a value of a property of interest on at least one unknown sample comprising: detecting at least one probable outlier in at least one predicted value from measurements on the at least one unknown sample; identifying at least one good probable outlier from a group comprising at least one detected probable outlier; and extending the training set using the good probable outlier to develop a revised calibration model.
123. The method of claim 122 further comprising detecting at least one probable outlier using Mahalanobis distance and a first threshold value as a lower limit for identifying the probable outlier.
124. The method of claim 122 further comprising identifying at least one good probable outlier using Mahalanobis distance, a first threshold value as a lower limit for identifying the probable outlier, and a second threshold value as an upper limit for identifying the good probable outlier.
125. A method of revising a calibration model developed from a training set for a calibration model development for predicting a value of a property of interest comprising: detecting at least one probable outlier in the training set; identifying at least one good probable outlier from a group of at least one detected probably outlier; and improving the training set by replacing one or more observations in the training set using the good probable outlier to develop a revised calibration model.
126. The method of claim 125 further comprising detecting at least one probable outlier using Mahalanobis distance and a first threshold value as a lower limit for identifying the probable outlier.
127. The method of claim 125 further comprising identifying at least one good probable outlier using Mahalanobis distance, a first threshold value as a lower limit for identifying the probable outlier, and a second threshold value as an upper limit for identifying the good probable outlier.
128. A method of revising a calibration model developed from a training set for calibration model development for predicting a value of a property of interest comprising: detecting at least one probable outlier in a validation set; identifying at least one good probable outlier from a group of at least one detected probable outlier; and improving the training set by replacing one or more observations in the training set using the good probable outlier to develop a revised calibration model.
129. The method of claim 128 further comprising detecting at least one probable outlier using Mahalanobis distance and a first threshold value as a lower limit for identifying the probable outlier.
130. The method of claim 128 further comprising identifying at least one good probable outlier using Mahalanobis distance, a first threshold value as a lower limit for identifying the probable outlier, and a second threshold value as an upper limit for identifying the good probable outlier.
131. A method of revising a calibration model developed from a training set for use in predicting a value of a property of interest on at least one unknown sample comprising: detecting at least one probable outlier in at least one predicted value from measurements on the at least one unknown sample; identifying at least one good probable outlier from a group comprising at least one detected probable outlier; and improving the training set by replacing one or more observations in the training set using the good probable outlier to develop a revised calibration model.
132. The method of claim 131 further comprising detecting at least one probable outlier using Mahalanobis distance and a first threshold value as a lower limit for identifying the probable outlier.
133. The method of claim 131 further comprising identifying at least one good probable outlier using Mahalanobis distance, a first threshold value as a lower limit for identifying the probable outlier, and a second threshold value as an upper limit for identifying the good probable outlier.
134. A method of evaluating an instrument for acceptability as a data acquisition device in combination with a calibration model such that a limit of precision is met by the instrument in combination with the calibration model in generating a result, comprising: generating a validation set with the instrument in combination with the calibration model; computing a RMSEP value from the validation set; and accepting the instrument if the RMSEP value is within the limit of precision.
135. A method of evaluating a component of an instrument for acceptability as a data acquisition device in combination with a calibration model such that a limit of precision is met by the instrument in combination with the calibration model in generating a result, comprising: generating a validation set with the component of the instrument in combination with the calibration model; computing a RMSEP value from the validation set; and accepting the component of the instrument if the RMSEP value is within the limit of precision.
136. A method of providing analysis services, comprising: providing analysis services on behalf of a plurality of customers using a plurality of data acquisition devices connected to a central processor into which is loaded at least one calibration model configured to generate a predicted result of a property of interest from data acquired from a plurality of samples using the data acquisition devices, wherein providing analysis services includes transmitting the predicted value of the property of interest to a customer for which analysis services is required for a particular sample of a material.
137. The method of 136 further comprising updating one or more calibration models for all of the data acquisition devices from time to time in response to detecting good outliers when processing data acquired on a plurality of samples by at least one data acquisition device.
138. The method of claim 136, further comprising receiving a fee from a customer in response to providing analysis services on behalf thereof.
139. A program product, comprising: a program configured to define at least one acceptable region of data, wherein the data are generated from an instrument response prior to evaluation of the data by a calibration model to determine if pretreatment can compensate for an influential factor in predicting a property of interest using the calibration model, by identifying at least one trial region containing at least one subregion of data within an entire region of data from a plurality of trial regions comprising discrete combinations of subregions; evaluating the calibration model for each identified trial region using a validation set spanning at least a portion of a range of the influential factor; calculating a RMSEP for each identified trial region; and selecting at least one acceptable region from the at least one identified trial region having the RMSEP within a limit of precision identified in terms of RMSEP, wherein the at least one acceptable region is selected based on a comparative evaluation of RMSEP, number of subregions, and rank of the calibration model, wherein lower values for each of RMSEP, number of regions and rank are preferred; and a signal bearing medium bearing the program.
140. The method of claim 139 wherein the at least one acceptable region is selected based on lowest RMSEP.
141. The method of claim 139 wherein the at least one acceptable region is selected based on lowest number of subregions.
142. The method of claim 139 wherein the at least one acceptable region is selected based on lowest rank of the calibration model.
143. A program product comprising: a program configured to predict a value of a property of interest of a material by using a calibration model to process data obtained from a sample of the material, wherein the calibration model is configured to compensate for instrument variance in predicting the value of the property of interest; and a signal bearing medium bearing the program.
144. The method of claim 143 wherein the calibration model is also configured to compensate for sample variance.
145. The method of claim 143 wherein the calibration model is also configured to compensate for environmental variance.
146. The method of claim 143 wherein the calibration model is also configured to compensate for sample presentation variance.
147. The method of claim 144 wherein the calibration model is also configured to compensate for environmental variance.
148. The method of claim 144 wherein the calibration model is also configured to compensate for sample presentation variance.
149. The method of claim 147 wherein the calibration model is also configured to compensate for sample presentation variance.
150. The program product of claim 143, wherein the signal bearing medium includes at least one of a transmission medium and a recordable medium.
151. A product program comprising: a program configured to generate a calibration model for predicting a value of a property of interest from data acquired on an unknown sample of material by obtaining a preliminary model for predicting the property of interest, the model developed from a training set using at least one instrument; identifying at least one factor which may influence the predictive ability of the preliminary model for the property of interest; determining the at least one factor which influences the predictive ability of the preliminary model outside a limit of defined precision; and revising the preliminary model to compensate for variation in the at least one factor which influences the property of interest to generate the calibration model, the models predicting the value of the property of interest within the limits of defined precision; and a signal bearing medium bearing the program.
152. The method of claim 151 further comprising pretreating at least a portion of data in the training set.
153. The method of claim 152 further wherein pretreating at least a portion of the training set precedes creating the preliminary model.
154. The method of claim 152 further wherein pretreating at least a portion of the training set follows creating the preliminary model.
155. The method of claim 152 wherein pretreating at least a portion of the training set comprises mathematically transforming the training set.
156. The method of claim 152 wherein pretreating at least a portion of the training set comprises filtering the training set.
157. A program product comprising: a program configured to evaluate a component of an instrument for acceptability as a data acquisition device in combination with a calibration model such that a limit of precision is met by the instrument in combination with the calibration model in generating a result, by generating a validation set with the component of the instrument in combination with the calibration model; computing a RMSEP value from the validation set; and accepting the component of the instrument if the RMSEP value is within the limit of precision; and a signal bearing medium bearing the program.
158. An apparatus, comprising: at least one microprocessor; and a program configured to execute on the at least one microprocessor to define at least one acceptable region of data, wherein the data are generated from an instrument response prior to evaluation of the data by a calibration model to determine if pretreatment can compensate for an influential factor in predicting a property of interest using the calibration model, by identifying at least one trial region containing at least one subregion of data within an entire region of data from a plurality of trial regions comprising discrete combinations of subregions; evaluating the calibration model for each identified trial region using a validation set spanning at least a portion of a range of the influential factor; calculating a RMSEP for each identified trial region; and selecting at least one acceptable region from the at least one identified trial region having the RMSEP within a limit of precision identified in terms of RMSEP, wherein the at least one acceptable region is selected based on a comparative evaluation of RMSEP, number of subregions, and rank of the calibration model, wherein lower values for each of RMSEP, number of regions and rank are preferred.
159. The apparatus of claim 158 wherein the at least one acceptable region is selected based on lowest RMSEP.
160. The apparatus of claim 158 wherein the at least one acceptable region is selected based on lowest number of subregions.
161. The apparatus of claim 158 wherein the at least one acceptable region is selected based on lowest rank of the calibration model.
162. An apparatus, comprising: at least one microprocessor; and a program configured to execute on the at least one microprocessor to predict a value of a property of interest of a material by using a calibration model to process data obtained from a sample of the material, wherein the calibration model is configured to compensate for instrument variance in predicting the value of the property of interest.
163. The apparatus of claim 162 wherein the calibration model is also configured to compensate for sample variance.
164. The apparatus of claim 162 wherein the calibration model is also configured to compensate for environmental variance.
165. The apparatus of claim 162 wherein the calibration model is also configured to compensate for sample presentation variance.
166. The apparatus of claim 163 wherein the calibration model is also configured to compensate for environmental variance.
167. The apparatus of claim 163 wherein the calibration model is also configured to compensate for sample presentation variance.
168. The apparatus of claim 166 wherein the calibration model is also configured to compensate for sample presentation variance.
169. An apparatus, comprising: at least one microprocessor; and a program configured to execute on the at least one microprocessor to generate a calibration model for predicting a value of a property of interest from data acquired on an unknown sample of material by obtaining a preliminary model for predicting the property of interest, the model developed from a training set using at least one instrument; identifying at least one factor which may influence the predictive ability of the preliminary model for the property of interest; determining the at least one factor which influences the predictive ability of the preliminary model outside a limit of defined precision; and revising the preliminary model to compensate for variation in the at least one factor which influences the property of interest to generate the calibration model, the models predicting the value of the property of interest within the limits of defined precision.
170. The apparatus of claim 169 further comprising pretreating at least a portion of data in the training set.
171. The apparatus of claim 170 further wherein pretreating at least a portion of the training set precedes creating the preliminary model.
172. The apparatus of claim 170 further wherein pretreating at least a portion of the training set follows creating the preliminary model.
173. The apparatus of claim 170 wherein pretreating at least a portion of the training set comprises mathematically transforming the training set.
174. The apparatus of claim 170 wherein pretreating at least a portion of the training set comprises filtering the training set.
175. An apparatus, comprising: at least one microprocessor; and a program configured to execute on the at least one microprocessor to evaluate a component of an instrument for acceptability as a data acquisition device in combination with a calibration model such that a limit of precision is met by the instrument in combination with the calibration model in generating a result, by generating a validation set with the component of the instrument in combination with the calibration model; computing a RMSEP value from the validation set; and accepting the component of the instrument if the RMSEP value is within the limit of precision.
176. An apparatus comprising: a memory; a calibration model resident in the memory and configured to compensate for instrument variance in predicting a value of a property of interest for a material; and a program configured to process data obtained from a sample of the material by using the calibration model, wherein the calibration model is configured to compensate for instrument variance in predicting the value of the property of interest.
177. An apparatus comprising: a memory; a calibration model resident in the memory for predicting a value of a property of interest from data acquired on an unknown sample of material; and a program configured to generate the calibration model by obtaining a preliminary model for predicting the property of interest, the model developed from a training set using at least one instrument; identifying at least one factor which may influence the predictive ability of the preliminary model for the property of interest; determining the at least one factor which influences the predictive ability of the preliminary model outside a limit of defined precision; and revising the preliminary model to compensate for variance in the at least one factor which influences the property of interest to generate the calibration model, the models predicting the value of the property of interest within the limits of defined precision.
178. An apparatus, comprising: a memory; a calibration model resident in the memory and configured for use in evaluating data; and a program configured to define at least one trial region of data wherein the data are generated from an instrument response prior to evaluation of the data by the calibration model to determine if pretreatment can compensate for an influential factor in predicting a property of interest using the calibration model by identifying at least one trial region containing at least one subregion of data within an entire region of data from a plurality of trial regions comprising discrete combinations of subregions; evaluating the calibration model for each identified trial region using a validation set spanning at least a portion of a range of the influential factor; calculating a RMSEP for each identified trial regions; and selecting at least one identified trial region used for generating a RMSEP within a limit of precision defined in terms of RMSEP, wherein the at least one trial region is selected based on a comparative evaluation of RMSEP, number of subregions, and rank of the calibration model, wherein lower values for each of RMSEP, number of subregions, and rank of the calibration model, wherein lower values for each of RMSEP, number of subregions and rank are preferred.
179. An apparatus, comprising: a memory; a calibration model resident in the memory; and a program configured to evaluate a component of an instrument for acceptability as a data acquisition devise in combination with the calibration model such that a limit of precision is met by the instrument in combination with the calibration model in generating a result, by generating a validation set with a component of the instrument in combination with the calibration model; computing a RMSEP value from the validation set; and accepting the component of the instrument if the RMSEP value is within the limit of precision.

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of two U.S. Provisional Patent Applications, Serial Nos. 60/307,347 and 60/307,348, both filed on Jul. 23, 2001, the disclosures of which are incorporated by reference herein. This application is also related to U.S. patent application Ser. No. ______, filed on even date herewith by James Thomas Kent, et al., entitled “EXTENSIBLE MODULAR COMMUNICATION EXECUTIVE WITH ACTIVE MESSAGE QUEUE AND INTELLIGENT MESSAGE PRE-VALIDATION,” the disclosure of which is incorporated by reference herein.

Provisional Applications (2)

	Number	Date	Country
	60307347	Jul 2001	US
	60307348	Jul 2001	US

On-site analysis system with central processor and method of analyzing

Information

Publication Number

Date Filed

Date Published

Inventors

CPC

US Classifications

International Classifications

Abstract

Description

Claims

CROSS-REFERENCE TO RELATED APPLICATIONS

Provisional Applications (2)