Patent Application
20120327410
Media—materials or object of various textures, purity, and colors—can be identified or sensed in a variety of ways. Humans are equipped with five primary senses to gather information about the surrounding environment. Human vision provides a basic way of detecting what is around us by the amount of light it reflects, changes the path of (refraction), or absorbs. When an object absorbs a relatively large amount of light, it appears darker than other objects, approaching black for highly absorptive media. When an object has a particular color it is absorbing more of that color band, or wavelength of light relative to other wavelengths. For example, a lime can be readily recognized from a lemon as a result of their different light absorption characteristics. Light, as detectible by the human eye, covers only a portion of a much wider spectrum of electromagnetic energy. All matter will interact with a wide range of wavelengths in the electromagnetic spectrum both inside and outside the visible light bands. This interaction occurs in energy exchanges at the quantum level. This interaction, the effect of matter and energy change in the presence of electromagnetic energy, is the essence of media identification spectroscopy.
One method of identifying materials is through the use of spectroscopy such as reflection/absorption (R/A) spectroscopy. By directing electromagnetic energy at a target and observing the reflected and absorbed energy levels the media identification can be inferred as a function of energy returned at select known wavelengths. Traditionally, spectroscopy identification methods require elaborate laboratory equipment such as precision lasers, high quality optics and filters, diffraction grating, intricate moving parts, and precision electronic devices.
In addition to measuring the returned energy of a certain transmitted and reflected wavelength, certain media are known to exhibit other properties such as fluorescence. When these effects occur, the reflected energy, which may have a wavelength other than the wavelength of the excitation source, can also be captured.
The following presents a simplified summary of the innovation in order to provide a basic understanding of some aspects of the innovation. This summary is not an extensive overview of the innovation. It is not intended to identify key/critical elements of the innovation or to delineate the scope of the innovation. Its sole purpose is to present some concepts of the innovation in a simplified form as a prelude to the more detailed description that is presented later.
The innovation disclosed and claimed herein, in one aspect thereof, comprises a non-contact media detection system. The system can have one or more electromagnetic (EM) sources that direct EM energy toward a target surface of an unknown medium and one or more EM detectors that measure EM energy reflected from the target surface of the unknown medium. Additionally, the system can have a control component that receives measurement data from the one or more EM detectors, determines a measured profile based at least in part on the measurement data, and analyzes the measured profile to determine one or more likely candidates for the medium based at least in part on the analyzed profile.
In other aspects, the innovation can include a method of non-contact media detection. The method can include the step of conducting a sequence of one or more measurement steps, wherein each measurement step comprises activating one or more electromagnetic (EM) sources to reflect an EM signal off of an unknown medium and making one or more reading of an intensity of the reflected EM signal with one or more EM detectors. Additionally, the method can include the steps of assembling the one or more readings from each measurement step into a measured profile and determining one or more likely candidates for the unknown medium based at least in part on the measured profile.
In some embodiments, the innovation can comprise a non-contact media detection system. The system can have means for reflecting an EM signal off of an unknown medium at least once for each of one or more measurement steps in a sequence. Additionally, the system can have means for making one or more reading of an intensity of the reflected EM signal once for each measurement step and means for determining one or more likely candidates for the unknown medium based at least in part on the one or more readings of the intensity.
In certain embodiments, the innovation relates to Reflection/Absoption (R/A) spectroscopy-based media identification systems, and methods related thereto. The innovation actively directs Electro-Magnetic (EM) energy toward objects using one or more EM source(s). One or more EM detectors in the system can observe this energy-media interaction and produces medium specific signals. The resulting signals are processed and interpreted to infer the identity of the medium.
Typical medium identification processes involve visual image recognition systems with sophisicated software algorithms to decern object properties, mimicking the mental thought processes of humans to contruct a comparison color or greyscale and form-factor discernable image for comparison with an expected range of items. However, the subject innovation can use one or more simple EM energy producing devices, such as LEDs of known wavelength, and one or more simple receptor devices, such as a photodiode or CCD to capture the resultant EM energy reflection.
Accordingly, the innovation can deliver an inferred result as to which media has been observed using simple technology and principals describe above. With specifically selected component wavelengths in quantities sufficient for unique discernment between possible canidate media, the system can vary in component count as dictated by an application. As an illustrative example, green apples may be distinguished from red apples by use of a relatively small number of EM sources, such as a green and red LED system. Green apples will return a lower green to red ratio when both wavelengths are transmitted and observed by the detector.
Readily available on today's market are an increasing number of light emitting diodes (LEDs). LED technology is advancing to cover an expanding spectrum of energy extending from long wavelength infrared, through the visible light spectrum and into the UV group of wavelengths. These components can be used to direct energy at targets and the medium's intrinsic reflected responses can be readily detectible with simple wide spectrum detectors, such as photo diodes or charge coupled devices (CCDs). The detected responses can be processed and cross-referenced to profiles of known media and the best possible match produced to infer the media identity.
Systems and methods of the subject innovation are capable of discerning between various media in multiple ways, based for example, on whether the media are categorized in a library of known medium profiles or whether medium identification can be inferred based on calculable and quantifiable medium characteristics and thus identified by profile response type. Furthermore, new media can be ‘learned’ by the system as the presented medium's response can be measured and recorded by the system in situ.
Systems and methods of the subject innovation can be deployed without physical contact. Additionally, the excitation source and detector can be physically housed together, as the energy passing into or through the sample is not needed. Because of this, the subject innovation can be used in situations where objects are in motion, for example, in vehicular applications, or where the media is in motion such as in a continuous process. The ability for the system to be self-contained allows it to function in small confined spaces.
In various embodiments, one or more EM detectors can be used, with potential advantages for each embodiment. Although typical simple EM or light detecting systems vary in sensitivity over various wavelength and temperatures, the use of a single detector can allow common drift cancellation and preserve the relative response profiles over the wide range of wavelength sources. Detector signal normalization or auto-gaining can also be employed so that the spectral response characteristics needed for unique media identification can be preserved. This can be beneficial when uniform levels of dirt accumulation, electrical component drift, and aging, to a first order extent, is encountered in a self-contained system.
In yet another aspect thereof, an artificial intelligence component can be provided that employs a probabilistic and/or statistical-based analysis to prognose or infer an action that a user desires to be automatically performed.
To the accomplishment of the foregoing and related ends, certain illustrative aspects of the innovation are described herein in connection with the following description and the annexed drawings. These aspects are indicative, however, of but a few of the various ways in which the principles of the innovation can be employed and the subject innovation is intended to include all such aspects and their equivalents. Other advantages and novel features of the innovation will become apparent from the following detailed description of the innovation when considered in conjunction with the drawings.
The innovation is now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the subject innovation. It may be evident, however, that the innovation can be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing the innovation.
As used in this application, the terms “component” and “system” are intended to refer to a computer-related entity, either hardware, a combination of hardware and software, software, or software in execution. For example, a component can be, but is not limited to being, a process running on a processor, a processor, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a server and the server can be a component. One or more components can reside within a process and/or thread of execution, and a component can be localized on one computer and/or distributed between two or more computers.
As used herein, the term to “infer” or “inference” refer generally to the process of reasoning about or inferring states of the system, environment, and/or user from a set of observations as captured via events and/or data. Inference can be employed to identify a specific context or action, or can generate a probability distribution over states, for example. The inference can be probabilistic—that is, the computation of a probability distribution over states of interest based on a consideration of data and events. Inference can also refer to techniques employed for composing higher-level events from a set of events and/or data. Such inference results in the construction of new events or actions from a set of observed events and/or stored event data, whether or not the events are correlated in close temporal proximity, and whether the events and data come from one or several event and data sources.
While, for purposes of simplicity of explanation, the one or more methodologies shown herein, e.g., in the form of a flow chart, are shown and described as a series of acts, it is to be understood and appreciated that the subject innovation is not limited by the order of acts, as some acts may, in accordance with the innovation, occur in a different order and/or concurrently with other acts from that shown and described herein. For example, those skilled in the art will understand and appreciate that a methodology could alternatively be represented as a series of interrelated states or events, such as in a state diagram. Moreover, not all illustrated acts may be required to implement a methodology in accordance with the innovation.
As will be described in greater detail infra, the subject innovation provides for identification of various media types using a non-contact media detection system. Aspects of the innovation can effectively excite, measure, analyze, and determine the presense of certain media, materials, surface textures, colors, etc. As will be understood, non-contact media detection sensitivity can vary by the presense of externally present ambient EM energy sources, such as bright sun light. These and other environmental factors can be accounted for in a variety of ways, such as adaptive leveling of the one or more EM detector signals during a period wherein the one or more EM source are in an off state, optionally in concert with one or more other techniques, such as variable source power or received signal profile normalization. In addition to handling variations in background EM levels the subject innovation is also effective for handling temperature or aging effects of the system components. This adaptive compensation enhances the accuracy and dynamic range of such systems.
Referring initially to
In some embodiments, discussed further herein, each EM source 104 can produce a single disparate wavelength of light. However, in other embodiments, at least one of the EM sources 104 can emit multiple wavelengths, which may or may not overlap with one or more wavelengths of another source. These EM sources 104 can be any of a variety of types of sources, e.g., light emitting diodes (LEDs), lasers, narrow or broad spectrum sources, collimated or non-collimated sources, filtered or not, etc. The one or more EM sources 104 can illuminate at least a portion of medium 102 with EM energy. EM energy interacts with the medium at a quantum level and, in general, the incident EM energy can be partly reflected by the medium, and partly absorbed or transmitted by the medium (although in some situations, none or all may be reflected, or none or all may be absorbed).
Each of the one or more EM sources 104 can be exercised independently or in selective group concert to illuminate at least a portion of the medium 102 so as to produce a detectable response that can be measured by the one or more EM detectors. The one or more EM sources can be operated in various manners, including, but not limited to, simple on/off, variable continuous excitation, and pulse modulation source activations, as well as evaluations of steady-state, peak, root-mean-square (RMS), or decay responses, as well as other manners or combinations of the foregoing.
The non-contact media detection system 100 may also include at least one EM detector 106, which can detect at least a portion of the reflected EM energy from medium 102. The at least one EM detector 106 may consist of a single wide spectrum device, several narrow band devices, or a combination of devices including both wide spectrum and narrow band devices. Examples of EM detectors that can be used are photodiodes, single-pixel cameras, charge-coupled device (CCD) cameras, etc. The at least one EM detector 106 can collect data based on measured levels of EM energy for one or more wavelengths of EM energy emanating from medium 102, generally as reflected EM energy (although other processes, such as photoluminescence, flourescence, and phosphorescence may also contribute). The one or more EM detectors 106 can send the collected data to control component 108 for acquisition and processing (e.g., by converting the collected data into an electrical signal, wirelessly, etc.).
Control component 108 can be connected to the one or more EM sources 104, the one or more EM detectors 106, or both. The control component 108 can independently control the one or more EM sources 104 in a variety of ways. For example, the one or more EM sources 104 can be operated to illuminate the medium with a plurality of wavelengths of EM energy sequentially or simultaneously, with a plurality of narrow bands of wavelengths sequentially or simultaneously, with the one or more EM sources 104 sequentially or simultaneously, etc. Additionally, the control component 108 can receive and process signals or measurement data from the one or more EM detectors 106. The control component 108 can analyze the processed signals or data to determine one or more likely candidates for the medium based at least in part on the analyzed signals or data. This determination can be based at least in part on a comparison of the processed signals to one or more profiles of known media in a media profile library 110. Optionally, the control component 108 can control the one or more EM sources 104 and the one or more EM detectors 106 to sample the optical properties of the medium multiple times before determining the one or more likely candidates. Additionally or alternatively, multiple samples can be taken on an intermittent or ongoing basis, and the one or more likely candidates can be revised based at least in part on the multiple samples taken on an intermittent or ongoing basis. In some aspects, the control component 108 can compare the analyzed signals to a library of known media such as media profile library 110 to find a best match, or one or more likely candidates. In various embodiments, media profile library 110 can be stored one or more of locally or remotely.
Systems and methods of the subject innovation can be used to determine one or more likely candidates for a medium by comparing measurements obtained to one or more known media profiles. A collection of commonly expected media profiles can be maintained in a media profile library such as library 110. The location can be maintained locally, remotely, or a combination of the two. These commonly expected media profiles can be determined externally, or in situ, and optionally can be determined ahead of time and transferred to the library, or can have media profile information communicated to it from a remote source either ahead of time or as one or more updates to an already deployed system or device of the subject innovation. Additionally, in some aspects, media profile information obtained in situ can be used to provide additional data to further improve media identification locally, remotely, or both.
In operation, the media profile information can be used in conjunction with other aspects of the subject innovation. For example, the one or more EM sources can be activated to produce a response from the medium (e.g., reflected EM energy, fluorescence, etc.) that can be detected by the one or more EM detectors. Data associated with the detected response can be acquired by the control component, and analysis (e.g., probalistic, etc.) can be executed to compute the closest media profile match. Based at least in part on the analysis, the identity of the medium can be inferred.
Although only four media profiles are shown in
Identification of a medium (or likely candidates for the medium) can occur based on a comparison of measurements of the medium to one or more media profiles in a library. Thus, in aspects, profiles of media likely to be encountered can be compiled in a library such as library 110 before encountering the media.
The one or more EM detectors 106 can be set to a baseline level by sensing a baseline reference measurement. This baseline can be observed with all of the one or more EM sources 104 off, and can be obtained with a relatively low ambient EM energy level (low light condition). In other aspects, the baseline can be recalibrated at intervals to correspond to a current ambient EM energy level.
In aspects, the media profiles in media profile library 110 can be obtained by operation of a system or device of the subject innovation. In one example method of learning a medium profile (or, alternatively, identifying an unknown medium), the one or more EM sources 104 can be sequenced, with or without variable amplitude modulation, to produce a response from the medium that is measured by the one or more EM detectors 106. The activations of the one or more EM sources 104 and the corresponding responses measured by the one or more EM detectors 106 can be analyzed by the control component 108. If the sequence is being performed for training to learn a profile of a medium, then the analysis results can be stored as or added to a profile for the medium in the library 110. If an unknown medium is being identified, the results of the analysis can be compared to known profiles in library 110 to determine one or more likely candidates for the medium.
Continued operation of the non-contact media detection system can begin at step 708. At step 710, one or more ‘dark’ reference baseline readings can be made by each of the one or more EM detectors. The one or more EM detectors can periodically record a ‘dark’ measurement by recording a measurement with all of the one or more EM sources off. These ‘dark’ measurements can be used to form one or more baseline reference point. In aspects, a ‘dark’ reference baseline reading can be made with varying frequency, such as between each sequence, more than once per sequence, or less than once for each sequence, such as once every several sequences, or after specific intervals. Both an ambient reference point and a noise floor can thus be captured, allowing for the ability to adapt to various operating conditions through periodic updates via ‘dark’ measurements. In addition, a power level of the one or more EM source power may be modulated in response to the sensed bias level and noise floor to elevate one or more dark to excited state signal ratios. Similarly, the one or more EM source power levels may be adjusted as needed to prevent saturation of the one or more EM detectors. Depending on the situation, either or both of modulating or adjusting the power level or levels may be used to maximize the response signal quality for variable conditions. In other words, each of the one or more EM sources may be deterministically adjusted to suit a given media response in variable operating environments, as explained herein.
Continuing the discussion of
In certain aspects, a sequence can include multiple repeated measurement steps before being completed. For example, a set of measurements can be taken at one or more wavelengths, and then the set of measurements can be repeated one or more times before proceeding with further steps of method 700. In some situations, a sequence with a repeated set of measurements can improve the accuracy of media identification.
At step 714, for each measurement step in the sequence, the one or more EM detector(s) can make one or more readings of the intensity of the signal reflected from the medium. At step 716, a determination can be made as to whether the sequence is completed, or whether there are more measurement steps in the sequence. If there are more measurement steps, method 700 can return to step 710 for an optional ‘dark’ reference baseline reading, or can proceed directly to step 712 to perform the next measurement step in the sequence, and the corresponding one or more readings at step 714. If the sequence is completed, the method can proceed to step 718, where the results can be assembled into a measured profile, and optionally normalized. The optional normalizing can be based at least in part on the one or more ‘dark’ reference baseline readings made during method 700.
At step 720, the measured profile can be compared to one or more library profiles in a media profile library. This comparison can include calculating one or more measures of fitness (e.g., statistical or probabilistic measures such as a least squares method, etc.) to determine one or more qualities of fitness between the measured profile and the one or more library profiles. Based on this comparison, one or more likely candidates for the medium can be determined. Optionally, if the likelihood of the two or more most likely candidates is close enough to one another (e.g., within some pre-determined threshold, etc.), then the sequence of measurement steps can be repeated to obtain additional measurements before proceeding. Continuing at step 722, the results of the comparison can be output in any of a number of manners. For example, the one or more most likely candidates can be output, or all candidates can be output. After outputting results, the method can optionally return (either immediately, or after some period of time) to step 710 for an optional ‘dark’ reference baseline reading, and then to step 712 to begin a new sequence of measurement steps.
Optionally, a measure of likelihood or confidence associated with one or more candidates can be output along with the one or more candidates. Identification system errors may occur, and systems and devices of the subject innovation can declare relative confidence in the ability to identify candidate media. For example, a system of the subject innovation can be mounted on a vehicle driving on a road surface where candidate media profiles for concrete, blacktop, snow, and ice are preselected choices that the unknown medium will be compared against. As snow conditions increase the medium indication may progress from blacktop to snow in variable degrees. The result may be presented in a variety of ways, such as blacktop, ice, a most likely candidate along with an associated confidence or likelihood measure, a probability of being one or more media (e.g., 40% probability of being blacktop, 40% probabilty of being snow, 15% being ice, and 5% being concrete, etc.), etc. Furthermore, should the surface become unknown, it may be reported as such.
In further aspects, the subject innovation can include diagnostics to ensure proper functioning. As a measure of self diagnosis, provisions for proper function can be stated by executing one or more test sequences of activations of the one or more EM sources and corresponding measurements of the one or more EM detectors to determine if the responses meet qualifying thresholds. In the presence of failed components, excess obstruction from dirt or physical damage, or certain calibration media templates, the system may make available diagnostic responses. The system can account for manual or self-corrective actions such as calling for a cleaning procedure or autonomous self-recalibration.
Additionally, by employing the use of external support equipment, such as a computer or specially developed calibration fixtures, the non-contact media detection system can be re-trained (e.g., to learn new media types, etc.), reprogrammed, serviced, maintained, recertified, etc. In various aspects, such support and similar activities can be performed on site, or remotely, for example, by using any of a number of wireless communications technologies in connection with the subject innovation for re-training, reprogramming, providing an alert or notification that service or maintenance is needed, etc.
In some aspects, such as the road treatment aspects discussed herein, the detection and identification system may be used in conjunction with road treatment materials to both determine when treatment materials should be used as well as when a road has already been treated, for example via inclusion into road treatment materials of certain add-in materials such as tracers, catalytic agents, aids, etc. For example, in an ice treatment application, the addition of one or more tracer agents (e.g., UV tracer) may be added such that the level of pre-existing ice melting agents can be more readily recognized, thus allowing for the conservation of additional treatment agents. In various aspects, systems and methods of the subject innovation can act in conjunction with systems that disperse road treatment materials (e.g., by sending instructions or other data) such that material is dispersed when certain media are detected (e.g., ice, snow, etc.), unless other media such as add-in materials like tracers, catalytic agents, aids, etc. are detected.
Furthermore, the invention can be used in concert with other sensing systems, such as sensors to determine an air or road temperature, for example an infrared temperature monitor to further qualify the media identification.
In aspects, systems and methods of the subject innovation can be used in conjunction with wireless communications techniques. For example, reprogramming or updates to a media profile library can occur remotely from a source of the reprogramming or update, and can occur while the system is deployed and operational. In other aspects, information collected by embodiments of the subject innovation can be combined. This information can be used in a variety of ways. For example, in a vehicle-mounted scenario, it could be used to build a map of road media, including road conditions (e.g., water, ice, snow, etc.), or could be used to identify roads or portions thereof that need treatment, that already have been treated, or both. In one example, this information could be used by an organization to coordinate multiple vehicles to efficiently apply treatment materials to roads where needed while minimizing effort and materials.
The subject innovation (e.g., in connection with media identification and learning new media profiles) can employ various AI-based schemes for carrying out various aspects thereof. For example, a process for learning or updating one or more media profiles can be facilitated via an automatic classifier system and process. Moreover, where the subject innovation is used to determine an unknown medium based on comparison with a library of known media profiles, the classifier can be employed to determine which profile from the media profile library best corresponds to the unknown medium.
A classifier is a function that maps an input attribute vector, x=(x1, x2, x3, x4, xn), to a confidence that the input belongs to a class, that is, f(x)=confidence(class). Such classification can employ a probabilistic and/or statistical-based analysis (e.g., factoring into the analysis utilities and costs) to prognose or infer an action that a user desires to be automatically performed. In the case of media identification, for example, attributes can be measured data corresponding to an unknown medium or other data-specific attributes derived from the measured data (e.g., a measured or adjusted profile), and the classes can be categories or areas of interest (e.g., library profiles that may correspond to the unknown medium).
A support vector machine (SVM) is an example of a classifier that can be employed. The SVM operates by finding a hypersurface in the space of possible inputs, which the hypersurface attempts to split the triggering criteria from the non-triggering events. Intuitively, this makes the classification correct for testing data that is near, but not identical to training data. Other directed and undirected model classification approaches include, e.g., naïve Bayes, Bayesian networks, decision trees, neural networks, fuzzy logic models, and probabilistic classification models providing different patterns of independence can be employed. Classification as used herein also is inclusive of statistical regression that is utilized to develop models of priority.
As will be readily appreciated from the subject specification, the subject innovation can employ classifiers that are explicitly trained (e.g., via a generic training data) as well as implicitly trained (e.g., via observing user behavior, receiving extrinsic information). For example, SVM's are configured via a learning or training phase within a classifier constructor and feature selection module. Thus, the classifier(s) can be used to automatically learn and perform a number of functions, including but not limited to determining according to predetermined criteria determining sets of most likely media candidates, determining associated likelihoods, determining an adjustment to be applied to the profile of an unknown medium, etc.
What has been described above includes examples of the innovation. It is, of course, not possible to describe every conceivable combination of components or methodologies for purposes of describing the subject innovation, but one of ordinary skill in the art may recognize that many further combinations and permutations of the innovation are possible. Accordingly, the innovation is intended to embrace all such alterations, modifications and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the term “includes” is used in either the detailed description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
