Patent Grant
12017255
The present disclosure relates in general to the sorting of materials, and in particular, to the sorting of materials to achieve a specific composition of chemical elements within the sorted materials.
Recycling is the process of collecting and processing materials that would otherwise be thrown away as trash, and turning them into new products. Recycling has benefits for communities and for the environment, since it reduces the amount of waste sent to landfills and incinerators, conserves natural resources, increases economic security by tapping a domestic source of materials, prevents pollution by reducing the need to collect new raw materials, and saves energy. After collection, recyclables are generally sent to a material recovery facility to be sorted, cleaned, and processed into materials that can be used in manufacturing.
Various detailed embodiments of the present disclosure are disclosed herein. However, it is to be understood that the disclosed embodiments are merely exemplary of the disclosure, which may be embodied in various and alternative forms. The figures are not necessarily to scale; some features may be exaggerated or minimized to show details of particular components. Therefore, specific structural and functional details disclosed herein are not to be interpreted as limiting, but merely as a representative basis for teaching one skilled in the art to employ various embodiments of the present disclosure.
As used herein, “chemical element” means a chemical element of the periodic table of chemical elements, including chemical elements that may be discovered after the filing date of this application. As used herein, a “material” may include a solid composed of a compound or mixture of one or more chemical elements, wherein the complexity of a compound or mixture may range from being simple to complex (all of which may also be referred to herein as a material having a specific “chemical composition”).
As used herein, an “aggregate chemical composition” means the composition of chemical elements and their relative percentages by weight (wt %) within a collection or group of individual, separate material pieces. (Note that the percentage by weight (or weight percentage) is also referred to as the mass fraction, which is the percentage of the mass of a specific chemical element within a material or substance to the total mass of the material or substance.) For example, if a collection of individual pieces of metal alloys were melted together, the resultant “melt” would possess a chemical composition equivalent to the aggregate chemical composition. As referenced herein, a “melt” is when selected material pieces are melted together, and a composition analysis is performed on the melted together material pieces to determine the percentages (e.g., percentages by weight) of the various chemical elements existing within the melt.
Classes of materials may include metals (ferrous and nonferrous), metal alloys, plastics (including, but not limited to, PCB, HDPE, UHMWPE, and various colored plastics), rubber, foam, glass (including, but not limited to, borosilicate or soda lime glass, and various colored glass), ceramics, paper, cardboard, Teflon, PE, bundled wires, insulation covered wires, rare earth elements, leaves, wood, plants, parts of plants, textiles, bio-waste, packaging, electronic waste, batteries, accumulators, scrap pieces from end-of-life vehicles, mining, construction, and demolition waste, crop wastes, forest residues, purpose-grown grasses, woody energy crops, microalgae, urban food waste, food waste, hazardous chemical and biomedical wastes, construction debris, farm wastes, biogenic items, non-biogenic items, objects with a specific carbon content, any other objects that may be found within municipal solid waste, and any other objects, items, or materials disclosed herein, including further types or classes of any of the foregoing that can be distinguished from each other, including but not limited to, by one or more sensor systems, including but not limited to, any of the sensor technologies disclosed herein. Within this disclosure, the terms “scrap,” “scrap pieces,” “materials,” “material pieces,” and “pieces” may be used interchangeably. As used herein, a material piece or scrap piece referred to as having a metal alloy composition is a metal alloy having a specific chemical composition that distinguishes it from other metal alloys.
As well known in the industry, a “polymer” is a substance or material composed of very large molecules, or macromolecules, composed of many repeating subunits. A polymer may be a natural polymer found in nature or a synthetic polymer.
“Multilayer polymer films” are composed of two or more different compositions and may possess a thickness of up to about 7.5′×10−4 m. The layers are at least partially contiguous and preferably, but optionally, coextensive.
As used herein, the terms “plastic,” “plastic piece,” and “piece of plastic material” (all of which may be used interchangeably) refer to any object that includes or is composed of a polymer composition of one or more polymers and/or multilayer polymer films.
As used herein, the term “chemical signature” refers to a unique pattern (e.g., fingerprint spectrum), as would be produced by one or more analytical instruments, indicating the presence of one or more specific elements or molecules (including polymers) in a sample. The elements or molecules may be organic and/or inorganic. Such analytical instruments include any of the sensor systems disclosed herein. In accordance with embodiments of the present disclosure, one or more sensor systems disclosed herein may be configured to produce a chemical signature of a material piece (e.g., a plastic piece).
As used here in, a “fraction” refers to any specified combination of organic and/or inorganic elements or molecules, polymer types, plastic types, polymer compositions, chemical signatures of plastics, physical characteristics of the plastic piece (e.g., color, transparency, strength, melting point, density, shape, size, manufacturing type, uniformity, reaction to stimuli, etc.), etc., including any and all of the various classifications and types of plastics disclosed herein. Non-limiting examples of fractions are one or more different types of plastic pieces that contain: LDPE plus a relatively high percentage of aluminum; LDPE and PP plus a relatively low percentage of iron; PP plus zinc; combinations of PE, PET, and HDPE; any type of red-colored LDPE plastic pieces; any combination of plastic pieces excluding PVC; black-colored plastic pieces; combinations of #3-#7 type plastics that contain a specified combination of organic and inorganic molecules; combinations of one or more different types of multi-layer polymer films; combinations of specified plastics that do not contain a specified contaminant or additive; any types of plastics with a melting point greater than a specified threshold; any thermoset plastic of a plurality of specified types; specified plastics that do not contain chlorine; combinations of plastics having similar densities; combinations of plastics having similar polarities; plastic bottles without attached caps or vice versa.
“Catalytic pyrolysis” involves the degradation of the polymeric materials by heating them in the absence of oxygen and in the presence of a catalyst.
The term “predetermined” refers to something that has been established or decided in advance.
“Spectral imaging” is imaging that uses multiple bands across the electromagnetic spectrum. While an ordinary camera captures light across three wavelength bands in the visible spectrum, red, green, and blue (“RGB”), spectral imaging encompasses a wide variety of techniques that include but go beyond RGB.
Spectral imaging may use the infrared, visible, ultraviolet, and/or x-ray spectrums, or some combination of the above. Spectral data, or spectral image data, is a digital data representation of a spectral image. Spectral imaging may include the acquisition of spectral data in visible and non-visible bands simultaneously, illumination from outside the visible range, or the use of optical filters to capture a specific spectral range. It is also possible to capture hundreds of wavelength bands for each pixel in a spectral image.
As used herein, the term “image data packet” refers to a packet of digital data pertaining to a captured spectral image of an individual material piece.
As used herein, the terms “classify,” “identify,” “select,” and “recognize” and the terms “classification,” “identification,” “selection,” and “recognition” and any derivatives of the foregoing, may be utilized interchangeably. As used herein, to “classify” a material piece is to determine (i.e., identify) a type or class of materials to which the material piece belongs (or at least should belong according to sensed characteristics of that material piece). For example, in accordance with certain embodiments of the present disclosure, a sensor system (as further described herein) may be configured to collect and analyze any type of information for classifying materials, which classifications can be utilized within a sorting system to selectively sort material pieces as a function of a set of one or more sensed physical and/or chemical characteristics (e.g., which may be user-defined), including but not limited to, color, texture, hue, shape, brightness, weight, density, composition, size, uniformity, manufacturing type, chemical signature, predetermined fraction, radioactive signature, transmissivity to light, sound, or other signals, and reaction to stimuli such as various fields, including emitted and/or reflected electromagnetic radiation (“EM”) of the material pieces. As used herein, “manufacturing type” refers to the type of manufacturing process by which the material piece was manufactured, such as a metal part having been formed by a wrought process, having been cast (including, but not limited to, expendable mold casting, permanent mold casting, and powder metallurgy), having been forged, a material removal process, etc.
The types or classes (i.e., classification) of materials may be user-definable and not limited to any known classification of materials. The granularity of the types or classes may range from very coarse to very fine. For example, the types or classes may include plastics, ceramics, glasses, metals, and other materials, where the granularity of such types or classes is relatively coarse; different metals and metal alloys such as, for example, zinc, copper, brass, chrome plate, and aluminum, where the granularity of such types or classes is finer; or between specific subclasses of metal alloys, where the granularity of such types or classes is relatively fine. Thus, the types or classes may be configured to distinguish between materials of significantly different compositions such as, for example, plastics and metal alloys, or to distinguish between materials of substantially similar or almost identical chemical composition such as, for example, different subclasses of metal alloys. It should be appreciated that the methods and systems discussed herein may be applied to identify/classify pieces of material for which the chemical composition is completely unknown before being classified.
As referred to herein, a “conveyor system” may be any known piece of mechanical handling equipment that moves materials from one location to another, including, but not limited to, an aero-mechanical conveyor, automotive conveyor, belt conveyor, belt-driven live roller conveyor, bucket conveyor, chain conveyor, chain-driven live roller conveyor, drag conveyor, dust-proof conveyor, electric track vehicle system, flexible conveyor, gravity conveyor, gravity skatewheel conveyor, lineshaft roller conveyor, motorized-drive roller conveyor, overhead I-beam conveyor, overland conveyor, pharmaceutical conveyor, plastic belt conveyor, pneumatic conveyor, screw or auger conveyor, spiral conveyor, tubular gallery conveyor, vertical conveyor, vibrating conveyor, and wire mesh conveyor.
The systems and methods described herein according to certain embodiments of the present disclosure receive a mixture of a plurality of material pieces, wherein at least one material piece within this mixture includes a chemical composition (e.g., a metal alloy composition, a chemical signature) different from one or more other material pieces, and/or at least one material piece within this mixture was manufactured differently from one or more other materials, and/or at least one material piece within this mixture is distinguishable (e.g., visually discernible characteristics or features, different chemical signatures, etc.) from other material pieces, and the systems and methods are configured to accordingly identify/classify/sort this material piece. Embodiments of the present disclosure may be utilized to sort any types or classes of materials, or fractions, as defined herein.
It should be noted that the material pieces to be sorted may have irregular sizes and shapes (e.g., see
Embodiments of the present disclosure will be described herein as sorting material pieces into such separate groups or collections by physically depositing (e.g., diverting or ejecting) the material pieces into separate receptacles or receptacles, or onto another conveyor system, as a function of user-defined groupings or collections (e.g., a predetermined specific aggregate chemical composition, specific material type classifications or fractions). As an example, within certain embodiments of the present disclosure, material pieces may be sorted into separate receptacles or receptacles in order to separate material pieces composed of a specific chemical composition, or compositions, from other material pieces composed of a different specific chemical composition in order to produce a predetermined specific aggregate chemical composition within the collection or group of sorted material pieces. In a non-limiting example, a collection of Twitch that includes various aluminum alloys (e.g., various different wrought and/or cast aluminum alloys), may be sorted in accordance with embodiments of the present disclosure in order to produce an aluminum alloy having a desired chemical composition (which may include an aluminum alloy having a unique chemical composition different from known aluminum alloys).
Furthermore, though the simplified illustration in
In accordance with certain embodiments of the present disclosure, some sort of suitable feeder mechanism (e.g., another conveyor system or hopper 102) may be utilized to feed the material pieces 101 onto the conveyor system 103, whereby the conveyor system 103 conveys the material pieces 101 past various components within the system 100. After the material pieces 101 are received by the conveyor system 103, an optional tumbler/vibrator/singulator 106 may be utilized to separate the individual material pieces from a combined mass of material pieces. Within certain embodiments of the present disclosure, the conveyor system 103 is operated to travel at a predetermined speed by a conveyor system motor 104. This predetermined speed may be programmable and/or adjustable by the operator in any well-known manner Monitoring of the predetermined speed of the conveyor system 103 may alternatively be performed with a position detector 105. Within certain embodiments of the present disclosure, control of the conveyor system motor 104 and/or the position detector 105 may be performed by an automation control system 108. Such an automation control system 108 may be operated under the control of a computer system 107 and/or the functions for performing the automation control may be implemented in software within the computer system 107.
Thus, as will be further described herein, through the utilization of the controls to the conveyor belt drive motor 104 and/or the automation control system 108 (and alternatively including the position detector 105), as each of the material pieces 101 travelling on the conveyor belt 103 are identified, they can be tracked by location and time (relative to the various components of the system 100) so that various components of the system 100 can be activated/deactivated as each material piece 101 passes within their vicinity. As a result, the automation control system 108 is able to track the location of each of the material pieces 101 while they travel along the conveyor belt 103.
In accordance with certain embodiments of the present disclosure, after the material pieces 101 are received by the conveyor belt 103, a tumbler and/or a vibrator may be utilized to separate the individual material pieces from a mass (e.g., a physical pile) of material pieces. In accordance with alternative embodiments of the present disclosure, the material pieces may be positioned into one or more singulated (i.e., single file) streams, which may be performed by an active or passive singulator 106. An example of a passive singulator is further described in U.S. Pat. No. 10,207,296. As previously discussed, incorporation or use of a singulator is not required. Instead, the conveyor system (e.g., the conveyor belt 103) may simply convey a collection of material pieces, which have been deposited onto the conveyor belt 103 in a random manner
Referring again to
The vision system 110 may be further, or alternatively, configured to perform certain types of identification (e.g., classification) of all or a portion of the material pieces 101, as will be further described herein. For example, such a vision system 110 may be utilized to capture or acquire information about each of the material pieces 101. For example, the vision system 110 may be configured (e.g., with a machine learning system) to capture or collect any type of information from the material pieces that can be utilized within the system 100 to classify and/or selectively sort the material pieces 101 as a function of a set of one or more characteristics (e.g., physical and/or chemical and/or radioactive, etc.) as described herein. In accordance with certain embodiments of the present disclosure, the vision system 110 may capture visual images of each of the material pieces 101 (including one-dimensional, two-dimensional, three-dimensional, or holographic imaging), for example, by using an optical sensor as utilized in typical digital cameras and video equipment. Such visual images captured by the optical sensor are then stored in a memory device as image data (e.g., formatted as image data packets). In accordance with certain embodiments of the present disclosure, such image data may represent images captured within optical wavelengths of light (i.e., the wavelengths of light that are observable by the typical human eye). However, alternative embodiments of the present disclosure may utilize sensor systems that are configured to capture an image of a material made up of wavelengths of light outside of the visual wavelengths of the human eye. All such images may also be referred to herein as spectral images.
In accordance with certain embodiments of the present disclosure, the system 100 may be implemented with one or more sensor systems 120, which may be utilized solely or in combination with the vision system 110 to classify/identify material pieces 101. A sensor system 120 may be configured with any type of sensor technology, including sensor systems utilizing irradiated or reflected electromagnetic radiation (e.g., utilizing infrared (“IR”), Fourier Transform IR (“FTIR”), Forward-looking Infrared (“FLIR”), Very Near Infrared (“VNIR”), Near Infrared (“NIR”), Short Wavelength Infrared (“SWIR”), Long Wavelength Infrared (“LWIR”), Medium Wavelength Infrared (“MWIR” or “MIR”), X-Ray Transmission (“XRT”), Gamma Ray, Ultraviolet (“UV”), X-Ray Fluorescence (“XRF”), Laser Induced Breakdown Spectroscopy (“LIBS”), Raman Spectroscopy, Anti-stokes Raman Spectroscopy, Gamma Spectroscopy, Hyperspectral Spectroscopy (e.g., any range beyond visible wavelengths), Acoustic Spectroscopy, NMR Spectroscopy, Microwave Spectroscopy, Terahertz Spectroscopy, and including one-dimensional, two-dimensional, three-dimensional, or holographic imaging with any of the foregoing), or by any other type of sensor technology, including but not limited to, chemical or radioactive. Implementation of an exemplary XRF system (e.g., for use as a sensor system 120 herein) is further described in U.S. Pat. No. 10,207,296.
It should be noted that though
Within certain embodiments of the present disclosure, the material tracking and measuring device 111 and accompanying control system 112 may be utilized and configured to measure the sizes and/or shapes of each of the material pieces 101 as they pass within proximity of the material tracking and measuring device 111, which may be utilized by the system 100 to determine the approximate masses of each of the material pieces, along with the position (i.e., location and timing) of each of the material pieces 101 on the moving conveyor system 103. Alternatively, the vision system 110 may be utilized to track the position (i.e., location and timing) of each of the material pieces 101 as they are transported by the conveyor system 103.
A non-limiting, exemplary operation of such a material tracking and measuring device 111 and control system 112 is described herein with respect to
Referring next to
The system and process 500 will continue (i.e., repeat process blocks 502-503) to measure the current distance as long as this threshold distance value has not been reached. Once a measured height greater than the threshold value has been detected, the process will proceed to process block 504 to record that a material piece 101 passing within proximity of the material tracking and measuring device has been detected on the conveyor system. Thereafter, in process block 505, the variable n may be incremented to indicate to the system 100 that another material piece 101 has been detected on the conveyor system. This variable n may be utilized in assisting with tracking of each of the material pieces 101. In process block 506, a time stamp is recorded for the detected material piece 101, which may be utilized by the system 100 to track the specific location and timing of a detected material piece 101 as it travels on the conveyor system, while also representing a length of the detected material piece 101. In optional process block 507, this recorded time stamp may then be utilized for determining when to activate (start) and deactivate (stop) the acquisition of a sensor-initiated measurement signal (e.g., an x-ray fluorescence spectrum from a material piece 101) associated with the time stamp. The start and stop times of the time stamp may correspond to the aforementioned pulse signal produced by the material tracking and measuring device. In process block 508, this time stamp along with the recorded height of the material piece 101 may be recorded within a table utilized by the system 100 to keep track of each of the material pieces 101 and their resultant classification.
Thereafter, in optional process block 509, signals may then be sent to the sensor system indicating the time period in which to activate/deactivate the acquisition of a sensor-initiated measurement signal from the material piece 101, which may include the start and stop times corresponding to the length of the material piece 101 determined by the material tracking and measuring device. Embodiments of the present disclosure are able to accomplish such a task because of the time stamp and known predetermined speed of the conveyor system received from the material tracking and measuring device indicating when a leading edge of the material piece 101 will pass by the irradiating source, and when the trailing edge of the material piece 101 will thereafter pass by the irradiating source.
The system and process 500 for distance measuring of each of the material pieces 101 travelling along the conveyor system may then be repeated for each passing material piece 101.
Within certain embodiments of the present disclosure that implement one or more sensor systems 120, the one or more sensor systems 120 may be configured to assist the vision system 110 to identify the chemical composition, relative chemical compositions, and/or manufacturing types of each of the material pieces 101 as they pass within proximity of the one or more sensor systems 120. The one or more sensor systems 120 may include an energy emitting source 121, which may be powered by a power supply 122, for example, in order to stimulate a response from each of the material pieces 101.
In accordance with certain embodiments of the present disclosure that implement an XRF system as a sensor system 120, the source 121 may include an in-line x-ray fluorescence (“IL-XRF”) tube, such as further described within U.S. Pat. No. 10,207,296. Such an IL-XRF tube may include a separate x-ray source each dedicated for one or more streams (e.g., singulated) of conveyed material pieces. In such a case, the one or more detectors 124 may be implemented as XRF detectors to detect fluoresced x-rays from material pieces 101 within each of the singulated streams.
Within certain embodiments of the present disclosure, as each material piece 101 passes within proximity to the emitting source 121, a sensor system 120 may emit an appropriate sensing signal towards the material piece 101. One or more detectors 124 may be positioned and configured to sense/detect one or more characteristics from the material piece 101 in a form appropriate for the type of utilized sensor technology. The one or more detectors 124 and the associated detector electronics 125 capture these received sensed characteristics to perform signal processing thereon and produce digitized information representing the sensed characteristics (e.g., spectral data), which is then analyzed in accordance with certain embodiments of the present disclosure, which may be used in order to classify (solely or in combination with the vision system 110) each of the material pieces 101. This classification, which may be performed within the computer system 107, may then be utilized by the automation control system 108 to activate one of the N (N>1) sorting devices 126 . . . 129 of a sorting apparatus for sorting (e.g., diverting/ejecting) the material pieces 101 into one or more N (N>1) sorting receptacles 136 . . . 139 according to the determined classifications. Four sorting devices 126 . . . 129 and four sorting receptacles 136 . . . 139 associated with the sorting devices are illustrated in
The sorting apparatus may include any well-known mechanisms for redirecting selected material pieces 101 towards a desired location, including, but not limited to, diverting the material pieces 101 from the conveyor belt system into a plurality of sorting receptacles. For example, a sorting apparatus may utilize air jets, with each of the air jets assigned to one or more of the classifications. When one of the air jets (e.g., 127) receives a signal from the automation control system 108, that air jet emits a stream of air that causes a material piece 101 to be diverted/ejected from the conveyor system 103 into a sorting bin (e.g., 137) corresponding to that air jet.
Other mechanisms may be used to divert/eject the material pieces, such as robotically removing the material pieces from the conveyor belt, pushing the material pieces from the conveyor belt (e.g., with paint brush type plungers), causing an opening (e.g., a trap door) in the conveyor system 103 from which a material piece may drop, or using air jets to divert the material pieces into separate receptacles as they fall from the edge of the conveyor belt. A pusher device, as that term is used herein, may refer to any form of device which may be activated to dynamically displace an object on or from a conveyor system/device, employing pneumatic, mechanical, or other means to do so, such as any appropriate type of mechanical pushing mechanism (e.g., an ACME screw drive), pneumatic pushing mechanism, or air jet pushing mechanism. Some embodiments may include multiple pusher devices located at different locations and/or with different diversion path orientations along the path of the conveyor system. In various different implementations, these sorting systems describe herein may determine which pusher device to activate (if any) depending on classifications of material pieces performed by the machine learning system. Moreover, the determination of which pusher device to activate may be based on the detected presence and/or characteristics of other objects that may also be within the diversion path of a pusher device concurrently with a target item (e.g., a classified material piece). Furthermore, even for facilities where singulation along the conveyor system is not perfect, the disclosed sorting systems can recognize when multiple objects are not well singulated, and dynamically select from a plurality of pusher devices which should be activated based on which pusher device provides the best diversion path for potentially separating objects within close proximity In some embodiments, objects identified as target objects may represent material that should be diverted off of the conveyor system. In other embodiments, objects identified as target objects represent material that should be allowed to remain on the conveyor system so that non-target materials are instead diverted.
In addition to the N sorting receptacles 136 . . . 139 into which material pieces 101 are diverted/ejected, the system 100 may also include a receptacle 140 that receives material pieces 101 not diverted/ejected from the conveyor system 103 into any of the aforementioned sorting receptacles 136 . . . 139. For example, a material piece 101 may not be diverted/ejected from the conveyor system 103 into one of the N sorting receptacles 136 . . . 139 when the classification of the material piece 101 is not determined (or simply because the sorting devices failed to adequately divert/eject a piece), when the material piece 101 contains a contaminant detected by the vision system 110 and/or the sensor system 120, or because the material piece 101 is not required to produce a particular aggregate chemical composition.
Alternatively, the receptacle 140 may be used to receive one or more classifications of material pieces that have deliberately not been assigned to any of the N sorting receptacles 136 . . . 139. These such material pieces may then be further sorted in accordance with other characteristics and/or by another sorting system.
Depending upon the specific requirements of the predetermined specific aggregate chemical composition, multiple classifications may be mapped to a single sorting device and associated receptacle.
In other words, there need not be a one-to-one correlation between classifications and receptacles. For example, it may be desired by the user to sort certain classifications of materials into the same receptacle in order to achieve a particular aggregate chemical composition. To accomplish this sort, when a material piece 101 is classified as meeting one or more requirements for achieving the particular aggregate chemical composition, the same sorting device may be activated to sort these into the same receptacle. Such combination sorting may be applied to produce any desired combination of sorted material pieces (e.g., one or more particular aggregate chemical compositions). The mapping of classifications may be programmed by the user (e.g., using the sorting algorithm (e.g., see
Within certain embodiments of the present disclosure, the conveyor system 103 may be divided into multiple belts configured in series such as, for example, two belts, where a first belt conveys the material pieces past the vision system 110 and/or an implemented sensor systems(s) 120, and a second belt conveys the certain sorted material pieces past an implemented sensor system 120 for a subsequent sort. Moreover, such a second conveyor belt may be at a lower height than the first conveyor belt, such that the material pieces fall from the first belt onto the second belt.
Within certain embodiments of the present disclosure that implement a sensor system 120, the emitting source 121 may be located above the detection area (i.e., above the conveyor system 103); however, certain embodiments of the present disclosure may locate the emitting source 121 and/or detectors 124 in other positions that still produce acceptable sensed/detected physical characteristics.
It should be appreciated that, although the systems and methods described herein are described primarily in relation to classifying material pieces in solid state, the disclosure is not so limited. The systems and methods described herein may be applied to classifying a material having any of a range of physical states, including, but not limited to a liquid, molten, gaseous, or powdered solid state, another state, and any suitable combination thereof.
Regardless of the type(s) of sensed characteristics/information captured of the material pieces, the information may then be sent to a computer system (e.g., computer system 107) to be processed by a machine learning system in order to identify and/or classify each of the material pieces. Such a machine learning system may implement any well-known machine learning system, including one that implements a neural network (e.g., artificial neural network, deep neural network, convolutional neural network, recurrent neural network, autoencoders, reinforcement learning, etc.), supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, self learning, feature learning, sparse dictionary learning, anomaly detection, robot learning, association rule learning, fuzzy logic, artificial intelligence (“AI”), deep learning algorithms, deep structured learning hierarchical learning algorithms, support vector machine (“SVM”) (e.g., linear SVM, nonlinear SVM, SVM regression, etc.), decision tree learning (e.g., classification and regression tree (“CART”), ensemble methods (e.g., ensemble learning, Random Forests, Bagging and Pasting, Patches and Subspaces, Boosting, Stacking, etc.), dimensionality reduction (e.g., Projection, Manifold Learning, Principal Components Analysis, etc.) and/or deep machine learning algorithms, such as those described in and publicly available at the deeplearning.net website (including all software, publications, and hyperlinks to available software referenced within this website), which is hereby incorporated by reference herein. Non-limiting examples of publicly available machine learning software and libraries that could be utilized within embodiments of the present disclosure include Python, OpenCV, Inception, Theano, Torch, PyTorch, Pylearn2, Numpy, Blocks, TensorFlow, MXNet, Caffe, Lasagne, Keras, Chainer, Matlab Deep Learning, CNTK, MatConvNet (a MATLAB toolbox implementing convolutional neural networks for computer vision applications), DeepLearnToolbox (a Matlab toolbox for Deep Learning (from Rasmus Berg Palm)), BigDL, Cuda-Convnet (a fast C++/CUDA implementation of convolutional (or more generally, feed-forward) neural networks), Deep Belief Networks, RNNLM, RNNLIB-RNNLIB, matrbm, deeplearning4j, Eblearn.lsh, deepmat, MShadow, Matplotlib, SciPy, CXXNET, Nengo-Nengo, Eblearn, cudamat, Gnumpy, 3-way factored RBM and mcRBM, mPoT (Python code using CUDAMat and Gnumpy to train models of natural images), ConvNet, Elektronn, OpenNN, NeuralDesigner, Theano Generalized Hebbian Learning, Apache Singa, Lightnet, and SimpleDNN.
In accordance with certain embodiments of the present disclosure, machine learning may be performed in two stages. For example, first, training occurs, which may be performed offline in that the system 100 is not being utilized to perform actual classifying/sorting of material pieces. The system 100 may be utilized to train the machine learning system in that homogenous sets (also referred to herein as control samples) of material pieces (i.e., having the same types or classes of materials, or falling within the same predetermined fraction) are passed through the system 100 (e.g., by a conveyor system 103); and all such material pieces may not be sorted, but may be collected in a common receptacle (e.g., receptacle 140). Alternatively, the training may be performed at another location remote from the system 100, including using some other mechanism for collecting sensed information (characteristics) of control sets of material pieces. During this training stage, algorithms within the machine learning system extract features from the captured information (e.g., using image processing techniques well known in the art). Non-limiting examples of training algorithms include, but are not limited to, linear regression, gradient descent, feed forward, polynomial regression, learning curves, regularized learning models, and logistic regression. It is during this training stage that the algorithms within the machine learning system learn the relationships between materials and their features/characteristics (e.g., as captured by the vision system and/or sensor system(s)), creating a knowledge base for later classification of a mixture of material pieces received by the system 100. Such a knowledge base may include one or more libraries, wherein each library includes parameters (e.g., neural network parameters) for utilization by the machine learning system in classifying material pieces. For example, one particular library may include parameters configured by the training stage to recognize and classify a particular type or class of material, or one or more materials that fall with a predetermined fraction. In accordance with certain embodiments of the present disclosure, such libraries may be inputted into the machine learning system and then the user of the system 100 may be able to adjust certain ones of the parameters in order to adjust an operation of the system 100 (for example, adjusting the threshold effectiveness of how well the machine learning system recognizes a particular material piece from a mixture of materials).
Additionally, the inclusion of certain materials (e.g., chemical elements or compounds) in material pieces (e.g., metal alloys), or combinations of certain chemical elements or compounds, can result in identifiable physical features (e.g., visually discernible characteristics) in materials. As a result, when a plurality of material pieces containing such a particular composition are passed through the aforementioned training stage, the machine learning system can learn how to distinguish such material pieces from others. Consequently, a machine learning system configured in accordance with certain embodiments of the present disclosure may be configured to sort between material pieces as a function of their respective chemical compositions. For example, such a machine learning system may be configured so that different aluminum alloys can be sorted as a function of the percentage of a specified alloying material contained within the aluminum alloys.
For example,
Embodiments of the present disclosure are not limited to the materials illustrated in
After parameters within the algorithms have been established and the machine learning system has sufficiently learned (been trained) the differences (e.g., visually discernible differences) for the material classifications (e.g., within a user-defined level of statistical confidence), the libraries for the different material classifications are then implemented into a material classifying and/or sorting system (e.g., system 100) to be used for identifying and/or classifying material pieces from a mixture of material pieces, and then sorting such classified material pieces if sorting is to be performed (e.g., to produce a specific aggregate chemical composition).
Techniques to construct, optimize, and utilize a machine learning system are known to those of ordinary skill in the art as found in relevant literature. Examples of such literature include the publications: Krizhevsky et al., “ImageNet Classification with Deep Convolutional Networks,” Proceedings of the 25th International Conference on Neural Information Processing Systems, Dec. 3-6, 2012, Lake Tahoe, Nev.; and LeCun et al., “Gradient-Based Learning Applied to Document Recognition,” Proceedings of the IEEE, Institute of Electrical and Electronic Engineers (IEEE), November 1998, both of which are hereby incorporated by reference herein in their entirety.
In an exemplary technique, data captured by a sensor and/or vision system with respect to a particular material piece may be processed as an array of data values within a data processing system (e.g., the data processing system 3400 of
In accordance with certain embodiments of the present disclosure in which a neural network is implemented, as a final layer (the “classification layer”), the final set of neurons' output is trained to represent the likelihood a material piece is associated with the captured data. During operation, if the likelihood that a material piece is associated with the captured data is over a user-specified threshold, then it is determined that the material piece is indeed associated with the captured data. These techniques can be extended to determine not only the presence of a type of material associated with particular captured data, but also whether sub-regions of the particular captured data belong to one type of material or another type of material. This process is known as segmentation, and techniques to use neural networks exist in the literature, such as those known as “fully convolutional” neural networks, or networks that otherwise include a convolutional portion (i.e., are partially convolutional), if not fully convolutional. This allows for material location and size to be determined.
It should be understood that the present disclosure is not exclusively limited to machine learning techniques. Other common techniques for material classification/identification may also be used. For instance, a sensor system may utilize optical spectrometric techniques using multi- or hyper-spectral cameras to provide a signal that may indicate the presence or absence of a type, class, or fraction of material by examining the spectral emissions (i.e., spectral imaging) of the material. Spectral images of a material piece may also be used in a template-matching algorithm, wherein a database of spectral images is compared against an acquired spectral image to find the presence or absence of certain types of materials from that database. A histogram of the captured spectral image may also be compared against a database of histograms. Similarly, a bag of words model may be used with a feature extraction technique, such as scale-invariant feature transform (“SIFT”), to compare extracted features between a captured spectral image and those in a database.
Therefore, as disclosed herein, certain embodiments of the present disclosure provide for the identification/classification of one or more different types, classes, or fractions of materials in order to determine which material pieces should be diverted from a conveyor system (i.e., sorted) in defined groups (e.g., in accordance with one or more predetermined specific aggregate chemical compositions). In accordance with certain embodiments, machine learning techniques are utilized to train (i.e., configure) a neural network to identify a variety of one or more different types, classes, or fractions of materials. Spectral images, or other types of sensed information, are captured of materials (e.g., traveling on a conveyor system), and based on the identification/classification of such materials, the systems described herein can decide which material piece should be allowed to remain on the conveyor system, and which should be diverted/removed from the conveyor system (for example, either into a collection receptacle, or diverted onto another conveyor system).
In accordance with certain embodiments of the present disclosure, a machine learning system for an existing installation (e.g., the system 100) may be dynamically reconfigured to identify/classify characteristics of a new type, class, or fraction of materials by replacing a current set of neural network parameters with a new set of neural network parameters.
A point of mention here is that, in accordance with certain embodiments of the present disclosure, the detected/captured features/characteristics (e.g., spectral images) of the material pieces may not be necessarily simply particularly identifiable or discernible physical characteristics; they can be abstract formulations that can only be expressed mathematically, or not mathematically at all; nevertheless, the machine learning system may be configured to parse the spectral data to look for patterns that allow the control samples to be classified during the training stage. Furthermore, the machine learning system may take subsections of captured information (e.g., spectral images) of a material piece and attempt to find correlations between the pre-defined classifications.
In accordance with certain embodiments of the present disclosure, instead of utilizing a training stage whereby control samples of material pieces are passed by the vision system and/or sensor system(s), training of the machine learning system may be performed utilizing a labeling/annotation technique whereby as data/information of material pieces are captured by a vision/sensor system, a user inputs a label or annotation that identifies each material piece, which is then used to create the library for use by the machine learning system when classifying material pieces within a mixture of material pieces.
In accordance with certain embodiments of the present disclosure, any sensed characteristics output by any of the sensor systems 120 disclosed herein may be input into a machine learning system in order to classify and/or sort materials. For example, in a machine learning system implementing supervised learning, sensor system 120 outputs that uniquely characterize a specific type or composition of material (e.g., a specific metal alloy) may be used to train the machine learning system.
In the process block 3501, the material pieces 101 may be deposited onto a conveyor system 103. In the process block 3502, the location on the conveyor system 103 of each material piece 101 is detected for tracking of each material piece 101 as it travels through the system 100. This may be performed by the vision system 110 (for example, by distinguishing a material piece 101 from the underlying conveyor system material while in communication with a conveyor system position detector (e.g., the position detector 105)). Alternatively, a material tracking device 111 can be used to track the material pieces 101. Or, any system that can create a light source (including, but not limited to, visual light, UV, and IR) and has a corresponding detector can be used to track the material pieces 101. In the process block 3503, when a material piece 101 has traveled in proximity to one or more of the vision system 110 and/or the sensor system(s) 120, sensed information/characteristics of the material piece 101 is captured/acquired. In the process block 3504, a vision system (e.g., implemented within the computer system 107), such as previously disclosed, may perform pre-processing of the captured information, which may be utilized to detect (extract) information of each of the material pieces 101 (e.g., from the background (e.g., the conveyor belt 103); in other words, the pre-processing may be utilized to identify the difference between the material piece 101 and the background). Well-known image processing techniques such as dilation, thresholding, and contouring may be utilized to identify the material piece 101 as being distinct from the background. In the process block 3505, segmentation may be performed. For example, the captured information may include information pertaining to one or more material pieces 101. Additionally, a particular material piece 101 may be located on a seam of the conveyor belt 103 when its image is captured. Therefore, it may be desired in such instances to isolate the image of an individual material piece 101 from the background of the image. In an exemplary technique for the process block 3505, a first step is to apply a high contrast of the image; in this fashion, background pixels are reduced to substantially all black pixels, and at least some of the pixels pertaining to the material piece 101 are brightened to substantially all white pixels. The image pixels of the material piece 101 that are white are then dilated to cover the entire size of the material piece 101. After this step, the location of the material piece 101 is a high contrast image of all white pixels on a black background. Then, a contouring algorithm can be utilized to detect boundaries of the material piece 101. The boundary information is saved, and the boundary locations are then transferred to the original image. Segmentation is then performed on the original image on an area greater than the boundary that was earlier defined. In this fashion, the material piece 101 is identified and separated from the background.
In the optional process block 3506, the material pieces 101 may be conveyed along the conveyor system 103 within proximity of the material tracking and measuring device 111 and/or a sensor system 120 in order to determine a size and/or shape of the material pieces 101. Such a material tracking and measuring device 111 may be configured to measure one or more dimensions of each material piece so that the system can calculate (determine) an approximate mass of each material piece. In the process block 3507, post processing may be performed. Post processing may involve resizing the captured information/data to prepare it for use in the machine learning system. This may also include modifying certain properties (e.g., enhancing image contrast, changing the image background, or applying filters) in a manner that will yield an enhancement to the capability of the machine learning system to classify the material pieces 101. In the process block 3509, the data may be resized. Data resizing may be desired under certain circumstances to match the data input requirements for certain machine learning systems, such as neural networks. For example, neural networks may require much smaller image data sizes (e.g., 225×255 pixels or 299×299 pixels) than the sizes of the images captured by typical digital cameras. Moreover, the smaller the input data size, the less processing time is needed to perform the classification. Thus, smaller data sizes can increase the throughput of the system 100 and increase its value.
In the process blocks 3510 and 3511, each material piece 101 is identified/classified based on the sensed/detected features. For example, the process block 3510 may be configured with a neural network employing one or more machine learning algorithms, which compare the extracted features with those stored in a previously generated knowledge base (e.g., generated during a training stage), and assigns the classification with the highest match to each of the material pieces 101 based on such a comparison. The algorithms of the machine learning system may process the captured information/data in a hierarchical manner by using automatically trained filters. The filter responses are then successfully combined in the next levels of the algorithms until a probability is obtained in the final step. In the process block 3511, these probabilities may be used for each of the N classifications to decide into which of the N sorting receptacles the respective material pieces 101 should be sorted. Each of the N classifications may pertain to N different predetermined specific aggregate chemical compositions. For example, each of the N classifications may be assigned to one sorting receptacle, and the material piece 101 under consideration is sorted into that receptacle that corresponds to the classification returning the highest probability larger than a predefined threshold. Within embodiments of the present disclosure, such predefined thresholds may be preset by the user. A particular material piece 101 may be sorted into an outlier receptacle (e.g., sorting receptacle 140) if none of the probabilities is larger than the predetermined threshold.
Next, in the process block 3512, a sorting device 126 . . . 129 corresponding to the classification, or classifications, of the material piece 101 is activated. Between the time at which the image of the material piece 101 was captured and the time at which the sorting device 126 . . . 129 is activated, the material piece 101 has moved from the proximity of the vision system 110 and/or sensor system(s) 120 to a location downstream on the conveyor system 103 (e.g., at the rate of conveying of a conveyor system). In embodiments of the present disclosure, the activation of the sorting device 126 . . . 129 is timed such that as the material piece 101 passes the sorting device 126 . . . 129 mapped to the classification of the material piece 101, the sorting device 126 . . . 129 is activated, and the material piece 101 is diverted/ejected from the conveyor system 103 into its associated sorting receptacle 136 . . . 139. Within embodiments of the present disclosure, the activation of a sorting device 126 . . . 129 may be timed by a respective position detector that detects when a material piece 101 is passing before the sorting device 126 . . . 129 and sends a signal to enable the activation of the sorting device 126 . . . 129. In the process block 3513, the sorting receptacle 136 . . . 139 corresponding to the sorting device 126 . . . 129 that was activated receives the diverted/ejected material piece 101.
The process 1000 may be configured to operate in conjunction with the process 3500. For example, in accordance with certain embodiments of the present disclosure, the process blocks 1003 and 1004 may be incorporated in the process 3500 (e.g., operating in series or in parallel with the process blocks 3503-3510) in order to combine the efforts of a vision system 110 that is implemented in conjunction with a machine learning system with a sensor system (e.g., a sensor system 120) that is not implemented in conjunction with a machine learning system in order to classify and/or sort material pieces 101, including in accordance with the system and method 400 of
Operation of the process 1000 may be performed by hardware and/or software, including within a computer system (e.g., computer system 3400 of
Next, if sorting of the material pieces 101 is to be performed, in the process block 1006, a sorting device 126 . . . 129 corresponding to the classification, or classifications, of the material piece 101 is activated. Between the time at which the material piece was sensed and the time at which the sorting device 126 . . . 129 is activated, the material piece 101 has moved from the proximity of the sensor system 120 to a location downstream on the conveyor system 103, at the rate of conveying of the conveyor system. In certain embodiments of the present disclosure, the activation of the sorting device 126 . . . 129 is timed such that as the material piece 101 passes the sorting device 126 . . . 129 mapped to the classification of the material piece 101, the sorting device 126 . . . 129 is activated, and the material piece 101 is diverted/ejected from the conveyor system 103 into its associated sorting receptacle 136 . . . 139. Within certain embodiments of the present disclosure, the activation of a sorting device 126 . . . 129 may be timed by a respective position detector that detects when a material piece 101 is passing before the sorting device 126 . . . 129 and sends a signal to enable the activation of the sorting device 126 . . . 129. In the process block 1007, the sorting receptacle 136 . . . 139 corresponding to the sorting device 126 . . . 129 that was activated receives the diverted/ejected material piece 101.
In accordance with various embodiments of the present disclosure, different types or classes of materials may be classified by different types of sensors each for use with a machine learning system, and combined to classify material pieces in a stream of scrap or waste.
In accordance with various embodiments of the present disclosure, data (e.g., spectral data) from two or more sensors can be combined using a single or multiple machine learning systems to perform classifications of material pieces.
In accordance with various embodiments of the present disclosure, multiple sensor systems can be mounted onto a single conveyor system, with each sensor system utilizing a different machine learning system. In accordance with various embodiments of the present disclosure, multiple sensor systems can be mounted onto different conveyor systems, with each sensor system utilizing a different machine learning system.
In accordance with embodiments of the present disclosure, the system 100 may be configured (e.g., in accordance with the system and method 400 of
A non-limiting example would be the production of an aluminum alloy possessing a chemical composition according to a predetermined (e.g., as designed by the user of the system 100) combination of specific weight percentages (wt. %) of aluminum, silicon, magnesium, iron, manganese, copper, and zinc. The scrap pieces of aluminum alloys available to be fed into the system 100 may be those listed in the table of
However, embodiments of the present disclosure can be configured to produce a collection of aluminum alloy scrap pieces possessing an aggregate chemical composition equivalent, or at least substantially equivalent, to the chemical composition listed in the table of
Since the individual aluminum alloy scrap pieces may have different sizes, and thus different masses, the material tracking and measuring device 111 may be utilized to estimate the mass for each aluminum alloy scrap piece. For example, the sizes of each of the scrap pieces measured by the material tracking and measuring device 111 may be utilized by the system 100 to determine (calculate) a mass, or at least an approximate mass, for each scrap piece. Since the system 100 has been configured to recognize and classify each scrap piece as belonging to one of the plurality of aluminum alloys listed in the table of
To produce a collection of the aluminum alloy scrap pieces possessing the aggregate chemical composition, the system 100 is configured to then classify and select for sorting those aluminum alloy scrap pieces fed into the system 100 that, when combined, achieve the aggregate chemical composition for the combined mass of the sorted aluminum alloy scrap pieces. In other words, if such a collection of aluminum alloy scrap pieces sorted and output by the system 100 were melted together (which they are likely to be at some point), the resultant melt would possess the aggregate chemical composition, or at least substantially close to the aggregate chemical composition within a desired threshold of accuracy.
Consequently, the system 100 may be configured to calculate on a running basis the contributions to the individual masses of each of the chemical elements within the aggregate chemical composition as each aluminum alloy scrap piece is added to the sorted-out collection so that the system 100 can then determine whether the next aluminum alloy scrap piece that is classified should be added to the collection or not (i.e., sorted from a mixture of aluminum alloy scrap pieces).
In the process block 401, the system 100 receives, or is input with, a predetermined specific aggregate chemical composition that is desired to be produced at the output of one of the sorting devices 126 . . . 129 within the system 100. In the process block 402, as each material piece 101 is conveyed past the material tracking and measuring device 111, the material tracking and measuring device 111 will determine the size and/or shape of each of the material pieces 101 as described herein. In the process block 403, a classification is assigned to each of the material pieces 101 by the vision system 110 and/or one or more of the sensor systems 120 in a manner as described herein (e.g., see
In the process block 405, the system 100 will sort each of the material pieces 101 based on the determined chemical compositions and masses so as to achieve the predetermined specific aggregate chemical composition. For example, the system 100 may be configured to sort (e.g., divert) each of these material pieces 101 into a predetermined receptacle (e.g., the receptacle 136) by a predetermined sorting device (e.g., the sorting device 126). The remainder of the material pieces 101 may be collected into the receptacle 140, or the system 100 may be configured to sort certain ones of the material pieces 101 into another receptacle (e.g., receptacle 137) to achieve a second (e.g., different) predetermined specific aggregate chemical composition. Alternatively, the system 100 may be configured to sort the remaining material pieces 101 based on any other type of desired classification(s), such as sorting the remaining material pieces 101 into two different classifications (e.g., wrought, extruded, and/or cast aluminum). In the process block 406, the sorted material pieces 101 for achieving the specific aggregate chemical composition are collected into the predetermined receptacle (e.g., the receptacle 136).
The process blocks 402-406 may be repeated as needed to achieve the specific aggregate chemical composition, to achieve the specific aggregate chemical composition within a specified threshold of accuracy, or to achieve the specific aggregate chemical composition for a desired (predetermined) collected mass of materials (as may be determined by counting the number of materials diverted into the receptacle). For example, as each material piece is sorted, the system may continually determine (i.e., update) the aggregate chemical composition of the then collected material pieces, and will then continue the sorting until the updated aggregate chemical composition is within a threshold level of the predetermined specific aggregate chemical composition. As each material piece is classified, the system will determine whether to divert that material piece to join the collection, such as whether that material piece would increase or decrease the aggregate weight percentage of a specific chemical element within the already sorted and collected material pieces. Additionally, the system may be configured to not divert certain material pieces into the collection because such material pieces contain a contaminant that is not desired to be included within the predetermined specific chemical composition (e.g., a wrought aluminum alloy piece that contains an iron-containing material such as a bolt). Alternatively, other systems may be implemented in order to remove material pieces that contain a particular contaminant.
The material tracking and measuring device 111 may be a well-known one-dimensional or two-dimensional line scanner. If it is a one-dimensional line scanner, then it will measure a length of each material piece along the direction of travel. If it can be assumed that the majority of material pieces are approximately equal in length and width, such a length measurement can be utilized to approximate the mass of each material piece. If a two-dimensional line scanner is utilized, then it can measure both the length and the width of each material piece for use in determining the masses.
Alternatively, one or more cameras may be utilized in a well-known manner to image each material piece and determine the approximate dimensions of each material piece. Such camera(s) may be positioned in proximity to the conveyor belt before the sorting apparatus, or could be positioned downstream from the sorting apparatus so that only the sorted material pieces are imaged to determine their approximate masses.
If it can be assumed that a sufficient majority of the material pieces are all of about the same size and mass, then such implementations for determining the mass of each piece can be omitted.
Alternatively, the receptacle that is collecting the diverted material pieces could be positioned on a weight scale that continually weighs the collected material pieces, thus providing an approximate weight and resultant mass for each material piece as it is sorted and collected within the receptacle. These masses can them be utilized in the system and process 400 as described herein.
In accordance with certain embodiments of the present disclosure, a plurality of at least a portion of the system 100 may be linked together in succession in order to perform multiple iterations or layers of sorting. For example, when two or more systems 100 are linked in such a manner, the conveyor system may be implemented with a single conveyor belt, or multiple conveyor belts, conveying the material pieces past a first vision system (and, in accordance with certain embodiments, a sensor system) configured for sorting material pieces of a first set of a mixture of materials by a sorter (e.g., the first automation control system 108 and associated one or more sorting devices 126 . . . 129) into a first set of one or more receptacles (e.g., sorting receptacles 136 . . . 139), and then conveying the material pieces past a second vision system (and, in accordance with certain embodiments, another sensor system) configured for sorting material pieces of a second set of a mixture of materials by a second sorter into a second set of one or more sorting receptacles. A further discussion of such multistage sorting is in U.S. published patent application no. 2022/0016675, which is hereby incorporated by reference herein.
Such successions of systems 100 can contain any number of such systems linked together in such a manner In accordance with certain embodiments of the present disclosure, each successive vision system or sensor system may be configured to sort out a different material than previous vision system(s) or sensor system(s) with the end result producing a collection of material pieces possessing the predetermined specific aggregate chemical composition.
With reference now to
Additional connections to the local bus 3405 may be made through direct component interconnection or through add-in boards. In the depicted example, a communication (e.g., network (LAN)) adapter 3425, an I/O (e.g , small computer system interface (“SCSI”) host bus) adapter 3430, and expansion bus interface (not shown) may be connected to the local bus 3405 by direct component connection. An audio adapter (not shown), a graphics adapter (not shown), and display adapter 3416 (coupled to a display 3440) may be connected to the local bus 3405 (e.g., by add-in boards inserted into expansion slots).
The user interface adapter 3412 may provide a connection for a keyboard 3413 and a mouse 3414, modem (not shown), and additional memory (not shown). The I/O adapter 3430 may provide a connection for a hard disk drive 3431, a solid state drive 3432, and a CD-ROM drive (not shown).
An operating system may be run on the one or more processors 3415 and used to coordinate and provide control of various components within the computer system 3400. In
Those of ordinary skill in the art will appreciate that the hardware in
As another example, the computer system 3400 may be a stand-alone system configured to be bootable without relying on some type of network communication interface, whether or not the computer system 3400 includes some type of network communication interface. As a further example, the computer system 3400 may be an embedded controller, which is configured with ROM and/or flash ROM providing non-volatile memory storing operating system files or user-generated data.
The depicted example in
As has been described herein, embodiments of the present disclosure may be implemented to perform the various functions described for identifying, tracking, classifying, and/or sorting material pieces. Such functionalities may be implemented within hardware and/or software, such as within one or more data processing systems (e.g., the data processing system 3400 of
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, process, method, and/or computer program product. Accordingly, various aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.), or embodiments combining software and hardware aspects, which may generally be referred to herein as a “circuit,” “circuitry,” “module,” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable storage medium(s) having computer readable program code embodied thereon. (However, any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium.)
A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, biologic, atomic, or semiconductor system, apparatus, controller, or device, or any suitable combination of the foregoing, wherein the computer readable storage medium is not a transitory signal per se. More specific examples (a non-exhaustive list) of the computer readable storage medium may include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a solid state memory, a random access memory (“RAM”) (e.g., RAM 3420 of
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, controller, or device.
The flowchart and block diagrams in the figures illustrate architecture, functionality, and operation of possible implementations of systems, methods, processes, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of code that includes one or more executable program instructions for implementing the specified logical function(s). It should also be noted that, in some implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.
In the description herein, a flow-charted technique may be described in a series of sequential actions. The sequence of the actions, and the party performing the actions, may be freely changed without departing from the scope of the teachings. Actions may be added, deleted, or altered in several ways. Similarly, the actions may be re-ordered or looped. Further, although processes, methods, algorithms, or the like may be described in a sequential order, such processes, methods, algorithms, or any combination thereof may be operable to be performed in alternative orders. Further, some actions within a process, method, or algorithm may be performed simultaneously during at least a point in time (e.g., actions performed in parallel), can also be performed in whole, in part, or any combination thereof.
Modules implemented in software for execution by various types of processors (e.g., GPU 3401, CPU 3415) may, for instance, include one or more physical or logical blocks of computer instructions, which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together, but may include disparate instructions stored in different locations that when joined logically together, include the module and achieve the stated purpose for the module. Indeed, a module of executable code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices. Similarly, operational data (e.g., material classification libraries described herein) may be identified and illustrated herein within modules, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including over different storage devices. The data may provide electronic signals on a system or network.
These program instructions may be provided to one or more processors and/or controller(s) of a general purpose computer, special purpose computer, or other programmable data processing apparatus (e.g., controller) to produce a machine, such that the instructions, which execute via the processor(s) (e.g., GPU 3401, CPU 3415) of the computer or other programmable data processing apparatus, create circuitry or means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. In a particular embodiment, computer program instructions may be configured to send sorting instructions to a sorting apparatus in order to direct sorting of certain ones of the material pieces from the plurality of material pieces to produce a collection of material pieces possessing a predetermined specific aggregate chemical composition.
It will also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by special purpose hardware-based systems (e.g., which may include one or more graphics processing units (e.g., GPU 3401)) that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions. For example, a module may be implemented as a hardware circuit including custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, controllers, or other discrete components. A module may also be implemented in programmable hardware devices, such as field programmable gate arrays, programmable array logic, programmable logic devices, or the like.
Computer program code, i.e., instructions, for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, Python, C++, or the like, conventional procedural programming languages, such as the “C” programming language or similar programming languages, or any of the machine learning software disclosed herein. The program code may execute entirely on the user' s computer system, partly on the user's computer system, as a stand-alone software package, partly on the user' s computer system (e.g., the computer system utilized for sorting) and partly on a remote computer system (e.g., the computer system utilized to train the sensor system), or entirely on the remote computer system or server. In the latter scenario, the remote computer system may be connected to the user's computer system through any type of network, including a local area network (“LAN”) or a wide area network (“WAN”), or the connection may be made to an external computer system (for example, through the Internet using an Internet Service Provider).
These program instructions may also be stored in a computer readable storage medium that can direct a computer system, other programmable data processing apparatus, controller, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
One or more databases may be included in a host for storing and providing access to data for the various implementations. One skilled in the art will also appreciate that, for security reasons, any databases, systems, or components of the present disclosure may include any combination of databases or components at a single location or at multiple locations, wherein each database or system may include any of various suitable security features, such as firewalls, access codes, encryption, de-encryption and the like. The database may be any type of database, such as relational, hierarchical, object-oriented, and/or the like. Common database products that may be used to implement the databases include DB2 by IBM, any of the database products available from Oracle Corporation, Microsoft Access by Microsoft Corporation, or any other database product. The database may be organized in any suitable manner, including as data tables or lookup tables.
Association of certain data (e.g., between a classified material piece and its known chemical composition, or between a classified material piece and its calculated approximate mass) may be accomplished through any data association technique known and practiced in the art. For example, the association may be accomplished either manually or automatically. Automatic association techniques may include, for example, a database search, a database merge, GREP, AGREP, SQL, and/or the like. The association step may be accomplished by a database merge function, for example, using a key field in each of the manufacturer and retailer data tables. A key field partitions the database according to the high-level class of objects defined by the key field. For example, a certain class may be designated as a key field in both the first data table and the second data table, and the two data tables may then be merged on the basis of the class data in the key field. In these embodiments, the data corresponding to the key field in each of the merged data tables is preferably the same. However, data tables having similar, though not identical, data in the key fields may also be merged by using AGREP, for example.
Aspects of the present disclosure provide a method that includes determining an approximate mass of each material piece of a plurality of material pieces, wherein at least one of the plurality of material pieces has a material classification different from the other material pieces; classifying each material piece of the plurality of material pieces as belonging to one of a plurality of different material classifications; and sorting certain ones of the material pieces from the plurality of material pieces as a function of the determined approximate mass and classification of each material piece of the plurality of material pieces, wherein the sorting produces a collection of material pieces possessing a predetermined specific aggregate chemical composition. The sorting may include diverting the certain ones of the material pieces into a receptacle. The sorting may include continually determining an aggregate chemical composition of the diverted material pieces. The sorting may include diverting a next material piece into the receptacle in order to increase a weight percentage of a specific chemical element of the aggregate chemical composition of the diverted material pieces. The sorting may include not diverting a next material piece into the receptacle in order to decrease a weight percentage of a specific chemical element of the aggregate chemical composition of the diverted material pieces. The sorting may include not diverting a next material piece into the receptacle because it contains a contaminant that is not desired within the predetermined specific aggregate chemical composition. The sorting may be continued until the aggregate chemical composition of a predetermined minimum number of diverted material pieces is equal to a threshold level of the predetermined specific aggregate chemical composition. The collection of material pieces possessing a predetermined specific aggregate chemical composition may contain at least one material piece that possesses a material classification different from the other material pieces in the collection. The plurality of material pieces may include material pieces possessing different metal alloy compositions. The predetermined specific aggregate chemical composition may be different than the chemical composition of each of the plurality of material pieces. The predetermined specific aggregate chemical composition may be different than the aggregate chemical composition of all of the plurality of material pieces. The collection of material pieces may include material pieces having different material classifications. The collection of material pieces may include at least one of the material pieces having a material classification different from the other material pieces. The plurality of pieces may include wrought aluminum alloy pieces and cast aluminum alloy pieces, wherein the collection of material pieces may include at least one wrought aluminum alloy piece and at least one cast aluminum alloy piece, and wherein the predetermined specific aggregate chemical composition is different than a chemical composition of the wrought aluminum alloy pieces, and wherein the predetermined specific aggregate chemical composition is different than a chemical composition of the cast aluminum alloy pieces. The classifying may include processing image data captured from each of the plurality of material pieces through a machine learning system.
Aspects of the present disclosure provide a system that includes a sensor configured to capture one or more characteristics of each of a mixture of material pieces, wherein the mixture of material pieces may include material pieces having different material classifications; a data processing system configured to classify each material piece of the mixture of material pieces as belonging to one of a plurality of different material classifications; and a sorting device configured to sort certain ones of the material pieces from the mixture of material pieces as a function of the classification of each material piece of the mixture of material pieces, wherein the sorting produces a collection of material pieces possessing a predetermined specific aggregate chemical composition. The sensor may be a camera, wherein the one or more captured characteristics were captured by the camera configured to capture images of each of the mixture of material pieces as they were conveyed past the camera, wherein the camera is configured to capture visual images of each of the mixture of materials to produce image data, and wherein the characteristics are visually observed characteristics. The data processing system may include a machine learning system implementing a neural network configured to classify each material piece of the mixture of material pieces as belonging to one of a plurality of different material classifications based on the captured visually observed characteristics. The system may further include an apparatus configured to determine an approximate mass of each material piece of a plurality of material pieces, wherein the sorting is performed as a function of the determined approximate mass and classification of each material piece. The apparatus may include a line scanner configured to measure an approximate size of each material piece.
Aspects of the present disclosure provide a computer program product stored on a computer readable storage medium, which when executed by a data processing system, performs a process that includes determining an approximate mass of each material piece of a plurality of material pieces, wherein at least one of the plurality of material pieces has a material classification different from the other material pieces; classifying each material piece of the plurality of material pieces as belonging to one of a plurality of different material classifications; and directing sorting of certain ones of the material pieces from the plurality of material pieces to produce a collection of material pieces possessing a predetermined specific aggregate chemical composition, wherein the sorting is performed as a function of the determined approximate mass and classification of each material piece of the plurality of material pieces, wherein the collection of material pieces includes material pieces having different material classifications. The classifying may include processing image data captured from each of the plurality of material pieces through a machine learning system. The predetermined specific aggregate chemical composition may be different than the chemical composition of each of the plurality of material pieces.
Reference is made herein to “configuring” a device or a device “configured to” perform some function. It should be understood that this may include selecting predefined logic blocks and logically associating them, such that they provide particular logic functions, which includes monitoring or control functions. It may also include programming computer software-based logic of a control device, wiring discrete hardware components, or a combination of any or all of the foregoing.
In the descriptions herein, numerous specific details are provided, such as examples of programming, software modules, user selections, network transactions, database queries, database structures, hardware modules, hardware circuits, hardware chips, controllers, etc., to provide a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the disclosure may be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures, materials, or operations may be not shown or described in detail to avoid obscuring aspects of the disclosure.
Those of skill in the art should appreciate that the various settings and parameters (including the neural network parameters) of the components of the system 100 may be customized, optimized, and reconfigured over time based on the types of materials being classified and sorted, the desired classification and sorting results, the type of equipment being used, empirical results from previous classifications, data that becomes available, and other factors.
Reference throughout this specification to “an embodiment,” “embodiments,” or similar language means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment,” “in an embodiment,” “embodiments,” “certain embodiments,” “various embodiments,” and similar language throughout this specification may, but do not necessarily, all refer to the same embodiment. Furthermore, the described features, structures, aspects, and/or characteristics of the disclosure may be combined in any suitable manner in one or more embodiments. Correspondingly, even if features may be initially claimed as acting in certain combinations, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination can be directed to a sub-combination or variation of a sub-combination.
Benefits, advantages, and solutions to problems have been described herein with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or elements of any or all the claims. Further, no component described herein is required for the practice of the disclosure unless expressly described as essential or critical.
While this specification contains many specifics, these should not be construed as limitations on the scope of the disclosure or of what can be claimed, but rather as descriptions of features specific to particular implementations of the disclosure. Headings herein may be not intended to limit the disclosure, embodiments of the disclosure or other matter disclosed under the headings.
Herein, the term “or” may be intended to be inclusive, wherein “A or B” includes A or B and also includes both A and B. As used herein, the term “and/or” when used in the context of a listing of entities, refers to the entities being present singly or in combination. Thus, for example, the phrase “A, B, C, and/or D” includes A, B, C, and D individually, but also includes any and all combinations and subcombinations of A, B, C, and D.
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a,” “an,” and “the” may be intended to include the plural forms as well, unless the context clearly indicates otherwise.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below may be intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed.
As used herein, terms such as “controller,” “processor,” “memory,” “neural network,” “interface,” “sorter,” “sorter apparatus,” “sorting device,” “device,” “pushing mechanism,” “pusher devices,” “imaging sensor,” “bin,” “receptacle,” “system,” and “circuitry” each refer to non-generic device elements that would be recognized and understood by those of skill in the art and are not used herein as nonce words or nonce terms for the purpose of invoking 35 U.S.C. 112(f).
As used herein with respect to an identified property or circumstance, “substantially” refers to a degree of deviation that is sufficiently small so as to not measurably detract from the identified property or circumstance. The exact degree of deviation allowable may in some cases depend on the specific context.
As used herein, a plurality of items, structural elements, compositional elements, exemplary fractions, and/or materials may be presented in a common list for convenience. However, these lists should be construed as though each member of the list is individually identified as a separate and unique member. Thus, no individual member of such list should be construed as a defacto equivalent of any other member of the same list solely based on their presentation in a common group without indications to the contrary.
Unless defined otherwise, all technical and scientific terms (such as acronyms used for chemical elements within the periodic table) used herein have the same meaning as commonly understood to one of ordinary skill in the art to which the presently disclosed subject matter belongs. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety, unless a particular passage is cited. In case of conflict, the present specification, including definitions, will control. In addition, the materials, methods, and examples are illustrative only, and not intended to be limiting.
To the extent not described herein, many details regarding specific materials, processing acts, and circuits are conventional, and may be found in textbooks and other sources within the computing, electronics, and software arts.
Unless otherwise indicated, all numbers expressing quantities of ingredients, reaction conditions, and so forth used in the specification and claims are to be understood as being modified in all instances by the term “about.” Accordingly, unless indicated to the contrary, the numerical parameters set forth in this specification and attached claims are approximations that can vary depending upon the desired properties sought to be obtained by the presently disclosed subject matter. As used herein, the term “about,” when referring to a value or to an amount of mass, weight, time, volume, concentration or percentage is meant to encompass variations of in some embodiments ±20%, in some embodiments ±10%, in some embodiments ±5%, in some embodiments ±1%, in some embodiments ±0.5%, and in some embodiments ±0.1% from the specified amount, as such variations are appropriate to perform the disclosed method. As used herein, the term “similar” may refer to values that are within a particular offset or percentage of each other (e.g., 1%, 2%, 5%, 10%, etc.).
This application claims priority to U.S. Provisional Patent Application Ser. No. 63/249,069 and to U.S. Provisional Patent Application Ser. No. 63/285,964. This application is a continuation-in-part application of U.S. patent application Ser. No. 17/667,397, which claims priority to U.S. Provisional Patent Application Ser. No. 63/146,892 and to U.S. Provisional Patent Application Ser. No. 63/173,301, and which is a continuation-in-part application of U.S. patent application Ser. No. 17/495,291, which is a continuation of U.S. patent application Ser. No. 17/380,928, which is a continuation-in-part application of U.S. patent application Ser. No. 17/227,245, which is a continuation-in-part application of U.S. patent application Ser. No. 16/939,011, which is a continuation application of U.S. patent application Ser. No. 16/375,675 (issued as U.S. Pat. No. 10,722,922), which is a continuation-in-part application of U.S. patent application Ser. No. 15/963,755 (issued as U.S. Pat. No. 10,710,119), which claims priority to U.S. Provisional Patent Application Ser. No. 62/490,219, and which is a continuation-in-part application of U.S. patent application Ser. No. 15/213,129 (issued as U.S. Pat. No. 10,207,296), which claims priority to U.S. Provisional Patent Application Ser. No. 62/193,332, which are all hereby incorporated by reference herein. U.S. patent application Ser. No. 17/495,291 is also a continuation-in-part application of U.S. patent application Ser. No. 17/491,415 (issued as U.S. Pat. No. 11,278,937), which is a continuation-in-part application of U.S. patent application Ser. No. 16/852,514 (issued as U.S. Pat. No. 11,260,426), which is a divisional application of U.S. patent application Ser. No. 16/358,374 (issued as U.S. Pat. No. 10,625,304), which is a continuation-in-part application of U.S. patent application Ser. No. 15/963,755 (issued as U.S. Pat. No. 10,710,119), which are all hereby incorporated by reference herein.
This disclosure was made with U.S. government support under Grant No. DE-AR0000422 awarded by the U.S. Department of Energy. The U.S. government may have certain rights in this disclosure.
| Number | Name | Date | Kind |
|---|---|---|---|
| 2194381 | Cadman | Sep 1937 | A |
| 2417878 | Luzietti et al. | Feb 1944 | A |
| 2942792 | Anderson et al. | Jul 1957 | A |
| 2953554 | Miller et al. | Sep 1960 | A |
| 3512638 | Chengges et al. | May 1970 | A |
| 3662874 | Muller | May 1972 | A |
| 3791518 | Vanderhoof | Feb 1974 | A |
| 3955678 | Moyer | May 1976 | A |
| 3973736 | Nilsson | Aug 1976 | A |
| 3974909 | Johnson | Aug 1976 | A |
| 4004681 | Clewett et al. | Jan 1977 | A |
| 4031998 | Suzuki et al. | Jun 1977 | A |
| 4044897 | Maxted | Aug 1977 | A |
| 4253154 | Russ et al. | Feb 1981 | A |
| 4317521 | Clark et al. | Mar 1982 | A |
| 4413721 | Bollier | Nov 1983 | A |
| 4488610 | Yankloski | Dec 1984 | A |
| 4572735 | Poetzschke et al. | Feb 1986 | A |
| 4586613 | Horii | May 1986 | A |
| 4726464 | Canziani | Feb 1988 | A |
| 4834870 | Osterberg et al. | May 1989 | A |
| 4848590 | Kelly | Jul 1989 | A |
| 5054601 | Sjogren et al. | Oct 1991 | A |
| 5114230 | Pryor | May 1992 | A |
| 5236092 | Krotkov et al. | Aug 1993 | A |
| 5260576 | Sommer, Jr. et al. | Nov 1993 | A |
| 5410637 | Kern et al. | Apr 1995 | A |
| 5433311 | Bonnet | Jul 1995 | A |
| 5462172 | Kumagai et al. | Oct 1995 | A |
| 5570773 | Bonnet | Nov 1996 | A |
| 5663997 | Willis et al. | Sep 1997 | A |
| 5676256 | Kumar et al. | Oct 1997 | A |
| 5733592 | Wettstein et al. | Mar 1998 | A |
| 5738224 | Sommer, Jr. et al. | Apr 1998 | A |
| 5836436 | Fortenbery et al. | Nov 1998 | A |
| 5911327 | Tanaka et al. | Jun 1999 | A |
| 6012659 | Nakazawa et al. | Jan 2000 | A |
| 6076653 | Bonnet | Jun 2000 | A |
| 6100487 | Schultz et al. | Aug 2000 | A |
| 6124560 | Roos | Sep 2000 | A |
| 6148990 | Lapeyre et al. | Nov 2000 | A |
| 6266390 | Sommer, Jr. et al. | Jul 2001 | B1 |
| 6273268 | Axmann | Aug 2001 | B1 |
| 6313422 | Anibas | Nov 2001 | B1 |
| 6313423 | Sommer | Nov 2001 | B1 |
| 6412642 | Charles et al. | Jul 2002 | B2 |
| 6457859 | Lu et al. | Oct 2002 | B1 |
| 6519315 | Sommer, Jr. et al. | Feb 2003 | B2 |
| 6795179 | Kumar | Sep 2004 | B2 |
| 6888917 | Sommer, Jr. et al. | May 2005 | B2 |
| 6983035 | Price et al. | Jan 2006 | B2 |
| 7073651 | Costanzo et al. | Jul 2006 | B2 |
| 7099433 | Sommer et al. | Aug 2006 | B2 |
| 7200200 | Laurila et al. | Apr 2007 | B2 |
| 7341154 | Boer | Mar 2008 | B2 |
| 7564943 | Sommer, Jr. et al. | Jul 2009 | B2 |
| 7616733 | Sommer et al. | Nov 2009 | B2 |
| 7674994 | Valerio | Mar 2010 | B1 |
| 7763820 | Sommer, Jr. | Jul 2010 | B1 |
| 7802685 | Allen et al. | Sep 2010 | B2 |
| 7848484 | Sommer, Jr. et al. | Dec 2010 | B2 |
| 7886915 | Shulman | Feb 2011 | B2 |
| 7903789 | Morton et al. | Mar 2011 | B2 |
| 7978814 | Sommer et al. | Jul 2011 | B2 |
| 7991109 | Golenhofen | Aug 2011 | B2 |
| 8073099 | Niu et al. | Dec 2011 | B2 |
| 8144831 | Sommer, Jr. et al. | Mar 2012 | B2 |
| 8172069 | Prakasam | May 2012 | B2 |
| 8429103 | Aradhye et al. | Apr 2013 | B1 |
| 8433121 | Kosarev | Apr 2013 | B2 |
| 8476545 | Sommer et al. | Jul 2013 | B2 |
| 8553838 | Sommer et al. | Oct 2013 | B2 |
| 8567587 | Faist et al. | Oct 2013 | B2 |
| 8576988 | Lewalter et al. | Nov 2013 | B2 |
| 8615123 | Dabic | Dec 2013 | B2 |
| 8654919 | Sabol et al. | Feb 2014 | B2 |
| 8855809 | Spencer et al. | Oct 2014 | B2 |
| 8903040 | Maeyama et al. | Dec 2014 | B2 |
| 9156162 | Suzuki et al. | Oct 2015 | B2 |
| 9316596 | Levesque | Apr 2016 | B2 |
| 9785851 | Torek et al. | Oct 2017 | B1 |
| 9927354 | Starr | Mar 2018 | B1 |
| 10036142 | Bamber | Jul 2018 | B2 |
| 10207296 | Garcia et al. | Feb 2019 | B2 |
| 10710119 | Kumar et al. | Jul 2020 | B2 |
| 20030038064 | Harbeck et al. | Feb 2003 | A1 |
| 20030147494 | Sommer, Jr. et al. | Aug 2003 | A1 |
| 20040151364 | Kenneway et al. | Aug 2004 | A1 |
| 20060239401 | Sommer, Jr. et al. | Oct 2006 | A1 |
| 20070029232 | Cowling | Feb 2007 | A1 |
| 20080029445 | Russcher et al. | Feb 2008 | A1 |
| 20080041501 | Platek et al. | Feb 2008 | A1 |
| 20080092922 | Dick | Apr 2008 | A1 |
| 20080257795 | Shuttleworth | Oct 2008 | A1 |
| 20090292422 | Eiswerth et al. | Nov 2009 | A1 |
| 20100017020 | Hubbard-Nelson et al. | Jan 2010 | A1 |
| 20100195795 | Golenhofen | Aug 2010 | A1 |
| 20100264070 | Sommer, Jr. et al. | Oct 2010 | A1 |
| 20100282646 | Looy et al. | Nov 2010 | A1 |
| 20110083871 | Lalancette et al. | Apr 2011 | A1 |
| 20110247730 | Yanar | Oct 2011 | A1 |
| 20120148018 | Sommer, Jr. et al. | Jun 2012 | A1 |
| 20120288058 | Maeyama et al. | Nov 2012 | A1 |
| 20130028487 | Stager et al. | Jan 2013 | A1 |
| 20130079918 | Spencer et al. | Mar 2013 | A1 |
| 20130092609 | Andersen | Apr 2013 | A1 |
| 20130126399 | Wolff et al. | May 2013 | A1 |
| 20130184853 | Roos | Jul 2013 | A1 |
| 20130229510 | Killmann | Sep 2013 | A1 |
| 20130264249 | Sommer, Jr. et al. | Oct 2013 | A1 |
| 20130304254 | Torek et al. | Nov 2013 | A1 |
| 20150012226 | Skaff | Jan 2015 | A1 |
| 20150092922 | Liu et al. | Apr 2015 | A1 |
| 20150170024 | Chatterjee et al. | Jun 2015 | A1 |
| 20150336135 | Corak | Nov 2015 | A1 |
| 20160016201 | Schons | Jan 2016 | A1 |
| 20160022892 | Eifler et al. | Jan 2016 | A1 |
| 20160066860 | Sternickel et al. | Mar 2016 | A1 |
| 20160180626 | Young | Jun 2016 | A1 |
| 20160299091 | Bamber et al. | Oct 2016 | A1 |
| 20160346811 | Iino | Dec 2016 | A1 |
| 20170014868 | Garcia, Jr. | Jan 2017 | A1 |
| 20170221246 | Zhong et al. | Aug 2017 | A1 |
| 20170232479 | Pietzka et al. | Aug 2017 | A1 |
| 20170261437 | Buchter et al. | Sep 2017 | A1 |
| 20180243800 | Kumar et al. | Aug 2018 | A1 |
| 20190247891 | Kumar et al. | Aug 2019 | A1 |
| 20190299255 | Chaganti | Oct 2019 | A1 |
| 20200050922 | Wu et al. | Feb 2020 | A1 |
| 20200084966 | Corban et al. | Mar 2020 | A1 |
| 20200361659 | Whitman et al. | Nov 2020 | A1 |
| 20200368786 | Kumar et al. | Nov 2020 | A1 |
| 20210094075 | Horowitz et al. | Apr 2021 | A1 |
| 20210217156 | Balachandran et al. | Jul 2021 | A1 |
| 20210229133 | Kumar et al. | Jul 2021 | A1 |
| 20210346916 | Kumar et al. | Nov 2021 | A1 |
| Number | Date | Country |
|---|---|---|
| 2893877 | Dec 2015 | CA |
| 1283319 | Feb 2001 | CN |
| 200953004 | Sep 2007 | CN |
| 201440132 | Apr 2010 | CN |
| 201464390 | May 2010 | CN |
| 101776620 | Jul 2010 | CN |
| 201552461 | Jul 2010 | CN |
| 102861722 | Jan 2013 | CN |
| 103501925 | Jan 2014 | CN |
| 103745901 | Apr 2014 | CN |
| 101776620 | Jun 2014 | CN |
| 103955707 | Jul 2014 | CN |
| 203688493 | Jul 2014 | CN |
| 204359695 | May 2015 | CN |
| 204470139 | Jul 2015 | CN |
| 204495749 | Jul 2015 | CN |
| 204537711 | Aug 2015 | CN |
| 204575572 | Aug 2015 | CN |
| 104969266 | Oct 2015 | CN |
| 106000904 | Oct 2016 | CN |
| 107403198 | Nov 2017 | CN |
| 107552412 | Jan 2018 | CN |
| 107790398 | Mar 2018 | CN |
| 21188005 | Nov 2020 | CN |
| 113272649 | Aug 2021 | CN |
| 202009006383 | Sep 2009 | DE |
| 0011892 | Nov 1983 | EP |
| 0074447 | Jan 1987 | EP |
| 0433828 | Dec 1990 | EP |
| 0351778 | Oct 1993 | EP |
| 2243089 | Oct 2010 | EP |
| 3263234 | Jan 2018 | EP |
| H07-275802 | Oct 1995 | JP |
| 2010-172799 | Aug 2010 | JP |
| 5083196 | Nov 2012 | JP |
| 2015-512075 | Apr 2015 | JP |
| 5969685 | Jul 2016 | JP |
| 2017-109197 | Jun 2017 | JP |
| 202163078 | Oct 2021 | JP |
| 20090106056 | Oct 2009 | KR |
| 2004101401 | Feb 2005 | RU |
| 2006136756 | Apr 2008 | RU |
| 2339974 | Nov 2008 | RU |
| 2361194 | Jul 2009 | RU |
| 200940989 | Oct 2009 | TW |
| WO9920048 | Apr 1999 | WO |
| WO2001022072 | Mar 2001 | WO |
| WO2009039284 | Mar 2009 | WO |
| WO2011159269 | Dec 2011 | WO |
| WO2012094568 | Jul 2012 | WO |
| WO2013033572 | Mar 2013 | WO |
| WO2013180922 | Dec 2013 | WO |
| WO2015195988 | Dec 2015 | WO |
| WO2016199074 | Dec 2016 | WO |
| WO2017001438 | Jan 2017 | WO |
| WO2017011835 | Jan 2017 | WO |
| WO2017221246 | Dec 2017 | WO |
| WO2021089602 | May 2021 | WO |
| WO2021126876 | Jun 2021 | WO |
| WO2019180438 | Sep 2022 | WO |
| Entry |
|---|
| United States International Searching Authority; International Search Report & Written Opinion for PCT/us2022/020657; Jun. 16, 2022; 10 pages; Alexandria, VA. |
| European Patent Office; Extended European Search Report for corresponding EP 19792330.3; Apr. 30, 2021; 7 pages; Munich, DE. |
| India Patent Office; Office Action issued for corresponding India Application Serial No. 201937044046; Jun. 4, 2020; 7 pages; IN. |
| United States International Searching Authority; International Search Report & Written Opinion for PCT/US2016/042850; Sep. 28, 2016; 15 pages; Alexandria, VA; US. |
| International Alloy Designations and Chemical Composition Limits for Wrought Aluminum and Wrought Aluminum Alloys, The Aluminum Association, Inc., revised Jan. 2015, 38 pages. |
| International Searching Authority, International Search Report and the Written Opinion, International Application No. PCT/US2016/042850, Sep. 28, 2016. |
| P. R. Schwoebel et al., “Studies of a prototype linear stationary x-ray source for tomosynthesis imaging,” Phys. Med Biol. 59, pp. 2393-2413, Apr. 17, 2014. |
| R. Sitko et al., “Quantification in X-Ray Fluorescence Spectrometry,” X-Ray Spectroscopy, Dr. Shatendra K Sharma (Ed.), ISBN: 978-953-307-967-7, InTech, 2012, pp. 137-163; Available from: http://www.intechopen.com/books/x-ray-spectroscopy/quantification-in-x-ray-fluorescence-spectrometry. |
| Scrap Specifications Circular, Institute of Scrap Recycling Industries, Inc., effective Jan. 21, 2016, 58 pages. |
| The International Bureau of WIPO, International Preliminary Report on Patentability, International Application No. PCT/US2016/42850, Jan. 25, 2018. |
| A. Lee, “Comparing Deep Neural Networks and Traditional Vision Algorithms in Mobile Robotics,” Swarthmore College, 9 pages, downloaded from Internet on May 1, 2018. |
| C. K. Lowe et al., “Data Mining With Different Types of X-Ray Data,” JCPDS-International Centre for Diffraction Data 2006, ISSN 1097-0002, pp. 315-321. |
| M. Razzak et al., “Deep Learning for Medical Image Processing: Overview, Challenges and Future,” 30 pages, downloaded from Internet on May 1, 2018. |
| J. Schmidhuber et al., “Deep Learning in Neural Networks: An Overview,” The Swiss AI Lab IDSIA, Technical Report IDSIA-03-14/arXiv:1404.7828 v4 [cs.NE], Oct. 8, 2014, 88 pages. |
| M. Singh et al., “Transforming Sensor Data to the Image Domain for Deep Learning—an Application to Footstep Detection,” International Joint Conference on Neural Networks, Anchorage, Alaska, 8 pages, May 14-19, 2017. |
| K. Tarbell et al., “Applying Machine Learning to the Sorting of Recyclable Containers,” University of Illinois at Urbana-Champaign, Urbana, Illinois, 7 pages, downloaded from Internet on May 1, 2018. |
| Wikipedia, Convolutional neural network, 18 pages https://en.wikipedia.org/w/index.php?title=Convolutional_neural_network, downloaded from Internet on May 1, 2018. |
| Wikipedia, TensorFlow, 4 pages https://en.wikipedia.org/w/index.php?title=TensorFlow&oldid=835761390, downloaded from Internet on May 1, 2018. |
| The United States Patent and Trademark Office, Non-Final Office Action, U.S. Appl. No. 15/213,129, filed Oct. 6, 2017. |
| International Searching Authority, International Search Report and the Written Opinion, International Application No. PCT/US2018/029640, Jul. 23, 2018; 23 pages; Alexandria, VA; US. |
| European Patent Office; Extended Search Report for 16825313.6; Jan. 28, 2019; 12 pages; Munich, DE. |
| India Patent Office; Office Action issued for corresponding India Application Serial No. 201817002365; Mar. 12, 2020; 6 pages; IN. |
| International Searching Authority, International Search Report and the Written Opinion, International Application No. PCT/US2019/022995, Jun. 5, 2019; 10 pages; Alexandria, VA; US. |
| T. Miller et al., “Elemental Imaging for Pharmaceutical Tablet Formulations Analysis by Micro X-Ray Fluorescence,” International Centre for Diffraction Data, 2005, Advances in X-ray Analysis, vol. 48, pp. 274-283. |
| T. Moriyama, “Pharmaceutical Analysis (5), Analysis of trace impurities in pharmaceutical products using polarized EDXRF spectrometer NEX CG,” Rigaku Journal, vol. 29, No. 2, 2013, pp. 19-21. |
| U.S. Appl. No. 15/213,129, filed Jul. 18, 2016. |
| M. Baudelet et al., “The first years of laser-induced breakdown spectroscopy,” J. Anal. At. Spectrom., Mar. 27, 2013, 6 pages. |
| International Searching Authority, International Search Report and the Written Opinion of the International Searching Authority, International Application No. PCT/US2016/45349, Oct. 17, 2016. |
| J. McComb et al., “Rapid screening of heavy metals and trace elements in environmental samples using portable X-ray fluorescence spectrometer, A comparative study,” Water Air Soil Pollut., Dec. 2014, vol. 225, No. 12, pp. 1-16. |
| J. Mondia, “Using X-ray fluorescence to measure inorganics in biopharmaceutical raw materials,” Anal. Methods, Mar. 18, 2015, vol. 7, pp. 3545-3550. |
| L. Goncalves, “Assessment of metal elements in final drug products by wavelength dispersive X-ray fluorescence spectrometry,” Anal. Methods, May 19, 2011, vol. 3, pp. 1468-1470. |
| L. Hutton, “Electrochemical X-ray Fluorescence Spectroscopy for Trace Heavy Metal Analysis: Enhancing X-ray Fluorescence Detection Capabilities by Four Orders of Magnitude,” Analytical Chemistry, Apr. 4, 2014, vol. 86, pp. 4566-4572. |
| L. Moens et al., Chapter 4, X-Ray Fluorescence, Modern Analytical Methods in Art and Archaeology, Chemical Analysis Series, vol. 155, pp. 55-79, copyright 2000. |
| H. Rebiere et al., “Contribution of X-Ray Fluorescence Spectrometry for the Analysis of Falsified Products,” ANSM, The French National Agency for Medicines and Health Products Safety, Laboratory Controls Division, France, 1 page, (date unknown). |
| B. Shaw, “Applicability of total reflection X-ray fluorescence (TXRF) as a screening platform for pharmaceutical inorganic impurity analysis,” Journal of Pharmaceutical and Biomedical Analysis, vol. 63, 2012, pp. 151-159. |
| Briefing Elemental Impurities-Limits, Revision Bulletin, The United States Pharmacopeial Convention, Feb. 1, 2013, 3 pages. |
| Chapter 6, Functional Description, S2 Picofox User Manual, 2008, pp. 45-64. |
| D. Bradley, “Pharmaceutical toxicity: AAS and other techniques measure pharma heavy metal,” Ezine, May 15, 2011, 2 pages. |
| E. Margui et al., “Determination of metal residues in active pharmaceutical ingredients according to European current legislation by using X-ray fluorescence spectrometry,” J. Anal. At. Spectrom., Jun. 16, 2009, vol. 24, pp. 1253-1257. |
| Elemental Impurity Analysis in Regulated Pharmaceutical Laboratories, a Primer, Agilent Technologies, Jul. 3, 2012, 43 pages. |
| Exova, X-ray fluorescence: a new dimension to elemental analysis, downloaded from www.exova.com on Jul. 26, 2016, 3 pages. |
| G. O'Neil, “Direct Identification and Analysis of Heavy Metals in Solution (Hg, Cu, Pb, Zn, Ni) by Use of in Situ Electrochemical X-ray Fluorescence,” Analytical Chemistry, Feb. 2015, 22 pages. |
| Guideline for Elemental Impurities, Q3D, International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use, ICH Harmonised Guideline, Current Step 4 version, Dec. 16, 2014, 77 pages. |
| The United States Patent and Trademark Office, Non-Final Office Action, U.S. Appl. No. 16/375,675, filed Jun. 28, 2019. |
| The United States Patent and Trademark Office, Final Office Action, U.S. Appl. No. 16/375,675, filed Jan. 17, 2020. |
| “Alloy Data: Aluminum Die Casting Alloys,” MES, Inc., 4 pages, downloaded from the internet Mar. 28, 2019, www.mesinc.com. |
| C.O. Augustin et al., “Removal of Magnesium from Aluminum Scrap and Aluminum-Magnesium Alloys,” Bulletin of Electrochemistry 2(6), Nov.-Dec. 1986; pp. 619-620. |
| E.A. Vieira et al., “Use of Chlorine to Remove Magnesium from Molten Aluminum,” Materials Transactions, vol. 53, No. 3, pp. 477-482, Feb. 25, 2012. |
| Skpecim Spectral Imaging; Hyperspectral Technology vs. RGB; at least as early as Mar. 9, 2021; 3 pages; Oulu, Finland. |
| Wikipedia; Digital image processing; Retrieved from https://en.wikipedia.org/w/index.php?title=Digital_image_processing&oldid=1015648152; at least as early as Apr. 2, 2021; Wikimedia Foundation, Inc.; US. |
| Wikipedia; Machine vision; Retrieved from https://en.wikipedia.org/w/index.php?title=Machine_vision&oldid= 1021673757; at least as early as May 6, 2021; Wikimedia Foundation, Inc.; US. |
| ACLIMA; How artificial intelligence helps recycling become more circular; retrieved frfom https://aclima.eus/how-artificial-intelligence-helps-recycling-become-more-circular; Jan. 27, 2022; (5 pages); VE. |
| Guillory et al; Analysis of Multi-layer polymer films; Apr. 2009; vol. 12; No. 4, Materials Today; 2 pages; Elseview; NL. |
| U.S. Department of Energy; Waste-to-Energy From Municipal Solid Wastes; Aug. 2019; 36 pages; US. |
| SAPP; Lehigh University scores $3.5M DOE Grant to use AI and spectroscopy to analyze waste materials; Biofuelsdigest.com; retrieved from https://www.biofuelsdigest.com/bdigest/2021/09/27/lehigh-university-scores-3-5m-doe-grant-to-use-ai-and-spectroscopy-to-analyse-waste-materials/; 1 page; Sep. 27, 2021; US. |
| Energy Information Administration; Methodoly for Allocating Municipal solid Waste to Biogenic and Non-Biogenic Energy; May 2007; 18 pages; US. |
| Fadillah et al.; Recent Progress in Low-Cost Catalysts for Pyrolysis of Plastic Waste to Fuels; 17 pages; Catalysts 2021, 11, 837; Jul. 10, 2021; mdpi.com; Basel, Switzerland. |
| Lukka et al; Robotic Sorting using Machine Learing; ZenRobotics Recycler; Sensor Based Sorting 2014; https://users.ics.aalto.fi > praiko > papers > SBS14; US. |
| Wikipedia; Plastic recycling; Retrieved from “https://en.wikipedia.org/w/index.php?title=Plastic_recycling&oldid=1067847730” Jan. 25, 2022; Wikimedia Foundation, Inc; US. |
| AIMPLAS; Classification and identification of plastics; https://www.aimplas.net/blog/plastics-identification-and-classification/; Jan. 27, 2022; 4 pages; València Parc Tecnològic; Valencia; Spain. |
| SPECIM; Plastics Sorting with SPECIM FX Cameras; 5 pages; Dec. 18, 2020; Oulu, FI. |
| Ruj et al; Sorting of plastic waste for effective recycling; Int. Journal of Applied Sciences and Engineering Research, vol. 4, Issue 4, 2015; 8 pages; International Journal of Applied Science and Engineering Research; ijaser.com; New Delhi, IN. |
| Wikipedia; Spectral imaging; Retrieved from “https://en.wikipedia.org/w/index.php?title=Spectral_imaging&oldid=1066379790”; Jan. 18, 2022; Wikimedia Foundation, Inc; US. |
| Acplasics, Inc.; 7 Different Types of Plastic; https://www.acplasticsinc.com/informationcenter/r/7-different-types-of-plastic-and-how-they-are-used; 4 pages; Jan. 27, 2022; A&C Plastics; Houston, TX. |
| Taiwan Patent Office; Allowance of corresponding Application No. 111112261; 3 pages; Oct. 4, 2023; Da'an Dist., Taipei City 106213, Taiwan. |
| Rozenstein, O. et al; Development of a new approach based on midwave infrared spectroscopy for post-consumer black plastic waste sorting in the recycling industry; Waste Management 68 (2017; pp. 38-44; abstract. |
| United States International Searching Authority; International Search Report & Written Opinion for PCT/US2022/015693; May 6, 2022; 9 pages; Alexandria, VA; US. |
| United States International Searching Authority; International Search Report & Written Opinion for PCT/US2022/015665; May 23, 2022; 10 pages; Alexandria, VA; US. |
| Chinese Patent Office; Office Action issued for corresponding Chinese Application No. 201980043725.X on Apr. 28, 2022; 21 pages; Beijing, CN. |
| Bishop, Christopher M.; Neural Networks for Pattern Recognition; 494 pages; Clarendon Press; 1995; Oxford, UK. |
| Zhou et al; SSA-CNN: Semantic Self-Attention CNN for Pedestrian Detetion:; arXiv: 1902.09080v3 [cs. CF] Jun. 6, 2019. Retrieved o Oct. 10, 2022; Retrieved from <URL: https://arxivorg/pdf/1902.09080.pdf>. |
| Jones et al., “Safe Steering Wheel Airbag Removal Using Active Disassemly”:, DS 30; Proceedings of Design 2002, the 7th International Design Converence, dubrovnik, Retrieved on Jul. 10, 2022, from <https://desigsociety.org/publiatio/29632/Safe+Steering+Wheel+Airbag+Removal+Using+Active+Dissembly>. |
| Zhang, et al.; Designing and verifying a disassembly line approach to cope with the upsurge of end-of-life vehicles in China:, Elsevier, Waste Management 2018, Retrieved on Jul. 10, 2022 from <https://isiarticles.com/budles/Article/pre/pdf/98926.pdf>. |
| Japan Patent Office; Office Action issued Jan. 10, 2023 for Serial No. 2021-509947; 9 pages (with translation). |
| GAO; Applying Improved Optical Recognition with Machine Learning on Sorting Cu Impurities in Steel Scrap; 1-12, Journal of Sustainable Metallurgy, Web, Dec. 7, 2020. |
| Taiwan Patent Office; Office Action for Serial No. 111112261; May 23, 2023; 9 pgs (w/translation); TW. |
| Number | Date | Country | |
|---|---|---|---|
| 20220203407 A1 | Jun 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| 63285964 | Dec 2021 | US | |
| 63249069 | Sep 2021 | US | |
| 63173301 | Apr 2021 | US | |
| 63146892 | Feb 2021 | US | |
| 62490219 | Apr 2017 | US | |
| 62193332 | Jul 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16358374 | Mar 2019 | US |
| Child | 16852514 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17380928 | Jul 2021 | US |
| Child | 17495291 | US | |
| Parent | 16375675 | Apr 2019 | US |
| Child | 16939011 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 17667397 | Feb 2022 | US |
| Child | 17696831 | US | |
| Parent | 17495291 | Oct 2021 | US |
| Child | 17667397 | US | |
| Parent | 17227245 | Apr 2021 | US |
| Child | 17380928 | US | |
| Parent | 16939011 | Jul 2020 | US |
| Child | 17227245 | US | |
| Parent | 15963755 | Apr 2018 | US |
| Child | 16375675 | US | |
| Parent | 15213129 | Jul 2016 | US |
| Child | 15963755 | US | |
| Parent | 17491415 | Sep 2021 | US |
| Child | 17495291 | Oct 2021 | US |
| Parent | 16852514 | Apr 2020 | US |
| Child | 17491415 | US | |
| Parent | 15963755 | Apr 2018 | US |
| Child | 16358374 | US |
