Patent Grant
12082531
The present descriptions relates to mobile machines, particularly mobile machines configured to carry and transport material.
There are a wide variety of different mobile machines. Some mobile machines carry, receive, and transport materials. For example, an agricultural harvester includes a material receptacle, such as an on-board grain tank, which receives and holds harvested crop material. In other examples, a material transport machine includes a towing vehicle, such as a truck or a tractor, and a towed material receptacle, such as a grain cart or trailer. The towed material receptacle receives and holds material, such as harvested crop material, and is transported by the towing vehicle.
The discussion above is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.
One or more information maps are obtained by an agricultural system. The one or more information maps map one or more characteristic values at different geographic locations in a worksite. An in-situ sensor detects a material dynamics characteristic value as a mobile machine operates at the worksite. A predictive map generator generates a predictive map that predicts a predictive material dynamics characteristic value at different geographic locations in the worksite based on a relationship between the values in the one or more information maps and the material dynamics characteristic value detected by the in-situ sensor. The predictive map can be output and used in automated machine control.
Example 1 is an agricultural system comprising:
Example 2 is the agricultural system of any or all previous examples, wherein the predictive map generator configures the functional predictive material dynamics map for consumption by a control system that generates control signals to control a controllable subsystem on the mobile machine based on the functional predictive material dynamics map.
Example 3 is the agricultural system of any or all previous examples, wherein the in-situ sensor detects, as the value of the material dynamics characteristic, a value of a material spill characteristic;
Example 4 is the agricultural system of any or all previous examples, wherein the in-situ sensor detects, as the value of the material dynamics characteristic, a value of a material movement characteristic;
Example 5 is the agricultural system of any or all previous examples, wherein the information map comprises a terrain map that maps, as the values of the characteristic, terrain values to the different geographic locations in the worksite, and wherein the predictive model generator is configured to identify a relationship between terrain values and material dynamics characteristic values based on the material dynamics characteristic value detected at the geographic location and the terrain value, in the terrain map, at the geographic location, the predictive material dynamics model being configured to receive a terrain value as a model input and generate a material dynamics characteristic value as a model output based on the identified relationship.
Example 6 is the agricultural system of any or all previous examples, wherein the information map comprises a speed map that maps, as the values of the characteristic, speed values to the different geographic locations in the worksite, and wherein the predictive model generator is configured to identify a relationship between speed values and material dynamics characteristic values based on the material dynamics characteristic value detected at the geographic location and the speed value, in the speed map, at the geographic location, the predictive material dynamics model being configured to receive a speed value as a model input and generate a material dynamics characteristic value as a model output based on the identified relationship.
Example 7 is the agricultural system of any or all previous examples, wherein the information map comprises a crop moisture map that maps, as the values of the characteristic, crop moisture values to the different geographic locations in the worksite, and wherein the predictive model generator is configured to identify a relationship between crop moisture values and material dynamics characteristic values based on the material dynamics characteristic value detected at the geographic location and the crop moisture value, in the crop moisture map, at the geographic location, the predictive material dynamics model being configured to receive a crop moisture value as a model input and generate a material dynamics characteristic value as a model output based on the identified relationship.
Example 8 is the agricultural system of any or all previous examples, wherein the information map comprises a fill level map that maps, as the values of the characteristic, fill level values to the different geographic locations in the worksite, and wherein the predictive model generator is configured to identify a relationship between fill level values and material dynamics characteristic values based on the material dynamics characteristic value detected at the geographic location and the fill level value, in the fill level map, at the geographic location, the predictive material dynamics model being configured to receive a fill level value as a model input and generate a material dynamics characteristic value as a model output based on the identified relationship.
Example 9 is the agricultural system of any or all previous examples, wherein the information map comprises two or more information maps, wherein each one of the two or more information maps include values of a respective characteristic corresponding to different geographic locations in the worksite, wherein the two or more information maps comprise two or more of:
Example 10 is the agricultural system of any or all previous examples, wherein the predictive map generator generates the functional predictive material dynamics map of the worksite, that maps predictive values of the material dynamics characteristic to the different geographic locations in the worksite, based on two or more of the terrain values, the speed values, the crop moisture values, and the fill level values in the respective information maps and based on the predictive material dynamics model.
Example 11 is the agricultural system of any or all previous examples, wherein the predictive map generator configures the functional predictive material dynamics map for consumption by a control system that generates controls signals to control one or more controllable subsystems on a mobile machine based on the functional predictive material dynamics map, wherein the one or more controllable subsystems comprises one or more of:
Example 12 is a computer implemented method of generating a functional predictive material dynamics map, comprising:
Example 13 is the computer implemented method of any or all previous examples, and further comprising:
Example 14 is the computer implemented method of any or all previous examples, wherein detecting, with an in-situ sensor, the value of the material dynamics characteristic comprises one of detecting a value of a material spill characteristic or detecting a value of a material movement characteristic.
Example 15 is the computer implemented method of any or all previous examples, wherein receiving the information map comprises receiving a terrain map that maps, as the values of the characteristic, terrain values to the different geographic locations in the worksite, and wherein generating the predictive material dynamics model comprises:
Example 16 is the computer implemented method of any or all previous examples, wherein receiving the information map comprises receiving a speed map that maps, as the values of the characteristic, speed values to the different geographic locations in the worksite, and wherein generating the predictive material dynamics model comprises:
Example 17 is the computer implemented method of any or all previous examples, wherein receiving the information map comprises receiving a crop moisture map that maps, as the values of the characteristic, crop moisture values to the different geographic locations in the worksite, and wherein generating the predictive material dynamics model comprises:
Example 18 is the computer implemented method of any or all previous examples, wherein receiving the information map comprises receiving a fill level map that maps, as the values of the characteristic, fill level values to the different geographic locations in the worksite, and wherein generating the predictive material dynamics model comprises:
Example 19 is the computer implemented method of any or all previous examples, further comprising:
Example 20 is a mobile machine comprising:
This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. The claimed subject matter is not limited to implementations that solve any or all disadvantages noted in the background.
For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the examples illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the disclosure is intended. Any alterations and further modifications to the described devices, systems, methods, and any further application of the principles of the present disclosure are fully contemplated as would normally occur to one skilled in the art to which the disclosure relates. In particular, it is fully contemplated that the features, components, and/or steps described with respect to one example may be combined with the features, components, and/or steps described with respect to other examples of the present disclosure.
In one example, the present description relates to using in-situ data taken concurrently with an operation to generate a predictive model, such as a predictive material dynamics model, that models a relationship between different characteristics (or values thereof) represented in the in-situ data. In some examples, the predictive material dynamics model can be used to control a mobile machine. In one example, the predictive material dynamics model is in the form of a predictive material movement model. In another example, the predictive material dynamics model is in the form of a predictive material spill model.
In other examples, the present description relates to using in-situ data taken concurrently with an operation, in combination with prior or predicted data, such as prior or predicted data represented in a map, to generate a predictive model and a predictive map, such as a predictive material dynamics model and a predictive material dynamics map. In one example, the predictive material dynamics model is in the form of a predictive material movement model and the predictive material dynamics map is in the form of a predictive material movement map. In another example, the predictive material dynamics model is in the form of a predictive material spill model and the predictive material dynamics map is in the form of a predictive material spill map. In some examples, the predictive material dynamics map can be used to control a mobile machine.
As discussed above, some mobile machines receive, hold (e.g., carry), and transport material, such as harvested crop material (e.g., grain). These mobile machines include a material receptacle, such as grain tank or bin. As the mobile machine operates across a worksite, such as a field, the material held by the machines can have varying dynamics, for example, the material can shift its position in the material receptacle or can spill out of the material receptacle, or both. The shifting movement of the material can have deleterious effects on the performance of the mobile machine, such as creating instability of the mobile machine, increasing load on some components which can, among other things, affect traction, affect compaction, as well as affect various other performance factors. Additionally, spillage of the material can reduce profitability of the operation, increase waste, increase cost, such as from supplemental cleanup operations, as well as lead to undesired planting of volunteer crops. These are just some examples of the deleterious effects of material movement and material spillage.
Dynamics of the material, such as movement and spillage of the material, may be affected or caused by various factors. For example, operation of the machine, such as travel speed, acceleration, and deceleration can affect or cause material movement and spillage. Additionally, worksite factors, such as terrain characteristics, for instance, topography and slope can affect or cause material movement and spillage. For example, the topography and slope of the worksite affects the orientation (e.g., pitch and roll) of the mobile machine, which can affect or cause material movement and spillage. Further, material factors, such as moisture and size of the material, can affect or cause material movement and spillage. For example, the moisture and size of the material, such as the moisture and size of grain, can affect the angle of repose for a pile of material as it sits in the material receptacle. Thus, an increased or decreased angle of repose can make material movement and spillage more or less likely. These are merely some examples of the factors that can affect or cause movement and spillage of material.
In one example, the present description relates to material dynamics characteristics and values thereof, such as material spill characteristics and material spill characteristic values or material movement characteristics and material movement characteristic values, or both. Thus, as used herein, material spill characteristics and material movement characteristics are material dynamics characteristics.
In one example, the present description relates to obtaining a map such as a terrain map. The terrain map includes geolocated values of terrain characteristics (terrain characteristic values, sometimes referred to herein as terrain values) across different locations at a worksite. For example, the terrain map can include elevation values indicative of the elevation of the worksite at various locations, as well as slope values indicative of the slope of the worksite at various locations. Additionally, the terrain map can include machine orientation values indicative of the pitch or roll, or both, of the mobile machine at various locations at the worksite. The terrain map, and the values therein, can be based on historical data, such as data from previous operations at the worksite by the same mobile machine or by a different mobile machine. The terrain map, and the values therein, can be based on fly-over or satellite-based sensor data, such as LIDAR data of the worksite, as well as scouting data provided by a user or operator such as from a scouting operation of the worksite. The terrain map, and the values therein, can be a predictive terrain map with predictive terrain values, for instance, the machine orientation values at the different locations can be predicted based on the elevation and/or slope values at those different locations and based on the dimensions of the mobile machine. The terrain map, and the values therein, can be a combination of the aforementioned. These are merely some examples.
In one example, the present descriptions relate to obtaining a map such as a speed map. The speed map includes geolocated values of speed characteristics (speed characteristic values, sometimes referred to herein as speed values) of the mobile machine, such as travel speed values, acceleration values, and deceleration values. The speed map, and the values therein, can be a prescribed speed map that includes speed values that are prescribed for the particular operation, such as by an operator or user or a control system. The speed map, and the values therein, can be based on historical speed values from prior operations at the worksite such as prior operation by the same mobile machine or a different mobile machine. The speed map, and the values therein, can be a predictive speed map with predictive speed values. In one example, the predictive speed map is generated by obtaining a map, such as a yield map of the worksite, and a sensed speed, a sensed acceleration, and/or a sensed deceleration of the agricultural machine (such as speed data obtained from a data signal from a speed sensor) and determining a relationship between the obtained map, and the values therein, and the sensed speed data. The determined relationship, in combination with the obtained map, is used to generate a predictive speed map having predictive speed values. The speed map, and the values therein, can be a combination of the aforementioned. These are merely examples.
In one example, the present description relates to obtaining a map such as a crop moisture map. The crop moisture map includes geolocated crop moisture values of crop at the worksite. The crop moisture map, and the values therein, can be based on historical crop moisture values from prior operations at the worksite such as prior operations by the same mobile machine or a different mobile machine. The crop moisture map, and the values therein, can be a predictive crop moisture map with predictive crop moisture values. In one example, the predictive crop moisture values can be based on vegetative index (VI) values of the field, such as normalized different vegetative index (NDVI) values or leaf area index (LAI) values, generated during a survey of the field, such as a fly-over or satellite-based survey of the field. In another example, the predictive crop moisture map is generated by obtaining a map, such as a vegetative index map of the worksite, and a sensed crop moisture (such as crop moisture data obtained from a data signal from a crop moisture sensor) and determining a relationship between the obtained map, and the values therein, and the sensed crop moisture data. The determined relationship, in combination with the obtained map, is used to generate a predictive crop moisture map having predictive crop moisture values. The crop moisture map, and the values therein, can be a combination of the aforementioned. These are merely examples.
In one example, the present description relates to obtaining a map such as a fill level map. The fill level map includes geolocated fill level values of the material receptacle of the mobile machine at the worksite. The fill level map, and the values therein, can be based on other characteristic values of the worksite, such as yield values (e.g., as derived from NDVI data of the field) along with parameters of the mobile machines, such as a commanded travel path and capacity of the material receptacle. In one example, the predictive fill level map is generated by obtaining a map, such as a yield map of the worksite, and sensed fill level (such as fill level data obtained from a data signal from a fill level sensor, or other sensor) and determining a relationship between the obtained map, and the values therein, and the sensed fill level data. The determined relationship, in combination with the obtained map, is used to generate a predictive fill level map having predictive fill level values. In another example, the predictive fill level map is generated by obtaining a map, such as a yield map of the worksite having yield values for various locations at the worksite, and a sensed current fill level of the material receptacle of the mobile machine at a given location. Based on the current fill level and the yield value(s) (as provided by the yield map) in areas around the mobile machine, such as in areas in front of the mobile machine relative to its travel path, a predictive fill level map is generated that predicts fill level(s) in areas around the mobile machine, such as in areas ahead of mobile machine relative to its travel path. The fill level map, and the values therein, can be a combination of the aforementioned. These are merely examples.
In one example, the present description relates to obtaining in-situ data from in-situ sensors on the mobile machine taken concurrently with an operation. The in-situ sensor data can include one or more of: in-situ machine orientation sensor data, such as pitch and roll of the mobile machine; in-situ speed sensor data, such as sensed travel speed, sensed acceleration, and/or sensed deceleration of the mobile machine; in-situ crop moisture data, such as sensed moisture of crop collected by the mobile machine; in-situ material mass data, such as sensed mass of material in a material receptacle of the mobile machine; in-situ material center of mass data, such as sensed center of mass of material in a material receptacle of the mobile machine; in-situ fill level data, such as a sensed fill level of a material receptacle; in-situ material dynamics data such as one or more of in-situ material spill data, such as sensed spillage of material out of a material receptacle of the mobile machine or sensed spillage of material during a material transfer operation, and in-situ material movement data, such as sensed movement of material in a material receptacle of the mobile machine. The various in-situ data is derived from various in-situ sensors on the mobile machine, as will be described in further detail herein. These are merely some examples of the in-situ data that can be obtained.
The present discussion proceeds, in some examples, with respect to systems that receive in-situ data and generate a predictive model that models a relationship between the in-situ data. For example, the predictive model models a relationship between one or more of in-situ machine orientation data, in-situ speed data, in-situ crop moisture data, in-situ material mass data, in-situ material center of mass data, and in-situ fill level data and in-situ material dynamics data, such as in-situ material movement data or in-situ material spill data, to generate a predictive material dynamics model, such as a predictive material dynamics model in the form of a predictive movement model or a predictive material spill model. In some examples, the predictive material dynamics model can be used for control of a mobile machine. In some examples, the systems obtain one or more maps of a worksite (such as one or more of a terrain map, speed map, crop moisture map, and fill level map) and use the predictive material dynamics model to generate a predictive map that predicts, for example, material dynamics such as movement of material in a material receptacle or spillage of material from a material receptacle.
The present discussion proceeds, in some examples, with respect to systems that obtain one or more maps of a worksite (such as one or more of a terrain map, speed map, crop moisture map, and fill level map) and also use an in-situ sensor to detect a variable indicative of an agricultural characteristic, such as material dynamics characteristics such as movement of material in a material receptacle or spillage of material from a material receptacle. The systems generate a model that models a relationship between the values on the obtained map(s) and the output values from the in-situ sensor. The model is used to generate a predictive map that predicts, for example, material dynamics such as movement of material in a material receptacle or spillage of material from a material receptacle. The predictive map, generated during an operation, can be presented to an operator or other user or used in automatically controlling a mobile machine during an operation or both. In some examples, the predictive map can be used to control one or more of a travel speed of the mobile machine, acceleration of the mobile machine, deceleration of the mobile machine, a route of the mobile machine, steering of the mobile machine, a material fill level of a material receptacle of the mobile machine, an amount of material transferred to or from a mobile machine, as well as various other parameters.
While the various examples described herein proceed with respect to mobile agricultural machines, such as agricultural harvesters, and agricultural material transport machines, such as towed grain carts and towed trailers, it will be appreciated that the systems and methods described herein are applicable to various other mobile machines, various other machine operations, as well as various other materials, for example forestry machines, forestry operations, and forestry materials, constructions machines, construction operations, and construction materials, and turf management machines, turf management operations, and turf management materials. Additionally, it will be appreciated that the systems and methods described herein are applicable to various mobile agricultural machines, for example, but not by limitation, agricultural harvesters, dry material spreaders, seeding and planting machines, as well as various other mobile agricultural machines configured to receive, hold, and transport material(s). For illustration, but not by limitation, a dry material spreader can include a dry material receptacle that receives, holds, and transports dry material, such as dry fertilizer, that is to be spread on a worksite. This dry material may spill out of or move within the dry material receptacle during operation of the dry material spreader.
Agricultural harvester 101 includes a set of front-end machines forming a cutting platform 102 that includes a header 104 having a cutter generally indicated at 106. It can also include a feeder house 108, a feed accelerator 109, and a thresher generally indicated at 111. Thresher 111 illustratively includes a threshing rotor 112 and a set of concaves 114. Further, agricultural harvester 101 can include a separator 116 that includes a separator rotor. Agricultural harvester 101 can include a cleaning subsystem (or cleaning shoe) 118 that, itself, can include a cleaning fan 120, a chaffer 122 and a sieve 124. The material handling subsystem in agricultural harvester 101 can include (in addition to a feeder house 108 and feed accelerator 109) discharge beater 126, tailings elevator 128, and clean grain elevator 130 (that moves clean grain into clean grain tank 132). Agricultural harvester 101 also includes a material transport subsystem that includes unloading auger 134, chute 135, spout 136, and can include one or more actuators that actuate movement of chute 135 or spout 136, or both, such that spout 136 can be positioned over an area in which grain is to be deposited. In operation, auger causes grain from grain tank 132 to be conveyed through chute 135 and out of spout 136. Agricultural harvester 101 can further include a residue subsystem 138 that can include chopper 140 and spreader 142. Agricultural harvester 101 can also have a propulsion subsystem that includes an engine (or other power source) that drives ground engaging elements 144 (such as wheels, tracks, etc.). It will be noted that agricultural harvester 101 can also have more than one of any of the subsystems mentioned above (such as left and right cleaning shoes, separators, etc.).
As shown in
In operation, and by way of overview, the height of header 104 is set and agricultural harvester 101 illustratively moves over a field in the direction indicated by arrow 146. As it moves, header 104 engages the crop to be harvested and gather it towards cutter 106. After it is cut, the crop can be engaged by reel 105 that moves the crop to a feeding system. The feeding system move the crop to the center of header 104 and then through a center feeding system in feeder house 108 toward feed accelerator 109, which accelerates the crop into thresher 111. The crop is then threshed by rotor 112 rotating the crop against concaves 114. The threshed crop is moved by a separator rotor in separator 116 where some of the residue is moved by discharge beater 126 toward a residue subsystem. It can be chopped by a residue chopper 140 and spread on the field by spreader 142. In other implementations, the residue is simply dropped in a windrow, instead of being chopped and spread.
Grain falls to cleaning shoe (or cleaning subsystem) 118. Chaffer 122 separates some of the larger material from the grain, and sieve 124 separates some of the finer material from the clean grain. Clean grain falls to an auger in clean grain elevator 130, which moves the clean grain upward and deposits it in clean grain tank 132. Residue can be removed from the cleaning shoe 118 by airflow generated by cleaning fan 120. That residue can also be moved rearwardly in combine 100 toward the residue handling subsystem 138.
Tailings can be moved by tailing elevator 128 back to thresher 110 where they can be re-threshed. Alternatively, the tailings can also be passed to a separate re-threshing mechanism (also using a tailings elevator or another transport mechanism) where they can re-threshed as well.
Cleaning shoe loss sensors 152 illustratively provide an output signal indicative of the quantity of grain loss by both the right and left sides of the cleaning shoe 118. In one example, sensors 152 are strike sensors which count grain strikes per unit of time (or per unit of distance traveled) to provide an indication of the cleaning shoe grain loss. The strike sensors for the right and left sides of the cleaning shoe can provide individual signals, or a combined or aggregated signal. It will be noted that sensors 152 can comprise on a single sensor as well, instead of separate sensors for each shoe.
Separator loss sensors 148 provide signals indicative of grain loss in the left and right separators. The sensors associated with the left and right separators can provide separate grain loss signals or a combined or aggregate signal. This can be done using a wide variety of different types of sensors as well. It will be noted that separator loss sensors 148 may also comprise only a single sensor, instead of separate left and right sensors.
Material dynamics sensors 179 provide sensor signals indicative of material dynamics characteristics, such as material having spilled out of a material receptacle, such as clean grain tank 132, as well as provide sensor signals indicative of movement of material within a material receptacle, such as clean grain tank 132. Material dynamics sensors 179 include material spill and movement sensors 180 (shown in
One example of a material spill and movement sensor 180 is illustratively shown in
In addition to imaging system 157, material spill and movement sensors 180 can include a variety of other material spill and movement sensors not illustratively shown in
It will be appreciated that agricultural harvester 101 can include a variety of other sensors not illustratively shown in
Some additional examples of the types of sensors that can be used are described below, including. but not limited to a variety of position sensors that can generate sensor signals indicative of a position (e.g., geographic location, orientation, elevation, etc.) of agricultural harvester 101 on the field over which agricultural harvester 101 travels or a position of various components of agricultural harvester 101 (e.g., header 104) relative to, for example, the field over which agricultural harvester 101 travels.
In operation, material receptacle implement 203 receives material, such as harvested crop material, from an agricultural harvester, such as agricultural harvester 101, via a material transfer subsystem, such as material transfer subsystem 341 (shown below). The material receptable implement 203 holds the received material within material receptacle 208 and is towed by towing vehicle 205 to a desired location. One or more material spill and movement sensors 180. of material dynamics sensors 179, can detect spillage of material out of material receptacle 208, spillage of material during a material transfer operation, as well as movement of material within material receptacle 208. While not shown in
In operation, material receptacle implement 253 receives material, such as harvested crop material, from an agricultural harvester, such as agricultural harvester 101, via a material transfer subsystem, such as material transfer subsystem 341 (shown below or from another material transport machine, such as material transport machine 201 via a material transfer subsystem, such as a material transfer subsystem 341 (shown below). The material receptable implement 253 holds the received material within material receptacle 258 and is towed by towing vehicle 255 to a desired location. One or more material spill and movement sensors 180, of material dynamics sensors 179, can detect spillage of material out of material receptacle 258, spillage of material during a material transfer operation, as well as movement of material within material receptacle 258. While not shown in
Both transferring machine 270 and receiving machine 272 can include material dynamics sensors 179. During the material transfer operation the material dynamics sensors 179 can detect material dynamics characteristics, as well as various other characteristics such as a fill level of material in receiving machine 272. Ideally, material 282 is transferred from transferring machine 270 to transferring machine 272 according to a fill profile (e.g., front to back, back to front) until the transferring machine 270 is at a desired emptiness or until the material receptacle 307 of the receiving machine 272 is at a desired fill level. In some examples, material stream 280, and thus material 282, may not land in material receptacle 307 of receiving machine 272. For instance, the relative positioning or relative speed between the machines may be such that the material 282 does not land in the interior of material receptacle 307. In other examples, the machines may be correctly positioned or traveling at correct speeds, but the wind may blow the material stream 280, and thus the material 282, in an unexpected course, thus causing the material 282 to spill (e.g., land outside of material receptacle 307). In any case, these material dynamics characteristics (as well as other characteristics) can be detected by material dynamics sensors 179 during a material transfer operation, as will be described in greater detail herein.
The in-situ sensors 308 can be on-board mobile machine 100, remote from mobile machine, such as deployed at fixed locations on the worksite or on another machine operating in concert with mobile machine 100, such as an aerial vehicle, and other types of sensors, or a combination thereof. In-situ sensors 308 sense characteristics of or at a worksite during the course of an operation. In-situ sensors 308 illustratively include material dynamics sensors 179, heading/speed sensors 325, crop moisture sensors 326, machine orientation sensors 327, and can include various other sensors 328. Material dynamics sensors 179 illustratively include one or more material spill and movement sensors 180 and can include other sensors 181. Material spill and movement sensors 180 illustratively include one or more mass sensors 380, one or more audible/acoustic sensors 382, one or more electromagnetic radiation (ER) sensors 384, one or more imaging system 386, one or more contact sensors 388, and can include other types of material spill and movement sensors 389. Material spill and movement sensors provide sensors signals indicative of material, such as crop material, having spilled out of a material receptacle 307 of mobile machine 100 or provide sensor signals indicative of movement of material, such as crop material, within a material receptacle 307 of mobile machine 100. It will be understood that in some examples, one or more types of material spill and movement sensor 180 can detect both material spill and material movement. In some examples, a plurality of the same type of material spill and movement sensor 180 are included on mobile machine 100, for example a first set of one or more disposed to detect material spill with a second set of one or more disposed to detect material movement. Material spill and movement sensors 180 can be disposed outside of material receptacles 307 or inside of material receptacles 307, or both. Material spill sensors can be disposed to sense within material receptacles 307 or without material receptacles 307, or both.
Mass sensors 380 detect a mass (i.e., a weight) of material within material receptacles 307 and generate sensor signals indicative of the mass of the material within material receptacles 307. Mass sensors 380 can include load cells, strain gauges, pressure sensors, as well as various other types of mass sensors. Mass sensors 380 can be positioned between components of mobile machine 100. For instance, mass sensors 380 can be positioned between material receptacles 307, or a frame, or other body, that supports material receptacle 307, and axle(s) of mobile machine 100, these mass sensors 380 generate a signal in response to force applied by the weight of the material receptacle 307 and material within material receptacle 307. Additionally, mass sensors 380 can also be included on a hitch of mobile machine 100 (e.g., hitch of towing vehicle) or a tongue of mobile machine 100 (e.g., tongue of towed implement) and generate a signal in response to force applied by towed implement on the hitch. In some examples, one or more mass sensors 380 can be disposed within material receptacles 307, for instance, one or more mass sensors 380 disposed at different locations within material receptacles 307. As discussed, mobile machine 100 can include more than one mass sensor 380, where each mass sensor 380 is associated with a different location and thus, a mass distribution (as well as change in the mass distribution) of the mass within material receptacle can be derived. Additionally, the mass distribution can be used to derive a center of mass of the material within material receptacle.
Audible/acoustic sensors 382 detect noise (e.g., sound waves) generated by movement of material within material receptacles 307 or noise generated by material contacting surfaces outside of the interior of material receptacles 307, such as a cab roof of mobile machine 100, an exterior side (e.g., top side or outer side, or both) of material receptacle 307, a frame of mobile machine 100, surface of worksite, as well as various other surfaces, and generate sensor signals based on the noise generated by the material movement or the material contacting surfaces outside of the interior of material receptacle 307, or both. In some examples, audible/acoustic sensors 382 include one or more microphones. In some examples, one or more of audible/acoustic sensors 382 may include and/or may be positioned proximate to a strike plate which is positioned on portions of mobile machine 100 in areas where material may contact the strike plate when the material has spilled out of the material receptacle 307. Contact between the strike plate, or other surfaces outside of material receptacle 307, produces a sound which is detected by audible/acoustic sensors 382.
ER sensors 384 detect electromagnetic radiation that travels through an area, such as an area outside of material receptacles 307 or inside of material receptacles 307, or both. Material in the area(s) through which the electromagnetic radiation travels of interacts with the electromagnetic radiation, such as by attenuating the electromagnetic radiation received by ER sensors 384, blocking at least a portion of the electromagnetic radiation from being received by ER sensors 384, or by causing reflection of electromagnetic radiation back towards ER sensors 384. In some examples, ER sensors 384 can include an ER transmitter that transmits electromagnetic radiation and a receiver that receives the transmitted electromagnetic radiation. Material, present in the area through which the electromagnetic radiation is transmitted, disrupts (e.g., blocks, attenuates, etc.) the reception of and/or the strength of the received electromagnetic radiation at the receiver. The disruption can be detected to indicate the presence of material in the area. In other examples, material present in the area through which the electromagnetic radiation is transmitted is reflected form the material and back towards the receiver. The received reflected electromagnetic radiation can indicate the presence of material in the area. In other examples, the ER sensors 384 only include an electromagnetic radiation receiver that receives electromagnetic radiation. Material presence in an area in which the receiver is disposed to view (i.e., receive electromagnetic radiation through) disrupts the reception of the electromagnetic radiation. This disruption can be detected to indicate the presence of material in the area. If the area is outside of the interior of material receptacle 307, this may indicate material spillage, whereas if the area is within the interior of material receptacle 307, this may indicate movement of the material.
Imaging systems 386 image the interior of material receptacle or areas outside of material receptacle 307, or both. Imaging systems 386 can include on or more imaging devices, such as one or more cameras. Material in the images in areas outside of the material receptacle 307 can indicate spillage of the material from material receptacle 307. Imaging systems 386 can also capture images of the material within material receptacles 307 As will be discussed in further detail, the images generated by imaging systems 386 can be processed to detect movement of the material with material receptacles 307 as well as to detect a center of mass of the material (e.g., material pile) within material receptacles 307. In one example, the field of view of imaging systems 386 can include areas inside and outside of material receptacle 307. The areas outside of material receptacle 307 can be identified and zoned by subsequent image processing of images generated by imaging systems 386 such that material in the image in the identified zones can be identified to detect material in areas outside of material receptacle 307.
Contact sensors 388 are disposed inside of material receptacle 307 or outside of material receptacle 307, or both. Contact sensors 388 can include a contact member, such as a pad, body, etc., for instance a piezo electric contact member, which generates an electrical signal in response to force of contact between the contact member and the material. In other examples, contact sensors 388 can include a displaceable object which is displaced by the force of contact between the displaceable object and the material. The displacement can be detected by a sensor, such as a potentiometer or a Hall effect sensor and can be used to detect the spillage or movement of material. For instance, contact with a contact sensor 388 disposed within material receptacle 307 can indicate movement of material, whereas contact with a contact sensor 388 disposed outside of the interior of material receptacle can indicated spillage of material. In some examples, contact sensors 388 can be disposed on an exterior side or top side of material receptacle 307, a frame of mobile machine 100, a cab roof of mobile machine 100, as well as various other locations. In some examples, contact sensors 388 can be disposed on interior walls of material receptacle 307 and can be a certain distance from a top perimeter of the material receptacle 307.
Material spill and movement sensors 180 can include other types of material spill and movement sensors, as indicated by 389.
Fill level sensors 324 sense a characteristic indicative of a fill level of material receptacle 307. In some examples, fill level sensors 324 can be the same as some of the material spill and movement sensors or can utilize the signals received from the material spill and movement sensors 180. In other examples, fill level sensors 324 can be separate from spill and material sensors 180, such as one or more imaging systems, one or more ER sensors, one or more mass sensors, or a one or more mass flow sensors that measure an amount of material entering material receptacle 307. For instance, a mass flow sensor that senses a flow of grain through grain elevator 130.
In other examples, fill level sensors 324 may utilize signals and data from other sources to detect a fill level of mobile machine 100. For example, sensor data from geographic position sensor or heading/speed sensors 325, or both, can provide an indication of a route and distance that mobile machine 100 has traveled and a map of the field, such as a yield map, can provide yield values along that route which can be aggregated for the distance traveled along that route to detect a fill level (as represented by a harvested yield along that traveled distance). This is merely one example.
Heading/speed sensors 325 detect a heading and speed at which mobile machine 100 is traversing the worksite during the operation. This can include sensors that sense the movement of ground-engaging elements (e.g., wheels or tracks 144, 207, 209. 257, 259, etc.) or can utilize signals received from other sources, such as geographic position sensor 304, thus, while heading/speed sensors 325 as described herein are shown as separate from geographic position sensor 304, in some examples, machine heading/speed is derived from signals received from geographic positions sensors and subsequent processing. In other examples, heading/speed sensors 325 are separate sensors and do not utilize signals received from other sources. Detecting a speed includes detecting a travel speed of mobile machine 100 as well as detecting a change in the travel speed, that is the acceleration and deceleration of mobile machine 100.
Crop moisture sensors 326 sense a characteristic indicative of a moisture level of crop. Without limitation, these crop moisture sensors may include a capacitance sensor, a microwave sensor, or a conductivity sensor, among others. In some examples, the crop moisture sensor may utilize one or more bands of electromagnetic radiation in detecting the crop moisture. Crop moisture sensors 326 can include a capacitive moisture sensor. In one example, the capacitance moisture sensor can include a moisture measurement cell for containing the crop material sample and a capacitor for determining the dielectric properties of the sample. In other examples, a crop moisture sensor 326 may be a microwave sensor or a conductivity sensor. In other examples, a crop moisture sensor 326 may utilize wavelengths of electromagnetic radiation 14 for sensing the moisture content of the crop material. One or more crop moisture sensors 326 can be disposed along the flow path of gathered crop within mobile machine 100, such as within the chute 135, feeder house 108, or clean grain elevator 130 (or otherwise have sensing access to crop material within chute 135, feeder house 108, or clean grain elevator 130). In other examples, one or more crop moisture sensors 326 may be located at other areas within mobile machine 100, such as material receptacles 307. It will be noted that these are merely examples of crop moisture sensors, and that various other crop moisture sensors are contemplated. The moisture of crop material affects the cohesive and frictional forces between individual crop material (e.g., individual grains) and thus affects the angle of repose. Thus, moisture of crop material may make it more or less likely that the crop material within material receptacle 307 will move.
Machine orientation sensors 327 can include one or more inertial measurement units (IMUs) which can provide orientation information relative to mobile machine 100, such as pitch, roll, and yaw data of mobile machine 100. The one or more IMUs can include accelerometers, gyroscopes, and magnetometers.
Other in-situ sensors 328 may be any of the sensors described above with respect to
In-situ data includes data taken from a sensor on-board the mobile machine 100 or taken by any sensor where the data are detected during the operation of mobile machine 100 at a worksite.
Processing system 338 processes the sensor signals generated by in-situ sensors 308 to generate processed sensor data indicative of one or more characteristics. For example, processing system generates processed sensor data indicative of characteristic values based on the sensor signals generated by in-situ sensors, such as material dynamics values based on sensor signals generated by material dynamics sensors 179, which can include one or more of material movement values based on sensors signals generated by material spill and movement sensors 180 and material spill values based on sensor signals generated by material spill and movement sensors 180. Further examples include mass values based on sensor signals generated by mass sensors 180, center of mass values based on images from imaging systems 186, crop moisture values based on sensor signals generated by crop moisture sensors 326, machine orientation (pitch, roll, etc.) values based on sensors signals generated by machine orientation sensors 327, machine speed (travel speed, acceleration, deceleration, etc.) values based on sensor signals generated by heading/speed sensors 325, as well as various other values based on sensors signals generated by various other in-situ sensors 328.
It will be understood that processing system 338 can be implemented by one or more processers or servers, such as processors or servers 301. Additionally, processing system 338 can utilize various sensor signal filtering techniques, noise filtering techniques, sensor signal categorization, aggregation, normalization, as well as various other processing functionality. Similarly, processing system 338 can utilize various image processing techniques such as, sequential image comparison, RGB, edge detection, black/white analysis, machine learning, neural networks, pixel testing, pixel clustering, shape detection, as well any number of other suitable image processing and data extraction functionality. The operation of processing system 338 will be described in more detail in
Control system 314 includes communication system controller 329, interface controller 330, propulsion controller 331, path planning controller 334, material transfer controller 335, zone controller 336, and control system 314 can include other items 246. Controllable subsystems 316 include material transfer subsystem 341, propulsion subsystem 350, steering subsystem 352, and subsystem 316 can include a wide variety of other subsystems 356. Material transfer subsystem 341, itself, includes spout 342, chute 344, auger 345, and can include other items 346, such as one or more controllable actuators to drive movement of spout 342, chute 344, and auger 345.
Information maps 358 may be downloaded onto mobile machine 100 over network 359 and stored in data store 302, using communication system 306 or in other ways. In some examples, communication system 306 may be a cellular communication system, a system for communicating over a wide area network or a local area network, a system for communicating over a near field communication network, or a communication system configured to communicate over any of a variety of other networks or combinations of networks, including a variety of other wired or wireless networks. Network 359 illustratively represents any or a combination of any of the variety of networks. Communication system 306 may also include a system that facilitates downloads or transfers of information to and from a secure digital (SD) card or a universal serial bus (USB) card or both.
Geographic position sensor 304 illustratively senses or detects the geographic position or location of agricultural harvester 100. Geographic position sensor 304 can include, but is not limited to, a global navigation satellite system (GNSS) receiver that receives signals from a GNSS satellite transmitter. Geographic position sensor 304 can also include a real-time kinematic (RTK) component that is configured to enhance the precision of position data derived from the GNSS signal. Geographic position sensor 304 can include a dead reckoning system, a cellular triangulation system, or any of a variety of other geographic position sensors.
Predictive model generator 310 generates a model that is indicative of a relationship between the value(s) sensed by the in-situ sensors 308 and value(s) mapped to the field by the information maps 358. For example, if the information map 358 maps a terrain value to different locations in the worksite, and the in-situ sensor 308 is sensing a value indicative of material spill or material movement, then model generator 310 generates a predictive material spill model that models the relationship between the terrain value and the material spill value or generates a predictive material movement model that models the relationship between the terrain value and the material movement value. In another example, if the information map 358 maps a speed value to different locations in the worksite, and the in-situ sensor 308 is sensing a value indicative of material spill or material movement, then model generator 310 generates a predictive material spill model that models the relationship between the speed value and the material spill value or generates a predictive material movement model that models the relationship between the speed value and the material movement value. In another example, if the information map 358 maps a crop moisture value to different locations in the field, and the in-situ sensor 308 is sensing a value indicative of material spill or material movement, then model generator 310 generates a predictive material spill model that models the relationship between the crop moisture value and the material spill value or generates a predictive material movement model that models the relationship between the crop moisture value and the material movement value. In another example, if the information map 358 maps a fill level value to different locations in the field, and the in-situ sensor 308 is sensing a value indicative of material spill or material movement, then model generator 310 generates a predictive material spill model that models the relationship between the fill level value and the material spill value or a material movement model that models the relationship between the fill level value and the material movement value. As will be shown below, both a predictive material spill model and a predictive material movement model are examples of a predictive material dynamics model. Similarly, both a predictive material spill map and a predictive material movement map are examples of a predictive material dynamics map.
In another example, predictive model generator generates a model that is indicative of a relationship between a first set of values sensed by the in-situ sensors 308 and a second set of values sensed by in-situ sensors. For example, the first set of values may include material dynamics values, such as material spill values or material movement values sensed material spill and movement sensors 180, and the second set of values may include one or more of machine orientation values sensed by machine orientation sensors 327, speed values sensed by heading/speed sensors 325, crop moisture values sensed by crop moisture sensors 326, fill level values sensed by fill level sensors 324, material center of mass values sensed by material spill and movement sensors 180, such as imaging systems 386, and material mass values sensed by material spill and movement sensors 180, such as mass sensors 380. In said example, predictive model generator generates a predictive material dynamics model, such as a predictive material movement model that models the relationship between the material movement values and one or more of the machine orientation values, speed values, crop moisture values, fill level values, material center of mass values, and material mass values or a predictive material spill model that models the relationship between material spill values and one or more of the machine orientation values, speed values, crop moisture values, fill level values, material center of mass values, and material mass values.
In some examples, the predictive map generator 312 uses the predictive models generated by predictive model generator 310 to generate functional predictive maps that predict the value of one or more characteristics, such as one or more material dynamics characteristics such as one or more material movement characteristics or one or more material spill characteristics, sensed by the in-situ sensors 308 at different locations in the worksite based upon one or more of the information maps 358. For example, where the predictive model is a predictive material dynamics model in the form of a predictive material spill model that models a relationship between material spill characteristic(s) sensed by one or more in-situ sensors 308 and one or more of terrain values from a terrain map, speed values from a speed map, crop moisture values from a crop moisture map, and fill level values from a fill level map, then predictive map generator 312 generates a functional predictive material dynamics map in the form of a functional predictive material spill map that predicts one or more material spill characteristics at different locations at the worksite field based on one or more of the terrain values. the speed values, the crop moisture values, and the fill level values at those locations and the predictive material spill model. In another example, where the predictive model is a predictive material dynamics model in the form of a predictive material movement model that models a relationship between one or more material movement characteristics sensed by one or more in-situ sensors 308 and one or more of terrain values from a terrain map, speed values from a speed map, crop moisture values from a crop moisture map, and fill level values from a fill level map, then predictive map generator 312 generates a functional predictive material dynamics map in the form of a functional predictive material spill map that predicts one or more material movement characteristics at different locations at the worksite based on one or more of the terrain values, the speed values, the crop moisture values, and the fill level values at those locations and the predictive material movement model. It will be understood that functional predictive map 263 encompasses the various functional predictive maps that can be generated by predictive map generator 312.
In some examples, the type of values in the functional predictive map 263 may be the same as the in-situ data type sensed by the in-situ sensors 308. In some instances, the type of values in the functional predictive map 263 may have different units from the data sensed by the in-situ sensors 308. In some examples, the type of values in the functional predictive map 263 may be different from the data type sensed by the in-situ sensors 308 but have a relationship to the type of data type sensed by the in-situ sensors 308. For example, in some examples, the data type sensed by the in-situ sensors 308 may be indicative of the type of values in the functional predictive map 263. In some examples, the type of data in the functional predictive map 263 may be different than the data type in the information maps 358. In some instances, the type of data in the functional predictive map 263 may have different units from the data in the information maps 358. In some examples, the type of data in the functional predictive map 263 may be different from the data type in the information map 358 but has a relationship to the data type in the information map 358. For example, in some examples, the data type in the information maps 358 may be indicative of the type of data in the functional predictive map 263. In some examples, the type of data in the functional predictive map 263 is different than one of, or both of, the in-situ data type sensed by the in-situ sensors 308 and the data type in the information maps 358. In some examples, the type of data in the functional predictive map 263 is the same as one of, or both of, of the in-situ data type sensed by the in-situ sensors 308 and the data type in information maps 358. In some examples, the type of data in the functional predictive map 263 is the same as one of the in-situ data type sensed by the in-situ sensors 308 or the data type in the information maps 358, and different than the other.
As shown in
Some variations in the data types that are mapped in the information maps 358, the data types sensed by in-situ sensors 308, and the data types predicted on the predictive map 264 will now be described.
In some examples, the data type in one or more information maps 358 is different from the data type sensed by in-situ sensors 308, yet the data type in the predictive map 264 is the same as the data type sensed by the in-situ sensors 308. For instance, the information map 358 may be a speed map, and the variable sensed by the in-situ sensors 308 may be material movement. The predictive map 264 may then be a predictive material movement map that maps predicted material movement values to different geographic locations in the in the worksite.
Also, in some examples, the data type in the information map 358 is different from the data type sensed by in-situ sensors 308, and the data type in the predictive map 264 is different from both the data type in the information map 358 and the data type sensed by the in-situ sensors 208.
In some examples, the information map 358 is from a prior pass through the field during a prior operation and the data type is different from the data type sensed by in-situ sensors 308, yet the data type in the predictive map 264 is the same as the data type sensed by the in-situ sensors 308. For instance, the information map 258 may be a terrain map generated during a previous operation on the worksite, and the variable sensed by the in-situ sensors 308 may be material movement. The predictive map 264 may then be a predictive material movement map that maps predicted material movement values to different geographic locations in the worksite.
In some examples, the information map 358 is from a prior pass through the field during a prior operation and the data type is the same as the data type sensed by in-situ sensors 308, and the data type in the predictive map 264 is also the same as the data type sensed by the in-situ sensors 308. For instance, the information map 358 may be a material spill map generated during a previous year, and the variable sensed by the in-situ sensors 308 may be material spill. The predictive map 264 may then be a predictive material spill map that maps predicted material spill values to different geographic locations in the field. In such an example, the relative material spill differences in the georeferenced information map 358 from the prior year can be used by predictive model generator 310 to generate a predictive model that models a relationship between the relative material spill differences on the information map 358 and the material spill values sensed by in-situ sensors 308 during the current operation. The predictive model is then used by predictive map generator 310 to generate a predictive material spill map.
In another example, the information map 358 may be a terrain map generated during a prior operation in the same year, such as fertilizer application operation performed by towing vehicle 205 (or another towing vehicle) and a towed fertilizer applicator implement, and the variable sensed by the in-situ sensors 308 during the current operation may be material movement. The predictive map 264 may then be a predictive material movement map that maps predicted material movement values to different geographic locations in the worksite. In such an example, a map of the terrain values at time of fertilizer application is geo-referenced recorded and provided to mobile machine 100 as an information map 358 of terrain values. In-situ sensors 308 during a current operation can detect material movement at geographic locations in the field and predictive model generator 310 may then build a predictive model that models a relationship between material movement at time of the current operation and terrain values at the time of fertilizer application. This is because the terrain values at the time of nutrient application are likely to be the same as at the time of the current operation.
In some examples, predictive map 264 can be provided to the control zone generator 313. Control zone generator 313 groups adjacent portions of an area into one or more control zones based on data values of predictive map 264 that are associated with those adjacent portions. A control zone may include two or more contiguous portions of a worksite, such as a field, for which a control parameter corresponding to the control zone for controlling a controllable subsystem is constant. For example, a response time to alter a setting of controllable subsystems 316 may be inadequate to satisfactorily respond to changes in values contained in a map, such as predictive map 264. In that case, control zone generator 313 parses the map and identifies control zones that are of a defined size to accommodate the response time of the controllable subsystems 316. In another example, control zones may be sized to reduce wear from excessive actuator movement resulting from continuous adjustment. In some examples, there may be a different set of control zones for each controllable subsystem 316 or for groups of controllable subsystems 316. The control zones may be added to the predictive map 264 to obtain predictive control zone map 265. Predictive control zone map 265 can thus be similar to predictive map 264 except that predictive control zone map 265 includes control zone information defining the control zones. Thus, a functional predictive map 263, as described herein, may or may not include control zones. Both predictive map 264 and predictive control zone map 265 are functional predictive maps 263. In one example, a functional predictive map 263 does not include control zones, such as predictive map 264. In another example, a functional predictive map 263 does include control zones, such as predictive control zone map 265. In some examples, multiple crops may be simultaneously present in a field if an intercrop production system is implemented. In that case, predictive map generator 312 and control zone generator 313 are able to identify the location and characteristics of the two or more crops and then generate predictive map 264 and predictive map with control zones 265 accordingly.
It will also be appreciated that control zone generator 313 can cluster values to generate control zones and the control zones can be added to predictive control zone map 265, or a separate map, showing only the control zones that are generated. In some examples, the control zones may be used for controlling or calibrating mobile machine 100 or both. In other examples, the control zones may be presented to the operator 360 and used to control or calibrate mobile machine 100, and, in other examples, the control zones may be presented to the operator 360 or another user, such as a remote user 366, or stored for later use.
Predictive map 264 or predictive control zone map 265 or both are provided to control system 314, which generates control signals based upon the predictive map 264 or predictive control zone map 265 or both. In some examples, communication system controller 329 controls communication system 306 to communicate the predictive map 264 or predictive control zone map 265 or control signals based on the predictive map 264 or predictive control zone map 265 to other mobile machines that are operating at the same worksite or in the same operation. In some examples, communication system controller 329 controls the communication system 306 to send the predictive map 264, predictive control zone map 265, or both to other remote systems, such as remote computing systems 368.
Interface controller 330 is operable to generate control signals to control interface mechanisms, such as operator interface mechanisms 318 or user interfaces 364, or both. The interface controller 330 is also operable to present the predictive map 264 or predictive control zone map 265 or other information derived from or based on the predictive map 264, predictive control zone map 265, or both to operator 360 or a remote user 366, or both. Operator 360 may be a local operator or a remote operator. As an example, interface controller 330 generates control signals to control a display mechanism to display one or both of predictive map 264 and predictive control zone map 265 for the operator 260 or a remote user 366, or both. Interface controller 330 may generate operator or user actuatable mechanisms that are displayed and can be actuated by the operator or user to interact with the displayed map. The operator or user can edit the map by, for example, correcting a value displayed on the map, based on the operator's or the user's observation.
Path planning controller 334 illustratively generates control signals to control steering subsystem 352 to steer mobile machine 100 according to a desired path or according to desired parameters, such as desired steering angles based on one or more of the predictive map 264, the predictive control zone map 265, and the predictive model 311. Path planning controller 334 can control a path planning system to generate a route for agricultural harvester 100 and can control propulsion subsystem 350 and steering subsystem 352 to steer agricultural harvester 100 along that route.
Propulsion controller 331 illustratively generates control signals to control propulsion subsystem 350 to control a speed characteristic, such as one or more of travel speed, acceleration, and deceleration, based on one or more of the predictive map 264, the predictive control zone map 265, and the predictive model 311. In one example, propulsion controller 331 illustratively generates control signals to control a speed characteristic of mobile machine 100. In another example propulsion controller 331 illustratively generates control signals to control a speed characteristic of another machine 375, such as another machine operating in concert with mobile machine 100. For example, mobile machine 100 can be an agricultural harvester 101. another machine 375 can be a material transport machine such as material transport machine 201 or 251. Propulsion controller 331 can generate control signals to control a speed characteristic of mobile machine 100 or another machine 375, or both, such as during a material transfer operation to desirably align mobile machine 100 and another machine to desirably transfer material.
Material transfer controller 335 illustratively generates control signals to control material transfer subsystem 341 to control operation of the material transfer subsystem 341, such as the position of spout 342, the position of chute 344, the speed at which auger 345 operates to propel material from material receptacles 307 through chute 344 and spout 342, as well as to initiate or end a material transfer operation. Material transfer controller generates control signal to control material transfer subsystem based on one or more of the predictive map 264, the predictive control zone map 265, and the predictive model 311. In one example, the position of spout 342 or chute 344, or both, can be controlled relative to mobile machine 100 or relative to another machine 375, such as another machine operating in concert with mobile machine 100. For example, mobile machine 100 can be an agricultural harvester 101, another machine 375 can be a material transport machine such as material transport machine 201 or 251. Material transfer controller 335 can generate control signals to control a position of spout 342 or chute 344, or both, relative to mobile machine 100 or another machine 375, such as during a material transfer operation to desirably transfer material.
Zone controller 336 illustratively generates control signals to control one or more controllable subsystems 316 to control operation of the one or more controllable subsystems based on the predictive control zone map 265.
Other controllers 337 included on the mobile machine 100, or at other locations in agricultural system 300, can control other subsystems based on the predictive map 264 or predictive control zone map 265 or both as well.
While the illustrated example of
In operation, processing system 338 processes sensor data generated by in-situ sensors 308 to generate processed sensor data indicative of one or more characteristics. For example, processing system 338 can identify material dynamics characteristics, for instance one or more of material spill characteristics, such as the occurrence, amount, and location of material spill and material movement characteristics, such as the occurrence, amount, distance of material movement, and direction of material movement. In some examples, material movement characteristics can also include proximal location to which material moved and processing system 338 can also determine a proximal location to which the material moved to within the material receptacle. For instance, it may be that the material moved to such a degree that it came within a threshold distance of a perimeter of the material receptacle (and thus was in danger of spilling out of the material receptacle) In further examples, processing system 338 can identify material mass characteristics, material center of mass characteristics, machine heading, machine speed characteristics, such as travel speed, acceleration, and deceleration, machine orientation characteristics, such as pitch and roll of mobile machine 100, crop moisture characteristics, material fill level, as well as various other characteristics.
Data capture logic 406 captures or obtains data that can be used by other items in processing system 338. Sensor accessing logic 424 can be used by processing system 338 to obtain or otherwise access sensor data (or values indicative of the sensed variables/characteristics) provided from in-situ sensors 308. Additionally, data store accessing logic 426 can be used by processing system 338 to obtain or access data stored on data stores 302. Upon obtaining various data, processing system 338, processes the data to identify characteristics at the worksite.
For instance, material spill and movement analyzer 400 processes data, such as sensor data from material spill and movement sensors 180, to identify characteristics of material spill or material movement, or both, as well as other characteristics of the material, such as mass of the material, center of mass of the material, as well as material fill level.
Mass signal processing component 412 obtains mass sensor data generated by mass sensors 380 and processes the mass sensor data to identify a mass signal value indicative of a mass (i.e., weight) of material in material receptacles 307. In some examples, mass sensors 380 generate an electrical signal indicative of the mass of material in material receptacle 307. Mass signal processing component 412 identifies a value of the electrical signal to determine a mass of the material in material receptacles 307. Mass signal processing component 412 also identifies a change of the mass signal value over time to determine a change in the mass of the material in material receptacle 407. The identified change in the mass of the material in material receptacle 407 can be used by material spill and movement analyzer 400 to identify characteristics of material spill or material movement, or both. For example, a decrease in the mass signal value can indicate material spillage from material receptacles 307, and the amount of decrease can indicate the amount of material spilled from material receptacles 307. In another example, where multiple mass sensors 380 are utilized, a decrease in the signal value of one mass sensor and an increase in the signal value of another mass sensor can indicate characteristics of material movement, such as the occurrence of material movement, the amount of material moved, the distance of material movement, and the direction of material movement. In some examples, the change in the mass signal value can be compared to an expected change in the mass signal value to determine the occurrence of material spill and/or the amount of material spill. For instance, where mobile machine 100 is receiving material in material receptacles 307, such as from a material transfer subsystem of another machine, the mass signal value may not be increasing or may not be increasing at the expected rate. For instance, knowing the material transfer rate of the transferring vehicle (e.g., 10 bushels per second) and the weight of the material being transferred (e.g., 56 pounds per bushel of corn), an expected mass signal value change (560 pounds per second) can be determined. Where there is no change in the mass signal value or where the mass signal value does not change at the expected rate, material spill and movement analyzer 400 can identify characteristics of material spill, such as the occurrence of material spill and the amount of material spilled. It will be understood that a mass signal value indicative of an empty weight of the mobile machine 100, or an empty weight of material receptacles 307, can be stored in data stores 308 and accessed by mass signal processing component 412 such that the generated mass signal value can be compared to the empty weight mass signal value. In some examples, at the beginning of an operation, when the mobile machine is empty (or at least material from the current operation has not yet been stored in the material receptacle), a mass signal value can be generated and used as a reference.
Additionally, where more than one mass sensor 380 is used, the mass signal value generated by each mass sensor 380 can be aggregated to derive a total mass of the material in material receptacle 307, additionally, the multiple mass signal values can be used to derive a distribution of the mass in material receptacle 307 as well as to identify a center of mass of the material in material receptacle 307.
Audible/acoustic signal processing component 414 obtains audible/acoustic sensor data generated by audible/acoustic sensors 382 and processes the audible/acoustic sensor data to identify an audible/acoustic signal value indicative of a noise generated by contact between material from material receptacles 307 and objects outside of material receptacle 307 or noise generated by movement of the material within material receptacle 307, or both. In some examples, audible/acoustic sensors 382 generate an electrical signal in response to contact between material from material receptacles 307 and object(s) outside of material receptacle 307 or noise generated by movement of material in material receptacles 307, or both. Audible/acoustic signal processing component 414 identifies a value of the electrical signal. The identified audible/acoustic signal value can be used by material spill and movement analyzer 414 to identify characteristics of material spill or characteristics of material movement, or both. For instance, the audible/acoustic signal value may vary from a value at which no material spillage occurs and a value at which at least some material spillage occurs (e.g., a threshold value). An audible/acoustic signal value at or exceeding the threshold value can indicate the occurrence of material spill. The amount to which the audible/acoustic signal value exceeds the threshold value can indicate an amount of material spilled (e.g., a higher audible/acoustic signal value may indicate that more material spilled as compared to a lower audible acoustic signal value). Similarly, the audible/acoustic signal may vary from a value at which no material movement occurs and a value at which at least some material movement occurs (e.g., a threshold value). An audible/acoustic signal value at or exceeding the threshold value can indicate the occurrence of material movement. The amount to which the audible acoustic signal value exceeds the threshold value can indicate an amount of material spilled (e.g., higher audible/acoustic signal value may indicate that more material moved as compared to a lower audible acoustic signal). Additionally, audible/acoustic signal processing component 414 can identify an audible/acoustic signal width value which indicates an amount of time that the audible/acoustic signal value was at or exceeded a threshold value. The audible/acoustic signal width value can be used by material spill and movement analyzer 400 to determine an amount of material spilled or moved, or both, (e.g., the longer the audible/acoustic signal value is at or exceeds the threshold value, the more material that is spilled or moved). Audible/acoustic signal processing component 414 can also generate an aggregated audible/acoustic signal value indicative of an aggregate audible/acoustic signal value over a given period of time, such as during a period of time in which the audible/acoustic signal value was at or exceeded the threshold value. The aggregated audible/acoustic signal value can be used by material spill and movement analyzer 400 to determine an amount of material spilled or an amount of material moved. Additionally, based on the position of the audible/acoustic sensors 382 and the audible/acoustic signal, material spill and movement analyzer 400 can also identify a location of material spill or a direction of material movement as well, in some examples, a proximal location to which the material moved, such as a proximity to a perimeter of the material receptacle.
Electromagnetic radiation (ER) signal processing 416 obtains electromagnetic radiation (ER) sensor data from electromagnetic (ER) sensors 384 and processes the ER sensor data to identify an ER signal value indicative of a characteristic of the electromagnetic radiation received by ER sensors 384. It will be understood that electromagnetic radiation includes various types of radiation, including radio, microwave, infrared, visible light, ultraviolet, X-ray, and gamma ray. Thus, ER sensors 384, as discussed herein, contemplate the detection of any of the forms of radiation on the electromagnetic spectrum. In some examples, ER sensors 384 generate an electrical signal in response to received electromagnetic radiation. ER signal processing 416 identifies a value of the electrical signal. The identified ER signal value can be used by material spill and movement analyzer 400 to identify characteristics of material spill or material movement, or both. For instance, the ER signal value may vary from a value at which no material spillage occurs and a value at which at least some material spillage occurs (e.g., a threshold value). An ER signal value at or exceeding the threshold value can indicate the occurrence of material spill. The amount to which the ER signal value exceeds the threshold value can indicate an amount of material spilled (e.g., the amount to which the electromagnetic radiation was disturbed by the presence of material may indicate more material having spilled). Similarly, the ER signal value may vary from a value at which no material movement occurs and a value at which at least some material movement occurs (e.g., a threshold value). An ER signal value at or exceeding the threshold value can indicate the occurrence of material movement. The amount to which the ER signal value exceeds the threshold value can indicate an amount of material moved (e.g., the amount to which the electromagnetic radiation was disturbed by the presence of material may indicate more material having moved). Additionally, ER signal processing component 416 can identify an ER signal width value which indicates an amount of time that the ER signal value was at or exceeded the threshold value. The ER signal width value can be used by material spill and movement analyzer 400 to determine an amount of material spilled or moved (e.g., the longer the ER signal value is at or exceeds the threshold value, the more material that is spilled or moved). ER signal processing component 416 can also generate an aggregated ER signal value indicative of an aggregate ER signal value over a given period of time, such as during a period of time in which the ER signal value was at or exceeded the threshold value. The aggregated ER signal value can be used by material spill and movement analyzer 400 to determine an amount of material spilled or moved. Additionally, based on the position of the ER sensors 384 and the ER signal, material spill and movement analyzer 400 can also identify a location of material spill or a direction of material movement as well, in some examples, a proximal location to which the material moved, such as a proximity to a perimeter of the material receptacle.
Image processing component 418 obtains images generated by imaging systems 386 and processes the images to identify material spill characteristics, such as the occurrence of material spill based on the presence of material in an area outside of material receptacle 307 as well as to identify an amount of time in which material was present in an area outside of material receptacle 307 and an amount of material present within the area outside of material receptacle, as well as the location of the material spill. Additionally, image processing component 418 processes images generated by imaging system 386 to identify characteristics of material movement, such as the occurrence of material movement, the amount of material moved, a distance of material movement, the direction in which the material moved, as well as a proximal location to which the material moved, such as a proximity to a perimeter of the material receptacle. Image processing component 418 processes images generated by imaging system 386 to identify a characteristic indicative of the center of mass of the material within material receptacles 307, such as by identifying the mean position of the material composing the material pile within a material receptacle. Identifying the center of mass of material within the material receptacle can include identifying a top surface of the material pile relative to a perimeter (e.g., top edge) of the material receptacle and, knowing the dimensions of the material receptacle 307 (which can be stored in data store 302), identifying a center of mass of the material.
In some examples, the images obtained by image processing 418 may include areas outside of material receptacle 307 and areas inside of material receptacle 307. In such an example, image processing 426 can identify zones of the image which are outside of material receptacle 307, such that material present within those zones are indicative of characteristics of material spill, as well as identify zones of the image which are inside of the material receptacle such that material present within those zones are indicative of material movement characteristics or material center of mass characteristics, or both. In other examples, there may be dedicated imaging systems, with certain imaging systems dedicated to imaging only areas outside of material receptacle with other certain imaging systems dedicated to imaging only areas inside of the material receptacle.
Image processing component 418 processes the images to identify material, such as grain, in the image in areas outside of material receptacle 307. Based on the identification of material in the image in areas outside of material receptacle 307, material spill and movement analyzer 400 can determine the occurrence of material spill. Further, image processing component 418 can identify an amount of material present in areas outside of material receptacle 307, such as by summation of identified individual materials, such as individual grains, or by calculation of a volume based on an area of the image taken up by identified material. These are merely some examples. Based on the identified amount of material in the image in areas outside of material receptacle 307, material spill and movement analyzer 400 can identify the amount of material spilled. Additionally, image processing component 418 can identify an amount of time during which material was present in areas outside of material receptacle 307 as well as an amount of material present in areas outside of material receptacle 307 over that period of time, on the basis of which material spill and movement analyzer can identify the amount of material spilled.
It will be understood that image processing 426 can utilize a variety of image processing techniques or methods, such as sequential image comparison, RGB, edge detection, black/white analysis, machine learning, neural networks, pixel testing, pixel clustering, shape detection, as well any number of other suitable image processing and data extraction techniques and/or methods.
Contact signal processing component 420 obtains contact sensor data from contact sensors 388 and processes the contact sensor data to identify a contact signal value indicative of contact between material and contact sensors 388. In some examples, contact sensors 388 generate an electrical signal in response to contact between material and contact sensors 388. Contact signal processing component 420 identifies a value of the electrical signal. The identified contact signal value can be used by material spill and movement analyzer 400 to identify the occurrence of material spill as well as to identify an amount of material spilled. For instance, the contact signal value may vary from a value at which no material spillage occurs and a value at which at least some material spillage occurs (e.g., a threshold value). A contact signal value at or exceeding the threshold value can indicate the occurrence of material spill. Additionally, the amount to which the contact signal value exceeds the threshold value can indicate an amount of material spilled (e.g., a higher contact signal value may indicate that more material spilled as compared to a lower contact signal value). Similarly, the identified contact signal value can be used by material spill and movement analyzer 400 to identify the occurrence of material movement as well as to identify an amount of material moved. For instance, the contact signal value may vary from a value at which no material movement occurs and a value at which at least material movement occurs (e.g., a threshold value). A contact signal value at or exceeding the threshold value can indicate the occurrence of material movement. Additionally, the amount to which the contact signal value exceeds the threshold value can indicate an amount of material moved (e.g., a higher contact signal value may indicate that more material moved as compared to a lower contact signal value). Additionally, contact signal processing 428 can identify a contact signal value width value which indicates an amount of time that the contact signal value was at or exceeded the threshold value. The contact signal width value can be used by material spill and movement analyzer 400 to identify determine an amount of material spilled (e.g., the longer the contact signal value is at or exceeds the threshold value, the more material that is spilled) or an amount of material moved (e.g., the longer the contact signal value is at or exceeds the threshold value, the more material that is moved). Contact signal processing component 420 can also generate an aggregated contact signal value indicative of an aggregate contact signal value over a given period of time, such as during a period of time in which the contact signal value was at or exceeded the threshold value. The aggregated contact signal value can be used by material spill and movement analyzer 400 to determine an amount of material spilled or an amount of material moved. Additionally, based on the position of the contact sensors 388 and the contact signal, material spill and movement analyzer 400 can also identify a location of material spill, a direction of material movement or distance of material movement, or both. In other examples, based on the position of the contact sensors 388 and the contact signal, material spill and movement analyzer 400 can also identify a fill level of the material receptacle as well, in some examples, a proximal location to which the material moved, such as a proximity to a perimeter of the material receptacle.
Crop moisture signal processing component 402 obtains crop moisture sensor data from crop moisture sensors 326 and processes the crop moisture sensor data to identify a crop moisture signal value indicative of a moisture content of crop material. In some examples, crop moisture sensors 326 generate an electrical signal in response to crop moisture detection. Crop moisture signal processing component 402 identifies a value of the electrical signal to indicate a moisture content, such as a percentage or other value, of the crop material.
Heading/speed signal processing component 403 obtains heading and speed sensor data from heading/speed sensors 325 or from geographic position sensor 304, or both, and processes the heading and speed sensor data to identify a heading of mobile machine 100 as well as speed characteristic(s) (e.g., travel speed, acceleration, deceleration, etc.) of mobile machine 100. In some examples, speed sensors 325 sense the speed characteristics of mobile machine 100 by sensing the speed of rotation of ground engaging elements, speed of rotation of a drive shaft, speed of rotation of an axle, or various other component. The speed of rotation of the various components can indicate a travel speed of the mobile machine, an acceleration of the mobile machine 100, as well as a deceleration of mobile machine 100. In some examples, a change in speed of rotation over time can be identified to identify an acceleration or deceleration of mobile machine 100. In other examples, heading/speed signal processing component 403 utilizes geographic position information from geographic position sensors 304 to identify a heading, travel speed, acceleration, or deceleration of mobile machine 100.
Fill level signal processing component 404 obtains fill level sensor data from fill level sensors 324 or from material spill and movement sensors 180, or both, and processes the fill level sensor data to identify a fill level of a material receptacle 307 of mobile machine 100. For example, one or more ER sensors, such as 384 or other ER sensors, can be placed at various locations within the interior of material receptacle 307, and detected presence of material at that location, as indicated by an ER signal, can be used by fill level signal processing system 404 to identify a fill level value of material receptacle 307. In another example, one or more contact sensors, such as contact sensors 388 or other contact sensors, can be placed at various locations within the interior of material receptacle 307 and detected presence of material at that location, as indicated by a contact signal, can be used by fill level signal processing to identify a fill level value of material receptacle 307. In another example, one or more mass sensors, such as mass sensors 380 or other mass sensors, generate a mass signal indicative of the mass of the material within the material receptacle 307. The mass signal can be processed by fill level signal processing component 404 to identify a fill level value of material receptacle 307. For instance, the capacity of the material receptacle can be known (e.g., stored in data store 302), such as a bushel capacity, for instance 500 bushels. The material being placed in the material receptacle as well as the weight per unit of the material (e.g., weight per bushel) can also be known (e.g., stored in a data store). The current mass of the material in the material receptacle 307 can thus be used to derive a fill level of the material receptacle 307. In another example, fill level signal processing component 404 can process an image generated by an imaging system, such as imaging system 386 or other imaging systems, to identify a top surface of the material pile relative to a perimeter (e.g., top edge) of the material receptacle 307 and, knowing the dimensions of the material receptacle 307 (which can be stored in data store 302), identifying a fill level value. In another example, one or more mass flow sensors can measure the amount of material entering the material receptacle, such as a mass flow sensor disposed along clean grain elevator 130, and can generate a signal indicative of a mass flow of material. The mass flow signal can be used by fill level signal processing component 404 to identify a fill level value of material receptacle 307. In some examples, the sensor data value, such as a mass flow sensor signal value, can be aggregated by fill level signal processing component 404 to identify a fill level value of material receptacle 307. For instance, aggregating the mass flow signal value since the last time the material receptacle 307 was emptied.
Machine orientation signal processing component 405 obtains machine orientation sensor data from machine orientation sensors 327 and processes the machine orientation sensor data to identify a machine orientation signal value indicative of an orientation (e.g., pitch, roll, and/or yaw) of mobile machine 100 at the worksite. In some examples, machine orientation sensors 327 generate an electrical signal in response to machine orientation detection. Machine orientation signal processing component 405 identifies a value of the electrical signal to indicate an orientation value, such as a degree of pitch, roll, or yaw, of the mobile machine 100.
It will be understood that processing components 402, 403, 404, 405, 406, and 412-422 can utilize sensor signal filtering, such as noise filtering, sensor signal categorization, normalization, aggregation, as well as a variety of other processing techniques.
Machine learning component 408 can improve the identification of characteristics, by improving the algorithmic process for the determination, such as by improving the recognition of values and/or characteristics indicated by sensor data. Machine learning component 408 can also utilize a closed-loop style learning algorithm such as one or more forms of supervised machine learning.
As shown in
Material spill-to-terrain model generator 441 identifies a relationship between material spill detected in in-situ sensor data 440, at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and terrain value(s) from the terrain map 431 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-terrain model generator 441, material spill-to-terrain model generator 441 generates a predictive material spill model. The predictive material spill model is used by material spill map generator 452 to predict material spill at different locations in the worksite based upon the georeferenced terrain value(s) contained in the terrain map 431 at the same locations in the worksite. Thus, for a given location in the worksite, material spill can be predicted at the given location based on the predictive material spill model and the terrain value(s), from the terrain map 431, at that given location.
Material spill-to-speed model generator 442 identifies a relationship between material spill detected in in-situ sensor data 440, at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and speed value(s) from the speed map 432 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-speed model generator 442, material spill-to-speed model generator 442 generates a predictive material spill model. The predictive material spill model is used by material spill map generator 452 to predict material spill at different locations in the worksite based upon the georeferenced speed value(s) contained in the speed map 432 at the same locations in the worksite. Thus, for a given location in the worksite, material spill can be predicted at the given location based on the predictive material spill model and the speed value(s), from the speed map 432, at that given location.
Material spill-to-crop moisture model generator 443 identifies a relationship between material spill detected in in-situ sensor data 440, at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and crop moisture value(s) from the crop moisture map 433 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-crop moisture model generator 443, material spill-to-crop moisture model generator 443 generates a predictive material spill model. The predictive material spill model is used by material spill map generator 452 to predict material spill at different locations in the worksite based upon the georeferenced crop moisture value(s) contained in the crop moisture map 433 at the same locations in the worksite. Thus, for a given location in the worksite, material spill can be predicted at the given location based on the predictive material spill model and the crop moisture value(s), from the crop moisture map 433, at that given location.
Material spill-to-fill level model generator 445 identifies a relationship between material spill detected in in-situ sensor data 440, at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and fill level value(s) from the fill level map 435 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-fill level model generator 445, material spill-to-fill level model generator 445 generates a predictive material spill model. The predictive material spill model is used by material spill map generator 452 to predict material spill at different locations in the worksite based upon the georeferenced fill level value(s) contained in the fill level map 435 at the same locations in the worksite. Thus, for a given location in the worksite, material spill can be predicted at the given location based on the predictive material spill model and the fill level value(s), from the fill level map 435, at that given location.
In light of the above, the predictive model generator 310 is operable to produce a plurality of predictive material spill models, as examples of predictive material dynamics models, such as one or more of the predictive material spill models generated by model generators 441, 442, 443, 445 and 447. In another example, two or more of the predictive models described above may be combined into a single predictive material spill model, such as a predictive material spill model that predicts material spill based upon two or more of the terrain value(s), the speed value(s), the crop moisture values, and the fill level values at different locations in the field. Any of these material spill models, or combinations thereof, are represented collectively by predictive material spill model 450 in
The predictive material spill model 450 is provided to predictive map generator 312. In the example of
Material spill map generator 452 receives one or more of the terrain map 431, the speed map 432, the crop moisture map 433, and the fill level map 435 along with the predictive material spill model 450 which predicts material spill based upon one or more of a terrain value, a speed value, a crop moisture value, and a fill level value and generates a predictive map that predicts material spill at different locations in the worksite.
Predictive map generator 312 outputs a functional predictive material spill map 460 that is predictive of material spill. The functional predictive material spill map 460 is an example of a functional predictive material dynamics map 458. The functional predictive material spill map 460 is a predictive map 264. The functional predictive material spill map 460 predicts material spill at different locations in a worksite. The functional predictive material spill map 460 may be provided to control zone generator 313, control system 314, or both. Control zone generator 313 generates control zones and incorporates those control zones into the functional predictive material spill map 460 to produce a predictive control zone map 265, that is a functional predictive material spill control zone map 461. The functional predictive material spill control zone map 461 is an example of a functional predictive material dynamics control zone map 459. One or both of functional predictive material spill map 460 and functional predictive material spill control zone map 461 may be provided to control system 314, which generates control signals to control one or more of the controllable subsystems 316 based upon the functional predictive material spill map 460, the functional predictive material spill control zone map 461, or both.
At block 502, agricultural system 300 receives one or more information maps 358. Examples of information maps 358 or receiving information maps 358 are discussed with respect to blocks 504, 506, 508, and 509. As discussed above, information maps 358 map values of a variable, corresponding to a characteristic, to different locations, as indicated at block 506. As indicated at block 504, receiving the information maps 358 may involve selecting one or more of a plurality of possible information maps 358 that are available. For instance, one information map 358 may be a terrain map, such as terrain map 431. Another information map 358 may be a speed map, such as speed map 432. Another information map may be a crop moisture map, such as crop moisture map 433. Another information map may be a fill level map, such as fill level map 435. Various other maps, such as other maps 437, are contemplated herein. The process by which one or more information maps 358 are selected can be manual, semi-automated, or automated. The information maps 358 can be based on data collected prior to a current operation. For instance, the data may be collected based on aerial images taken during a previous year, or earlier in the current season, or at other times. The data may be based on data detected in ways other than using aerial images. For instance, the data may be collected during a previous operation on the worksite, such an operation during a previous year, or a previous operation earlier in the current season, or at other times. The machines performing those previous operations may be outfitted with one or more sensors that generate sensor data indicative of one or more characteristics. For example, the sensed speed and orientation of mobile machine 100 operating at the worksite in past year may be used as data to generate the information maps 358. In other examples, and as described above, the information maps 358 may be predictive maps having predictive values, such as a predictive speed map having predictive speed values, a predictive terrain map having predictive terrain values, a predictive crop moisture map having predictive crop moisture values, and a predictive fill level map having predictive fill level values. The predictive information map 358 can be generated by predictive map generator 312 based on a model generated by predictive model generator 310. The data for the information maps 358 can be obtained by agricultural system 300 using communication system 306 and stored in data store 302. The data for the information maps 358 can be obtained by agricultural system 300 using communication system 306 in other ways as well, and this is indicated by block 509 in the flow diagram of
As mobile machine 100 is operating, in-situ sensors 308 generate sensor signals indicative of one or more in-situ data values indicative of a characteristic, for example, material dynamics sensors 179, such as material spill and movement sensors 180, generate sensor signals indicative of one or more in-situ data values indicative of material dynamics, such as material spill, as indicated by block 512. In some examples, data from in-situ sensors 308 is georeferenced using position, heading, or speed data from geographic position sensor 304.
Predictive model generator 310 controls one or more material dynamics model generators 439, such as one or more of the material spill-to-terrain model generator 441, material spill-to-speed model generator 442, material spill-to-crop moisture model generator 443, and material spill-to-fill level model generator 445 to generate a model that models the relationship between the mapped values, such as the terrain values, the speed values, the crop moisture values, and the fill level values contained in the respective information map and the in-situ values sensed by the in-situ sensors 308 as indicated by block 514. Predictive model generator 310 generates a predictive material dynamics model 449, such as predictive material spill model 450 as indicated by block 515.
The relationship or model generated by predictive model generator 310 is provided to predictive map generator 312. Predictive map generator 312 controls a predictive material dynamics map generator 451, such as predictive material spill map generator 452, to generate a functional predictive material dynamics map 458, such as functional predictive material spill map 460 that predicts material spill (or sensor value(s) indictive of material spill) at different geographic locations in a worksite at which mobile machine 100 is operating using the predictive material spill model 450 and one or more of the information maps, such as terrain map 431, speed map 432, crop moisture map 433, and fill level map 435 as indicated by block 516.
It should be noted that, in some examples, the functional predictive material spill map 460 may include two or more different map layers. Each map layer may represent a different data type, for instance, a functional predictive material spill map 460 that provides two or more of a map layer that provides predictive material spill based on terrain values from terrain map 431, a map layer that provides predictive material spill based on speed values from speed map 432, a map layer that provides predictive material spill based on crop moisture values from crop moisture map 433, and a map layer that provides predictive material spill based on fill level values from fill level map 435. In other examples, functional predictive material spill map 460 may include a map layer that provides predictive material spill based on two or more of terrain values from terrain map 431, speed values from speed map 432, crop moisture values from crop moisture map 433, and fill level values from fill level map 435.
At block 518, predictive map generator 312 configures the functional predictive material spill map 460 so that the functional predictive material spill map 460 is actionable (or consumable) by control system 314. Predictive map generator 312 can provide the functional predictive material spill map 460 to the control system 314 or to control zone generator 313, or both. Some examples of the different ways in which the functional predictive material spill map 460 can be configured or output are described with respect to blocks 518, 520, 522, and 523. For instance, predictive map generator 312 configures functional predictive material spill map 460 so that functional predictive material spill map 460 includes values that can be read by control system 314 and used as the basis for generating control signals for one or more of the different controllable subsystems 316 of mobile machine 100, as indicated by block 518.
At block 520, control zone generator 313 can divide the functional predictive material spill map 460 into control zones based on the values on the functional predictive material spill map 460 to generate functional predictive material spill control zone map 461, as an example of a functional predictive material dynamics control zone map 459. Contiguously-geolocated values that are within a threshold value of one another can be grouped into a control zone. The threshold value can be a default threshold value, or the threshold value can be set based on an operator input, based on an input from an automated system, or based on other criteria. A size of the zones may be based on a responsiveness of the control system 314, the controllable subsystems 316, based on wear considerations, or on other criteria.
At block 522, predictive map generator 312 configures functional predictive material spill map 460 for presentation to an operator or other user. At block 522, control zone generator 313 can configure functional predictive material spill control zone map 461 for presentation to an operator or other user. When presented to an operator or other user, the presentation of the functional predictive material spill map 460 or of functional predictive material spill control zone map 461 or both may contain one or more of the predictive values on the functional predictive material spill map 460 correlated to geographic location, the control zones of functional predictive material spill control zone map 461 correlated to geographic location, and settings values or control parameters that are used based on the predicted values on predictive map 460 or control zones on predictive control zone map 461. The presentation can, in another example, include more abstracted information or more detailed information. The presentation can also include a confidence level that indicates an accuracy with which the predictive values on predictive 8 map 460 or the control zones on predictive control zone map 461 conform to measured values that may be measured by sensors on mobile machine 100 as mobile machine 100 operates at the worksite. Further where information is presented to more than one location, an authentication and authorization system can be provided to implement authentication and authorization processes. For instance, there may be a hierarchy of individuals that are authorized to view and change maps and other presented information. By way of example, an on-board display device may show the 14 maps in near real time locally on the machine, or the maps may also be generated at one or more remote locations, or both. In some examples, each physical display device at each location may be associated with a person or a user permission level. The user permission level may be used to determine which display elements are visible on the physical display device and which values the corresponding person may change. As an example, a local operator of mobile machine 100 may be unable to see the information corresponding to the predictive map 460 or make any changes to machine operation. A supervisor, such as a supervisor at a remote location, however, may be able to see the predictive map 460 on the display but be prevented from making any changes. A manager, who may be at a separate remote location, may be able to see all of the elements on predictive map 460 and also be able to change the predictive map 460. In some instances, the predictive map 460 accessible and changeable by a manager located remotely may be used in machine control. This is one example of an authorization hierarchy that may be implemented. The predictive map 460 or predictive control zone map 461 or both can be configured in other ways as well, as indicated by block 523.
At block 524, input from geographic position sensor 304 and other in-situ sensors 308 are received by the control system 314. Particularly, at block 526, control system 314 detects an input from the geographic position sensor 304 identifying a geographic location of mobile machine 100. Block 528 represents receipt by the control system 314 of sensor inputs indicative of trajectory or heading of mobile machine 100, and block 530 represents receipt by the control system 314 of a speed of agricultural harvester 100. Block 531 represents receipt by the control system 314 of other information from various in-situ sensors 308.
At block 532, control system 314 generates control signals to control the controllable subsystems 316 based on the functional predictive material spill map 460 or the functional predictive material spill control zone map 461 or both and the input from the geographic position sensor 304 and any other in-situ sensors 308, such as heading and/or speed inputs from heading/speed sensors 325. At block 534, control system 314 applies the control signals to the controllable subsystems 316. It will be appreciated that the particular control signals that are generated, and the particular controllable subsystems 316 that are controlled, may vary based upon one or more different things. For example, the control signals that are generated and the controllable subsystems 316 that are controlled may be based on the type of functional predictive material spill map 460 or functional predictive material spill control zone map 461 or both that is being used. Similarly, the control signals that are generated and the controllable subsystems 216 that are controlled and the timing of the control signals can be based on various latencies of mobile machine 100 and the responsiveness of the controllable subsystems 316.
By way of example, propulsion controller 331 of control system 314 can generate control signals to control propulsion subsystem 350 to control one or more propulsion parameters of mobile machine 100, such as one or more of the speed at which the mobile machine travels, the deceleration of mobile machine 100, and the acceleration of mobile machine 100. For instance, functional predictive material spill map 460 or functional predictive material spill control zone map 461 may predict material spill in area(s) of the worksite ahead of or around mobile machine 100, in which case, propulsion controller 331 can generate control signals to control propulsion system 350 to control a propulsion parameter, such as travel speed, acceleration, deceleration, etc., at those area(s). In another example, path planning controller 334 of control system 314 can generate control signals to control steering subsystem 352 to control a route parameter of mobile machine 100, such as one or more of a commanded path at the worksite over which mobile machine 100 travels, and the steering of mobile machine 100. For instance, functional predictive material spill map 460 or functional predictive material spill control zone map 461, or both, may predict material spill in area(s) of the worksite ahead of or around mobile machine 100, in which case, path planning controller 334 can generate control signals to control steering subsystem 352 to cause mobile machine 100 to avoid traveling those area(s) or to travel through them in a different manner, such as with limited steering angles. In another example, material transfer controller 335 of control system 314 can generate control signals to control material transfer subsystem 341 to initiate or end a material transfer operation. These are merely some examples. Control system 314 can generate a variety of different control signals to control a variety of different controllable subsystems 316 based on functional predictive material spill map 460 or functional predictive material spill control zone map 461, or both.
At block 536, a determination is made as to whether the operation has been completed. If the operation is not completed, the processing advances to block 538 where in-situ sensor data from geographic position sensor 304 and in-situ sensors 308 (and perhaps other sensors) continue to be read.
In some examples, at block 540, agricultural system 300 can also detect learning trigger criteria to perform machine learning on one or more of the functional predictive material spill map 460, functional predictive material spill control zone map 461, predictive material spill model 450, the zones generated by control zone generator 313, one or more control algorithms implemented by the controllers in the control system 314, and other triggered learning.
The learning trigger criteria can include any of a wide variety of different criteria. Some examples of detecting trigger criteria are discussed with respect to blocks 542, 544, 546, 548, and 549. For instance, in some examples, triggered learning can involve recreation of a relationship used to generate a predictive model when a threshold amount of in-situ sensor data are obtained from in-situ sensors 308. In such examples, receipt of an amount of in-situ sensor data from the in-situ sensors 308 that exceeds a threshold triggers or causes the predictive model generator 310 to generate a new predictive model that is used by predictive map generator 312. Thus, as mobile machine 100 continues an operation, receipt of the threshold amount of in-situ sensor data from the in-situ sensors 308 triggers the creation of a new relationship represented by a new predictive material spill model 450 generated by predictive model generator 310. Further, a new functional predictive material spill map 460, a new functional predictive material spill control zone map 461, or both can be generated using the new predictive material spill model 450. Block 542 represents detecting a threshold amount of in-situ sensor data used to trigger creation of a new predictive model.
In other examples, the learning trigger criteria may be based on how much the in-situ sensor data from the in-situ sensors 308 are changing, such as over time or compared to previous values. For example, if variations within the in-situ sensor data (or the relationship between the in-situ sensor data and the information in the one or more information maps 358) are within a selected range or is less than a defined amount, or below a threshold value, then a new predictive model is not generated by the predictive model generator 310. As a result, the predictive map generator 312 does not generate a new functional predictive material spill map 460, a new functional predictive material spill control zone map 461, or both. However, if variations within the in-situ sensor data are outside of the selected range, are greater than the defined amount, or are above the threshold value, for example, then the predictive model generator 310 generates a new predictive model 450 using all or a portion of the newly received in-situ sensor data that the predictive map generator 312 uses to generate a new predictive map 460 which can be provided to control zone generator 313 for the creation of a new predictive control zone map 461. At block 544, variations in the in-situ sensor data, such as a magnitude of an amount by which the data exceeds the selected range or a magnitude of the variation of the relationship between the in-situ sensor data and the information in the one or more information maps, can be used as a trigger to cause generation of one or more of a new predictive model 450, a new predictive map 460, and a new predictive control zone map 461. Keeping with the examples described above, the threshold, the range, and the defined amount can be set to default values; set by an operator or user interaction through a user interface; set by an automated system; or set in other ways.
Other learning trigger criteria can also be used. For instance, if predictive model generator 310 switches to a different information map (different from the originally selected information map), then switching to the different information map may trigger re-learning by predictive model generator 310, predictive map generator 312, control zone generator 313, control system 314, or other items. In another example, transitioning of mobile machine 100 to a different topography or to a different control zone may be used as learning trigger criteria as well.
In some instances, operator 360 or user 366 can also edit the functional predictive material spill map 460 or functional predictive material spill control zone map 461 or both. The edits can change a value on the functional predictive material spill map 460, change a size, shape, position, or existence of a control zone on functional predictive material spill control zone map 461, or both. Block 546 shows that edited information can be used as learning trigger criteria.
In some instances, it may also be that operator 360 or user 366 observes that automated control of a controllable subsystem 316, is not what the operator or user desires. In such instances, the operator 360 or user 366 may provide a manual adjustment to the controllable subsystem 316 reflecting that the operator 360 or user 366 desires the controllable subsystem 316 to operate in a different way than is being commanded by control system 314. Thus, manual alteration of a setting by the operator 360 or user 366 can cause one or more of predictive model generator 310 to relearn a model, predictive map generator 312 to regenerate functional predictive material spill map 460, control zone generator 313 to regenerate one or more control zones on functional predictive material spill control zone map 461, and control system 314 to relearn a control algorithm or to perform machine learning on one or more of the controller components 329 through 337 in control system 314 based upon the adjustment by the operator 360 or user 366, as shown in block 548. Block 549 represents the use of other triggered learning criteria.
In other examples, relearning may be performed periodically or intermittently based, for example, upon a selected time interval such as a discrete time interval or a variable time 14 interval, as indicated by block 550.
If relearning is triggered, whether based upon learning trigger criteria or based upon passage of a time interval, as indicated by block 550, then one or more of the predictive model generator 310, predictive map generator 312, control zone generator 313, and control system 314 performs machine learning to generate a new predictive model, a new predictive map, a new control zone, and a new control algorithm, respectively, based upon the learning trigger criteria. The new predictive model, the new predictive map, the new control zone, and the new control algorithm are generated using any additional data that has been collected since the last learning operation was performed. Performing relearning is indicated by block 552.
If the operation has not been completed, operation moves from block 552 to block 518 such that operation of the mobile machine 100 can be controlled based on the new predictive map, a new control zone, or a new control algorithm.
If the operation has been completed, operation moves from block 552 to block 554 where one or more of the functional predictive material spill map 460, functional predictive material spill control zone map 461, the predictive material spill model 450 generated by predictive model generator 310, control zone(s), and control algorithm(s) are stored. The predictive map 460, predictive control zone map 461, predictive model 450, control zone(s), and control algorithm(s) may be stored locally on data store 302 or sent to a remote system using communication system 306 for later use.
As shown in
Material movement-to-terrain model generator 641 identifies a relationship between material movement detected in in-situ sensor data 640, at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and terrain value(s) from the terrain map 631 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-terrain model generator 641, material movement-to-terrain model generator 641 generates a predictive material movement model. The predictive material movement model is used by material movement map generator 652 to predict material movement at different locations in the worksite based upon the georeferenced terrain value(s) contained in the terrain map 631 at the same locations in the worksite. Thus, for a given location in the worksite, material movement can be predicted at the given location based on the predictive material movement model and the terrain value(s), from the terrain map 631, at that given location.
Material movement-to-speed model generator 642 identifies a relationship between material movement detected in in-situ sensor data 640, at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and speed value(s) from the speed map 632 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-speed model generator 642, material movement-to-speed model generator 642 generates a predictive material movement model. The predictive material movement model is used by material movement map generator 652 to predict material movement at different locations in the worksite based upon the georeferenced speed value(s) contained in the speed map 632 at the same locations in the worksite. Thus, for a given location in the worksite, material movement can be predicted at the given location based on the predictive material movement model and the speed value(s), from the speed map 632, at that given location.
Material movement-to-crop moisture model generator 643 identifies a relationship between material movement detected in in-situ sensor data 640, at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and crop moisture value(s) from the crop moisture map 632 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-crop moisture model generator 643, material movement-to-crop moisture model generator 643 generates a predictive material movement model. The predictive material movement model is used by material movement map generator 652 to predict material movement at different locations in the worksite based upon the georeferenced crop moisture value(s) contained in the crop moisture map 633 at the same locations in the worksite. Thus, for a given location in the worksite, material movement can be predicted at the given location based on the predictive material movement model and the crop moisture value(s), from the crop moisture map 633, at that given location.
Material movement-to-fill level model generator 645 identifies a relationship between material movement detected in in-situ sensor data 640, at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and fill level value(s) from the fill level map 635 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-fill level model generator 645, material movement-to-fill level model generator 645 generates a predictive material movement model. The predictive material movement model is used by material movement map generator 652 to predict material movement at different locations in the worksite based upon the georeferenced fill level value(s) contained in the fill level map 635 at the same locations in the worksite. Thus, for a given location in the worksite, material movement can be predicted at the given location based on the predictive material movement model and the fill level value(s), from the fill level map 635, at that given location.
In light of the above, the predictive model generator 310 is operable to produce a plurality of predictive material movement models, as examples of predictive material dynamics models, such as one or more of the predictive material movement models generated by model generators 641, 642, 643, 645 and 647. In another example, two or more of the predictive models described above may be combined into a single predictive material movement model, such as a predictive material movement model that predicts material movement based upon two or more of the terrain value(s), the speed value(s), the crop moisture value(s), and the fill level value(s) at different locations in the worksite. Any of these material spill models, or combinations thereof, are represented collectively by predictive material movement model 650 in
The predictive material spill model 650 is provided to predictive map generator 312. In the example of
Material movement map generator 652 receives one or more of the terrain map 631. the speed map 632, the crop moisture map 633, and the fill level map 635 along with the predictive material movement model 650 which predicts material movement based upon one or more of a terrain value, a speed value, a crop moisture value, and a fill level value and generates a predictive map that predicts material movement at different locations in the worksite.
Predictive map generator 312 outputs a functional predictive material movement map 660 that is predictive of material movement. The functional predictive material movement map 660 is an example of a functional predictive material dynamics map 458. The functional predictive material movement map 660 is a predictive map 264. The functional predictive material movement map 660 predicts material movement at different locations in a worksite. The functional predictive material movement map 660 may be provided to control zone generator 313, control system 314, or both. Control zone generator 313 generates control zones and incorporates those control zones into the functional predictive material movement map 660 to produce a predictive control zone map 265, that is a functional predictive material movement control zone map 661. The functional predictive material movement control zone map 661 is an example of a functional predictive material dynamics control zone map 459. One or both of functional predictive material movement map 660 and functional predictive material movement control zone map 661 may be provided to control system 314, which generates control signals to control one or more of the controllable subsystems 316 based upon the functional predictive material movement map 660, the functional predictive material movement control zone map 661, or both.
At block 702, agricultural system 300 receives one or more information maps 358. Examples of information maps 358 or receiving information maps 358 are discussed with respect to blocks 704, 706, 708, and 709. As discussed above, information maps 358 map values of a variable, corresponding to a characteristic, to different locations, as indicated at block 706. As indicated at block 704, receiving the information maps 358 may involve selecting one or more of a plurality of possible information maps 358 that are available. For instance, one information map 358 may be a terrain map, such as terrain map 631. Another information map 358 may be a speed map, such as speed map 632. Another information map 358 may be a crop moisture map, such as crop moisture map 633. Another information map may be a fill level map, such as fill level map 635. Various other maps, such as other maps 637, are contemplated herein. The process by which one or more information maps 358 are selected can be manual, semi-automated, or automated. The information maps 358 can be based on data collected prior to a current operation. For instance, the data may be collected based on aerial images taken during a previous year, or earlier in the current season, or at other times. The data may be based on data detected in ways other than using aerial images. For instance, the data may be collected during a previous operation on the worksite, such an operation during a previous year, or a previous operation earlier in the current season, or at other times. The machines performing those previous operations may be outfitted with one or more sensors that generate sensor data indicative of one or more characteristics. For example, the sensed speed and orientation of mobile machine 100 operating at the worksite in a past year may be used as data to generate the information maps 358. In other examples, and as described above, the information maps 358 may be predictive maps having predictive values, such as a predictive speed map having predictive speed values, a predictive terrain map having predictive terrain values, a predictive crop moisture map having predictive crop moisture values, and a predictive fill level map having predictive fill level values. The predictive information map 358 can be generated by predictive map generator 312 based on a model generated by predictive model generator 310. The data for the information maps 358 or the information maps 358, or both, can be obtained by agricultural system 300 using communication system 306 and stored in data store 302. The data for the information maps 358 can be obtained by agricultural system 300 using communication system 306 in other ways as well, and this is indicated by block 709 in the flow diagram of
As mobile machine 100 is operating, in-situ sensors 308 generate sensor signals indicative of one or more in-situ data values indicative of a characteristic, for example, material dynamics sensors 179, such as material spill and movement sensors 180, generate sensor signals indicative of one or more in-situ data values indicative of material dynamics, such as material movement, as indicated by block 712. In some examples, data from in-situ sensors 308 is georeferenced using position, heading, or speed data from geographic position sensor 304.
Predictive model generator 310 controls one or more of the material dynamics model generators 439, such as one or more of the material movement-to-terrain model generator 641, the material movement-to-speed model generator 642, the material movement-to-crop moisture model generator 643, and the material movement-to-fill level model generator 645 to generate a model that models the relationship between the mapped values, such as the terrain values, the speed values, the crop moisture values, and the fill level values contained in the respective information map and the in-situ values sensed by the in-situ sensors 308 as indicated by block 714. Predictive model generator 310 generates a predictive material dynamics model 449, such as predictive material movement model 650 as indicated by block 715.
The relationship or model generated by predictive model generator 310 is provided to predictive map generator 312. Predictive map generator 312 controls predictive material 8 dynamics map generator 451, such as predictive material movement map generator 652 to generate a functional predictive material dynamics map 458, such as functional predictive material movement map 660 that predicts material movement (or sensor value(s) indicative of material movement) at different geographic locations in a worksite at which mobile machine 100 is operating using the predictive material movement model 650 and one or more of the information maps, such as terrain map 631, speed map 632, crop moisture map 633, and fill level map 635 as indicated by block 716.
It should be noted that, in some examples, the functional predictive material movement map 660 may include two or more different map layers. Each map layer may represent a different data type, for instance, a functional predictive material movement map 660 that provides two or more of a map layer that provides predictive material movement based on terrain values from terrain map 631, a map layer that provides predictive material movement based on speed values from speed map 632, a map layer that provides predictive material movement based on crop moisture values from crop moisture map 633, and a map layer that provides predictive material movement based on fill level values from fill level map 635. In other examples, functional predictive material movement can include a map layer that provides predictive material movement based on two or more of terrain values from terrain map 631, speed values from speed map 632, crop moisture values from crop moisture map 633, and fill level values from fill level map 635.
At block 718, predictive map generator 312 configures the functional predictive material movement map 660 so that the functional predictive material movement map 660 is actionable (or consumable) by control system 314. Predictive map generator 312 can provide the functional predictive material movement map 660 to the control system 314 or to control zone generator 313, or both. Some examples of the different ways in which the functional predictive material movement map 660 can be configured or output are described with respect to blocks 718, 720, 722, and 723. For instance, predictive map generator 312 configures functional predictive material movement map 660 so that functional predictive material movement map 660 includes values that can be read by control system 314 and used as the basis for generating control signals for one or more of the different controllable subsystems 316 of mobile machine 100, as indicated by block 718.
At block 720, control zone generator 313 can divide the functional predictive material movement map 660 into control zones based on the values on the functional predictive material movement map 660 to generate functional predictive material movement control zone map 661, as an example of a functional predictive material dynamics control zone map 459. Contiguously-geolocated values that are within a threshold value of one another can be grouped into a control zone. The threshold value can be a default threshold value, or the threshold value can be set based on an operator input, based on an input from an automated system, or based on other criteria. A size of the zones may be based on a responsiveness of the control system 314, the controllable subsystems 316, based on wear considerations, or on other criteria.
At block 722, predictive map generator 312 configures functional predictive material movement map 660 for presentation to an operator or other user. At block 722, control zone generator 313 can configure functional predictive material movement control zone map 661 for presentation to an operator or other user. When presented to an operator or other user, the presentation of the functional predictive material movement map 660 or of the functional predictive material movement control zone map 661 or both may contain one or more of the predictive values on the functional predictive material movement map 660 correlated to geographic location, the control zones of functional predictive material movement control zone map 661 correlated to geographic location, and settings values or control parameters that are used based on the predicted values on predictive map 660 or control zones on predictive control zone map 661. The presentation can, in another example, include more abstracted information or more detailed information. The presentation can also include a confidence level that indicates an accuracy with which the predictive values on predictive map 660 or the control zones on predictive control zone map 661 conform to measured values that may be measured by sensors on mobile machine 100 as mobile machine 100 operates at the worksite. Further where information is presented to more than one location, an authentication and authorization system can be provided to implement authentication and authorization processes. For instance, there may be a hierarchy of individuals that are authorized to view and change maps and other presented information. By way of example, an on-board display device may show the maps in near real time locally on the machine, or the maps may also be generated at one or more remote locations, or both. In some examples, each physical display device at each location may be associated with a person or a user permission level. The user permission level may be used to determine which display elements are visible on the physical display device and which values the corresponding person may change. As an example, a local operator of mobile machine 100 may be unable to see the information corresponding to the predictive map 660 or make any changes to machine operation. A supervisor, such as a supervisor at a remote location, however, may be able to see the predictive map 660 on the display but be prevented from making any changes. A manager, who may be at a separate remote location, may be able to see all of the elements on predictive map 660 and also be able to change the predictive map 660. In some instances, the predictive map 660 accessible and changeable by a manager located remotely may be used in machine control. This is one example of an authorization hierarchy that may be implemented. The predictive map 660 or predictive control zone map 661 or both can be configured in other ways as well, as indicated by block 723.
At block 724, input from geographic position sensor 304 and other in-situ sensors 308 are received by the control system 314. Particularly, at block 726, control system 314 detects an input from the geographic position sensor 304 identifying a geographic location of mobile machine 100. Block 728 represents receipt by the control system 314 of sensor inputs indicative of trajectory or heading of mobile machine 100, and block 730 represents receipt by the control system 314 of a speed of agricultural harvester 100. Block 731 represents receipt by the control system 314 of other information from various in-situ sensors 308.
At block 732, control system 314 generates control signals to control the controllable subsystems 316 based on the functional predictive material movement map 660 or the functional predictive material movement control zone map 661 or both and the input from the geographic position sensor 304 and any other in-situ sensors 308, such as heading and/or speed inputs from heading/speed sensors 325. At block 734, control system 314 applies the control signals to the controllable subsystems 316. It will be appreciated that the particular control signals that are generated, and the particular controllable subsystems 316 that are controlled, may vary based upon one or more different things. For example, the control signals that are generated and the controllable subsystems 316 that are controlled may be based on the type of functional predictive material movement map 660 or functional predictive material movement control zone map 661 or both that is being used. Similarly, the control signals that are generated and the controllable subsystems 316 that are controlled and the timing of the control signals can be based on various latencies of mobile machine 100 and the responsiveness of the controllable subsystems 316.
By way of example, propulsion controller 331 of control system 314 can generate control signals to control propulsion subsystem 350 to control one or more propulsion parameters of mobile machine 100, such as one or more of the speed at which the mobile machine travels, the deceleration of mobile machine 100, and the acceleration of mobile machine 100. For instance, functional predictive material movement map 660 or functional predictive material movement control zone map 661 may predict material movement in area(s) of the worksite ahead of or around mobile machine 100, in which case, propulsion controller 331 can generate control signals to control propulsion system 350 to control a propulsion parameter, such as travel speed, acceleration, deceleration, etc., at those area(s). In another example, path planning controller 334 of control system 314 can generate control signals to control steering subsystem 352 to control a route parameter of mobile machine 100, such as one or more of a commanded path at the worksite over which mobile machine 100 travels, and the steering of mobile machine 100. For instance, functional predictive material movement map 660 or functional predictive material movement control zone map 661, or both, may predict material movement in area(s) of the worksite ahead of or around mobile machine 100, in which case, path planning controller 334 can generate control signals to control steering subsystem 352 to cause mobile machine 100 to avoid traveling those area(s) or to travel through them in a different manner, such as with limited steering angles. In another example, material transfer controller 335 of control system 314 can generate control signals to control material transfer subsystem 341 to initiate or end a material transfer operation. These are merely some examples. Control system 314 can generate a variety of different control signals to control a variety of different controllable subsystems 316 based on functional predictive material movement map 660 or functional predictive material movement control zone map 661, or both.
At block 736, a determination is made as to whether the operation has been completed. If the operation is not completed, the processing advances to block 738 where in-situ sensor data from geographic position sensor 304 and in-situ sensors 308 (and perhaps other sensors) continue to be read.
In some examples, at block 740, agricultural system 300 can also detect learning trigger criteria to perform machine learning on one or more of the functional predictive material movement map 660, functional predictive material movement control zone map 661, predictive material movement model 650, the zones generated by control zone generator 313, one or more control algorithms implemented by the controllers in the control system 314, and other triggered learning.
The learning trigger criteria can include any of a wide variety of different criteria. Some examples of detecting trigger criteria are discussed with respect to blocks 742, 744, 746, 748, and 749. For instance, in some examples, triggered learning can involve recreation of a relationship used to generate a predictive model when a threshold amount of in-situ sensor data are obtained from in-situ sensors 308. In such examples, receipt of an amount of in-situ sensor data from the in-situ sensors 308 that exceeds a threshold triggers or causes the predictive model generator 310 to generate a new predictive model that is used by predictive map generator 312. Thus, as mobile machine 100 continues an operation, receipt of the threshold amount of in-situ sensor data from the in-situ sensors 308 triggers the creation of a new relationship represented by a new predictive material movement model 650 generated by predictive model generator 310. Further, a new functional predictive material movement map 660, a new functional predictive material movement control zone map 661, or both can be generated using the new predictive material movement model 650. Block 742 represents detecting a threshold amount of in-situ sensor data used to trigger creation of a new predictive model.
In other examples, the learning trigger criteria may be based on how much the in-situ sensor data from the in-situ sensors 308 are changing, such as over time or compared to previous values. For example, if variations within the in-situ sensor data (or the relationship between the in-situ sensor data and the information in the one or more information maps 358) are within a selected range or is less than a defined amount, or below a threshold value, then a new predictive model is not generated by the predictive model generator 310. As a result, the predictive map generator 312 does not generate a new functional predictive material movement map 660, a new functional predictive material movement control zone map 661, or both. However, if variations within the in-situ sensor data are outside of the selected range, are greater than the defined amount, or are above the threshold value, for example, then the predictive model generator 310 generates a new predictive model 650 using all or a portion of the newly received in-situ sensor data that the predictive map generator 312 uses to generate a new predictive map 660 which can be provided to control zone generator 313 for the creation of a new predictive control zone map 661. At block 744, variations in the in-situ sensor data, such as a magnitude of an amount by which the data exceeds the selected range or a magnitude of the variation of the relationship between the in-situ sensor data and the information in the one or more information maps, can be used as a trigger to cause generation of one or more of a new predictive model 650, a new predictive map 660, and a new predictive control zone map 661. Keeping with the examples described above, the threshold, the range, and the defined amount can be set to default values; set by an operator or user interaction through a user interface; set by an automated system; or set in other ways.
Other learning trigger criteria can also be used. For instance, if predictive model generator 310 switches to a different information map (different from the originally selected information map), then switching to the different information map may trigger re-learning by predictive model generator 310, predictive map generator 312, control zone generator 313, control system 314, or other items. In another example, transitioning of mobile machine 100 to a different topography or to a different control zone may be used as learning trigger criteria as well.
In some instances, operator 360 or user 366 can also edit the functional predictive material movement map 660 or functional predictive material movement control zone map 661 or both. The edits can change a value on the functional predictive material movement map 660, change a size, shape, position, or existence of a control zone on functional predictive material movement control zone map 661, or both. Block 746 shows that edited information can be used as learning trigger criteria.
In some instances, it may also be that operator 360 or user 366 observes that automated control of a controllable subsystem 316, is not what the operator or user desires. In such instances, the operator 360 or user 366 may provide a manual adjustment to the controllable subsystem 316 reflecting that the operator 360 or user 366 desires the controllable subsystem 316 to operate in a different way than is being commanded by control system 314. Thus, manual alteration of a setting by the operator 360 or user 366 can cause one or more of predictive model generator 310 to relearn a model, predictive map generator 312 to regenerate functional predictive material movement map 660, control zone generator 313 to regenerate one or more control zones on functional predictive material movement control zone map 661, and control system 314 to relearn a control algorithm or to perform machine learning on one or more of the controller components 329 through 337 in control system 314 based upon the adjustment by the operator 360 or user 366, as shown in block 748. Block 749 represents the use of other triggered learning criteria.
In other examples, relearning may be performed periodically or intermittently based, for example, upon a selected time interval such as a discrete time interval or a variable time interval, as indicated by block 750.
If relearning is triggered, whether based upon learning trigger criteria or based upon passage of a time interval, as indicated by block 750, then one or more of the predictive model generator 310, predictive map generator 312, control zone generator 313, and control system 314 performs machine learning to generate a new predictive model, a new predictive map, a new control zone, and a new control algorithm, respectively, based upon the learning trigger criteria. The new predictive model, the new predictive map, the new control zone, and the new control algorithm are generated using any additional data that has been collected since the last learning operation was performed. Performing relearning is indicated by block 752.
If the operation has not been completed, operation moves from block 752 to block 718 such that operation of the mobile machine 100 can be controlled based on the new predictive map, a new control zone, or a new control algorithm.
If the operation has been completed, operation moves from block 752 to block 754 where one or more of the functional predictive material movement map 660, functional predictive material movement control zone map 661, the predictive material movement model 650 generated by predictive model generator 310, control zone(s), and control algorithm(s), are stored. The predictive map 660, predictive control zone map 661, predictive model 650, control zone(s), and control algorithm(s), may be stored locally on data store 302 or sent to a remote system using communication system 306 for later use.
As shown in
Material movement-to-machine orientation model generator 841 identifies a relationship between material movement detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and machine orientation detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-machine orientation model generator 841, material movement-to-machine orientation model generator 841 generates a predictive material movement model. The predictive material movement model is used to predict material spill based on machine orientation. Thus, machine orientation characteristics, such as roll, pitch, and/or yaw, of the machine can be detected or input and based on the detected or input machine orientation characteristics and the model, a material movement can be predicted. The predictive material movement model generated by material movement-to-machine orientation model generator 841 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material movement-to-speed model generator 842 identifies a relationship between material movement detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and machine speed characteristics (e.g., travel speed, acceleration, deceleration, etc.) detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-speed model generator 842, material movement-to-speed model generator 842 generates a predictive material movement model. The predictive material movement model is used to predict material spill based on machine speed characteristics. Thus, speed characteristic(s) of the machine can be detected or input and based on the detected or input speed characteristics and the model, a material movement can be predicted. The predictive material movement model generated by material movement-to-speed model generator 842 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material movement-to-crop moisture model generator 843 identifies a relationship between material movement detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and crop moisture detected in in-situ sensor data 840. Based on this relationship established by material movement-to-crop moisture model generator 843, material movement-to-crop moisture model generator 843 generates a predictive material movement model. The predictive material movement model is used to predict material spill based on crop moisture. Thus, a moisture of the crop material can be detected or input and based on the detected or input crop moisture and the model, a material movement can be predicted. The predictive material movement model generated by material movement-to-crop moisture model generator 843 can be used in the control of mobile machine 100. As discussed above, the moisture level of crop carried by mobile machine 100 can affect the angle of repose for a crop material pile carried by mobile machine 100 and thus the crop material may be more or less likely to shift position under certain forces. The predictive material movement model generated by material movement-to-crop moisture model generator 843 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material movement-to-material mass model generator 844 identifies a relationship between material movement detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and mass of material carried by mobile machine 100 detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-material mass model generator 844, material movement-to-material mass model generator 844 generates a predictive material movement model. The predictive material movement model is used to predict material spill based on material mass. Thus, a mass of the material can be detected or input and based on the detected or input mass of the material and the model, a material movement can be predicted. The predictive material movement model generated by material movement-to-material mass model generator 844 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material movement-to-material center of mass model generator 845 identifies a relationship between material movement detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and the center of mass of material carried by mobile machine 100 detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-material center of mass model generator 845, material movement-to-material center of mass model generator 845 generates a predictive material movement model. The predictive material movement model generated by material movement-to-material center of mass model generator 845 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material movement-to-fill level model generator 846 identifies a relationship between material movement detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material movement were obtained, and the fill level of material receptacle 307 of mobile machine 100 detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material movement was detected. Based on this relationship established by material movement-to-fill level model generator 846, material movement-to-fill level model generator 846 generates a predictive material movement model. The predictive material movement model is used to predict material movement based on fill level of material receptacle 307. Thus, a fill level of material receptacle 307 can be detected or input and based on the detected or input fill level and the model, a material movement can be predicted. The predictive material movement model generated by material movement-to-fill level model generator 846 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
In light of the above, the predictive model generator 310 is operable to produce a plurality of predictive material movement models, such as one or more of the predictive material movement models generated by the model generators 841, 842, 843, 844, 845, 846, and 854. In another example, two or more of the predictive models described above may be combined into a single predictive material movement model, such as a predictive material movement model that predicts material movement based upon two or more of speed value(s), machine orientation value(s), crop moisture value(s), material mass value(s), material center of mass value(s), and fill level value(s). Any of these material movement models, or combinations thereof, are represented by predictive material movement model 860 in
Predictive material movement model 860 may be provided to control system 314 for use in control of mobile machine 100 or can be presented to the operator or other user, or both.
Material spill-to-machine orientation model generator 847 identifies a relationship between material spill detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and machine orientation detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-machine orientation model generator 847, material spill-to-machine orientation model generator 847 generates a predictive material spill model. The predictive material movement model is used to predict material spill based on machine orientation. Thus, machine orientation characteristics, such as roll, pitch, and/or yaw, of the machine can be detected or input and based on the detected or input machine orientation characteristics and the model, a material spill can be predicted. The predictive material spill model generated by material spill-to-machine orientation model generator 847 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material spill-to-speed model generator 848 identifies a relationship between material spill detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and machine speed characteristics (e.g., travel speed, acceleration, deceleration, etc.) detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-speed model generator 848, material spill-to-speed model generator 848 generates a predictive material spill model. The predictive material spill model is used to predict material spill based on machine speed characteristics. Thus, speed characteristic(s) of the machine can be detected or input and based on the detected or input speed characteristics and the model, a material spill can be predicted. The predictive material spill model generated by material spill-to-speed model generator 848 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material spill-to-crop moisture model generator 849 identifies a relationship between material spill detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and crop moisture detected in in-situ sensor data 840. Based on this relationship established by material spill-to-crop moisture model generator 849, material spill-to-crop moisture model generator 849 generates a predictive material spill model. The predictive material spill model is used to predict material spill based on crop moisture. Thus, a moisture of the crop material can be detected or input and based on the detected or input crop moisture and the model, a material spill can be predicted. The predictive material spill model generated by material spill-to-crop moisture model generator 843 can be used in the control of mobile machine 100. As discussed above, the moisture level of crop carried by mobile machine 100 can affect the angle of repose for a crop material pile carried by mobile machine 100 and thus the crop material may be more or less likely to shift position under certain forces, and thus may be more or less likely to spill out of the mobile machine 100. The predictive material spill model generated by material spill-to-crop moisture model generator 849 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material spill-to-material mass model generator 850 identifies a relationship between material spill detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and mass of material carried by mobile machine 100 detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-material mass model generator 850, material spill-to-material mass model generator 850 generates a predictive material spill model. The predictive material spill model is used to predict material spill based on material mass. Thus, a mass of the material can be detected or input and based on the detected or input mass of the material and the model, a material spill can be predicted. The predictive material spill model generated by material spill-to-material mass model generator 850 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material spill-to-material center of mass model generator 851 identifies a relationship between material spill detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the material spill were obtained, and the center of mass of material carried by mobile machine 100 detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-material center of mass model generator 851, material spill-to-material center of mass model generator 851 generates a predictive material spill 19 model. The predictive material spill model generated by material spill-to-material center of mass model generator 851 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
Material spill-to-fill level model generator 852 identifies a relationship between material spill detected in in-situ sensor data 840 at a geographic location corresponding to where the sensor data indicating the spill were obtained, and the fill level of material receptacle 307 of mobile machine 100 detected in in-situ sensor data 840 corresponding to the same location in the worksite where the material spill was detected. Based on this relationship established by material spill-to-fill level model generator 852, material spill-to-fill level model generator 852 generates a predictive material spill model. The predictive material spill model is used to predict material spill based on fill level of material receptacle 307. Thus, a fill level of material receptacle 307 can be detected or input and based on the detected or input fill level and the model, a material spill can be predicted. The predictive material spill model generated by material spill-to-fill level model generator 852 can be used in the control of mobile machine 100, such as to control one or more of controllable subsystems 316.
In light of the above, the predictive model generator 310 is operable to produce a plurality of predictive material spill models, such as one or more of the predictive material spill models generated by the model generators 847, 848, 849, 850, 851, 852, and 854. In another example, two or more of the predictive models described above may be combined into a single predictive material spill model, such as a predictive material spill model that predicts material spill based upon two or more of speed value(s), machine orientation value(s), crop moisture value(s), material mass value(s), material center of mass value(s), and fill level value(s). Any of these material spill models, or combinations thereof, are represented by predictive material spill model 861 in
Predictive material spill model 861 may be provided to control system 314 for use in control of mobile machine 100 or can be presented to the operator or other user, or both.
In some examples, the predictive material spill model(s) 449 (e.g., one or more of predictive material movement model 860 and predictive material spill model) are provided to predictive map generator 312. As illustrated in
As illustrated in
Material movement generator 855 receives one or more of the terrain map 831, the speed map 832, the crop moisture map 833, and the fill level map 835 along with the predictive material movement model 860 which predicts material movement based upon one or more of a terrain (or machine orientation) value, a speed value, a crop moisture value, and a fill level value and generates a predictive map that predicts material movement at different locations in the worksite.
Predictive map generator 312 outputs a functional predictive material movement map 870 that is predictive of material movement. The functional predictive material movement map 870 is an example of a functional predictive material dynamics map 458. The functional predictive material movement map 870 is a predictive map 264. The functional predictive material movement map 870 predicts material movement at different locations in a worksite. The functional predictive material movement map 870 may be provided to control zone generator 313, control system 314, or both. Control zone generator 313 generates control zones and incorporates those control zones into the functional predictive material movement map 870 to produce a predictive control zone map 265, that is a functional predictive material movement control zone map 873. The functional predictive material movement control zone map 873 is an example of a functional predictive material dynamics control zone map 459. One or both of functional predictive material movement map 870 and functional predictive material movement control zone map 873 may be provided to control system 314, which generates control signals to control one or more of the controllable subsystems 316 based upon the functional predictive material movement map 870, the functional predictive material movement control zone map 873, or both.
The functional predictive material movement map 870 includes predictive values of material movement at different locations at the worksite. The predictive values are based on the predictive material movement model 860 and one or more of the terrain (or machine orientation) value from the terrain map 831, the speed value from the speed map 832, the crop moisture value from the crop moisture map 833, and the fill level value from the fill level map 835 at those different locations.
Material spill map generator 856 receives one or more of the terrain map 831, the speed map 832, the crop moisture map 833, and the fill level map 835 along with the predictive material spill model 861 which predicts material spill based upon one or more of a terrain (or machine orientation) value, a speed value, a crop moisture value, and a fill level value and generates a predictive map that predicts material spill at different locations in the worksite.
Predictive map generator 312 outputs a functional predictive material spill map 868 that is predictive of material spill. The functional predictive material spill map 868 is an example of a functional predictive material dynamics map 458. The functional predictive material spill map 868 is a predictive map 264. The functional predictive material spill map 868 predicts material spill at different locations in a worksite. The functional predictive material spill map 868 may be provided to control zone generator 313, control system 314, or both. Control zone generator 313 generates control zones and incorporates those control zones into the functional predictive material spill map 868 to produce a predictive control zone map 265, that is a functional predictive material spill control zone map 871. The functional predictive material spill control zone map 871 is an example of a functional predictive material dynamics control zone map 459. One or both of functional predictive material spill map 868 and functional predictive material spill control zone map 871 may be provided to control system 314, which generates control signals to control one or more of the controllable subsystems 316 based upon the functional predictive material spill map 868, the functional predictive material spill control zone map 871, or both.
The functional predictive material spill map 868 includes predictive values of material spill at different locations at the worksite. The predictive values are based on the predictive material spill model 861 and one or more of the terrain (or machine orientation) value from the terrain map 831, the speed value from the speed map 832, the crop moisture value from the crop moisture map 833, and the fill level value from the fill level map 835 at those different locations.
At block 902, in-situ sensors 308 generate sensor signals indicative of one or more in-situ data values indicative of characteristics as the mobile machine 100 is operating at a worksite. Material dynamics sensors 179, such as material spill and movement sensors 180, generate sensor signals indicative of one or more in-situ data values indicative of material dynamics characteristics, such as material movement characteristics or material spill characteristics, as indicated by block 903. Heading/speed sensors 325 generate sensor signals indicative of one or more in-situ data values indicative of speed characteristic(s), such as one or more of travel speed, acceleration, and deceleration, as indicated by block 904. As described herein, in some examples, geographic position data generated by geographic position sensor 304 may also be used to generate one or more in-situ data values indicative of speed characteristic(s), such as one or more of travel speed, acceleration, and deceleration. Crop moisture sensors 326 generate sensor signals indicative of one or more in-situ data values indicative of moisture of crop material collected by mobile machine 100, as indicated by block 905. Machine orientation sensors 327 generate sensor signals indicative of machine orientation characteristics, such as one or more of roll, pitch, and yaw of the machine, as indicated by block 906. Fill level sensors 324 generate sensor signals indicative of a fill level of material receptacle 307, as indicated by block 907. As described herein, in some examples, material spill and movement sensors 180 may also generate sensor signals indicative of one or more in-situ data values indicative of fill level of material receptacle 307. Material spill and movement sensors 180 can generate sensor signals indicative of one or more in-situ data values indicative of a mass of material in material receptacle 307, as indicated by block 908. Material spill and movement sensors 180 can generate sensor signals indicative of one or more in-situ data values indicative of a center of mass of material in material receptacle 307, as indicated by block 909. Various other in-situ sensors can generate sensor signals indicative of various other in-situ data values indicative of various other characteristics, as indicated by block 912. In some examples, data from in-situ sensors 308 is georeferenced using position, heading, or speed data from geographic position sensor 304.
In one example, at block 914, predictive model generator 310 controls one or more of the material movement-to-machine orientation model generator 841, the material movement-to-speed model generator 842, the material movement-to-crop moisture model generator 843, the material movement-to-material mass model generator 844, the material movement-to-material center of mass model generator 845, and the material movement-to-fill level model generator 846 to generate a model that models the relationship between one or more of the in-situ machine orientation values, the in-situ speed values, the in-situ crop moisture values, the in-situ mass values, the in-situ center of mass values, and the in-situ fill level value and in-situ material movement values. Predictive model generator 310 generates a predictive material movement model 860 as indicated by block 915. Predictive material movement model 860 predicts material movement based on one or more of machine orientation characteristics, speed characteristics, crop moisture, material mass, material center of mass, and fill level.
In another example, at block 914, predictive model generator 310 controls one or more of the material spill-to-machine orientation model generator 847, the material spill-to-speed model generator 848, the material spill-to-crop moisture model generator 849, the material spill-to-material mass model generator 850, the material spill-to-material center of mass model generator 851, and the material spill-to-material fill level model generator 852 to generate a model that models the relationship between one or more of the in-situ machine orientation values, the in-situ speed values, the in-situ crop moisture values, the in-situ material mass values, the in-situ material center of mass values, and the in-situ material fill level values and in-situ material spill values. Predictive model generator 310 generates a predictive material spill model 861 as indicated by block 916. Predictive material spill model 861 predicts material spill based on one or more of machine orientation characteristics, speed characteristics, crop moisture, material mass, material center of mass, and fill level.
In some examples, the predictive material movement model 860 or the predictive material spill model 861, or both, is provided to control system 314 which generates control signals to control controllable subsystems 316 based on the predictive material movement model 860 or the predictive material spill model 861, or both. Thus, processing proceeds at block 918 where one or more characteristic values, such as one or more of speed values, crop moisture values, machine orientation values, fill level values, mass values, and center of mass values, are obtained. In some examples, one or more characteristic values can be detected by in-situ sensors 308 as indicated by block 920. The detected in-situ characteristic values can be input into the predictive material movement model 860 or the predictive material spill model 861, or both, to generate a predictive value of material movement or of material spill, or both, respectively. In some examples, one or more characteristic values can be provided by one or more maps as indicated by block 921. For example, some maps may include predictive values of machine speed characteristic, crop moisture, machine orientation, fill level, mass, or center of mass at different locations at the worksite. Thus, the values on the maps can be input into the predictive material movement model 860 or the predictive material spill model 861, or both, to generate a predictive value of material movement or of material spill, or both, respectively. The characteristic values can be obtained in various other ways, as indicated by block 922.
Thus, the control can be closed-loop in that the system, at block 918, receives characteristic values detected by the in-situ sensors 308 and generates control signals to optimize the operation of the mobile machine 100 based on the detected characteristic values and the output of the model. In other example, the control can be predictive in that the system, at block 918. receives characteristic values ahead of the machine as provided by a map, or another source, and predictively control operation of the mobile machine 100 based on the characteristic values ahead of the mobile machine and the output of the model.
In some examples, processing proceeds to block 923 where one or more threshold values are obtained. The threshold values can be provided by an operator 360 or user 364, can be stored in data store 302, or can be obtained in other ways. For example, one or more material dynamics threshold values, such as one or more of a material movement threshold value or a material spill threshold value can be obtained. The mobile machine 100 can be controlled based on the predictive material movement model 860 or the predictive material spill model 861, or both, to not exceed the material movement threshold value or the material spill threshold value, or both. For example, control system 314, at blocks 932 and 934, generates control signals and applies the control signals to control one or more controllable subsystems 316 based on the predictive material movement model 860 or the predictive material spill model 861, or both, and one or more of the characteristic values obtained at block 918 and the threshold values obtained at block 923. For example, control system 314 can generate control signals to control propulsion subsystem 350 to control a speed characteristic of mobile machine 100. In another example, control system 314 can generate control signals to control steering subsystem 352 to control a heading of mobile machine 100. In another example, control system 314 can generate control signals to control material transfer subsystem 341 to initiate or end a material transfer operation. In another examples, control 23 system 314 can generate control signal to control a material transfer subsystem of another machine 375 to control a material transfer operation (e.g., control the amount of material transferred).
Alternatively, or additionally, other threshold values can be obtained at block 923. For example, the operator or user of mobile machine 100 may provide a minimum and/or maximum travel speed, acceleration, or deceleration, at which mobile machine 100 can travel, and thus, where the speed characteristics of machine 100 can not be adjusted more, due to the speed characteristic threshold(s), other machine parameters can be controlled, such as the fill level of mobile machine 100, the travel path of mobile machine 100, as well as various other operating parameters, to satisfy the speed characteristic threshold(s) and to satisfy material movement or material spill, or both, levels (which may be indicated by a material movement threshold or a material spill threshold, or both). In some examples, processing proceeds to block 924 where input from geographic position sensor 304 and other in-situ sensors 308 are received by the control system 314. Particularly, at block 926, control system 314 detects an input from the geographic position sensor 304 identifying a geographic location of mobile machine 100. Block 928 represents receipt by the control system 314 of sensor inputs indicative of trajectory or heading of mobile machine 100, and block 930 represents receipt by the control system 314 of a speed of agricultural harvester 100. Block 931 represents receipt by the control system 314 of other information from various in-situ sensors 308. It may be that the mobile machine 100 has various latencies and that the various controllable subsystems 316 have certain responsiveness and thus the controllable subsystems 316 are controlled based on one or more of the geographic location, the travel speed, and the heading of the mobile machine 100 to control the controllable subsystems 316 to operate at the desired setting in a timely fashion, such as when mobile machine 100 reaches the area ahead of mobile machine 100 with the predictive characteristic value(s), as provided by the maps at block 921 (or other sources), that correspond to predicted material movement or the predicted material spill, or both. In some examples, such as where the control is closed-loop, obtaining a geographic position of the mobile machine 100 is not necessary or is not used in the generation of control signals.
Thus, in some examples, the mobile machine 100 can be controlled based on the predictive material movement model 860 or the predictive material movement model 861, or both, and one or more of the characteristic values obtained at block 918, the threshold values obtained at block 920, and the inputs at block 924, such as one or more of the geographic position, the heading, and the speed of the mobile machine 100. In such examples, control system 314, at blocks 932 and 934, generates control signals and applies the control signals to control one or more controllable subsystems 316.
Thus, it can be seen, that in some examples, the predictive material movement model 860 or the predictive material spill model 861, or both, can be used in closed-loop control to control operating parameters of mobile machine 100 (or other vehicles 375) based on characteristic values currently being experienced by the mobile machine 100, such as those detected by in-situ sensors 308 at block 920. In other examples, the predictive material movement model 860 or predictive material spill model 861, or both, can be used in predictive control to control operating parameters of mobile machine 100 (or other vehicles 375) based on predictive characteristic values obtained, such as from maps at block 920 or from other sources.
It will be appreciated that the particular control signals that are generated, and the particular controllable subsystems 316 that are controlled, may vary based upon one or more different things. For example, the control signals that are generated and the controllable subsystems 316 that are controlled may be based on the type of predictive material movement model 860 or the type of predictive material spill model 861, or both, that is being used. Similarly, the control signals that are generated and the controllable subsystems 316 that are controlled and the timing of the control signals can be based on various latencies of mobile machine 100 and the responsiveness of the controllable subsystems 316.
These are merely some examples. Control system 314 can generate a variety of different control signals to control a variety of different controllable subsystems 316 based on the predictive material movement model 860 or the predictive material spill model 861, or both, and one or more of the characteristic values obtained at block 918, the threshold values obtained at block 920, and the input(s) from the geographic position sensor or in-situ sensors 308, such as one or more of the geographic position, the heading, and the speed of mobile machine 100.
At block 936, a determination is made as to whether the operation has been completed. If the operation is not completed, the processing advances to block 938 where in-situ 19 sensor data from geographic position sensor 304 and in-situ sensors 308 (and perhaps other sensors) continue to be read.
In some examples, at block 940, agricultural system 300 can also detect learning trigger criteria to perform machine learning on one or more of the predictive material movement model 860 or the predictive material spill model 861, or both, one or more control algorithms implemented by the controllers in the control system 314, and other triggered learning.
The learning trigger criteria can include any of a wide variety of different criteria. Some examples of detecting trigger criteria are discussed with respect to blocks 942, 944, 946, 948, and 949. For instance, in some examples, triggered learning can involve recreation of a relationship used to generate a predictive model when a threshold amount of in-situ sensor data are obtained from in-situ sensors 308. In such examples, receipt of an amount of in-situ sensor data from the in-situ sensors 308 that exceeds a threshold triggers or causes the predictive model generator 310 to generate a new predictive model. Thus, as mobile machine 100 continues an operation, receipt of the threshold amount of in-situ sensor data from the in-situ sensors 308 triggers the creation of a new relationship represented by a new predictive material movement model 860 generated by predictive model generator 310 or triggers the creation of a new relationship represented by a new predictive material spill model 861 generated by predictive model generator 310, or both. Block 942 represents detecting a threshold amount of in-situ sensor data used to trigger creation of a new predictive model.
In other examples, the learning trigger criteria may be based on how much the in-situ sensor data from the in-situ sensors 308 are changing, such as over time or compared to previous values. For example, if variations within the in-situ sensor data (or the relationship between the in-situ sensor data) are within a selected range or is less than a defined amount, or below a threshold value, then a new predictive model is not generated by the predictive model generator 310. However, if variations within the in-situ sensor data are outside of the selected range, are greater than the defined amount, or are above the threshold value, for example, then the predictive model generator 310 generates a new predictive model 860 or 861, or both, using all or a portion of the newly received in-situ sensor data. At block 944, variations in the in-situ sensor data, such as a magnitude of an amount by which the data exceeds the selected range or a magnitude of the variation of the relationship between the in-situ sensor data, can be used as a trigger to cause generation of a new predictive model 860 or 861, or both. Keeping with the examples described above, the threshold, the range, and the defined amount can be set to default values; set by an operator or user interaction through a user interface; set by an automated system; or set in other ways.
Other learning trigger criteria can also be used. For instance, if predictive model generator 310 switches to a different set of sensor data (different from the originally selected set of sensor data), then switching to the different set may trigger re-learning by predictive model generator 310, control system 314, or other items. In another example, transitioning of mobile machine 100 to a different topography or to a different area of the worksite may be used as learning trigger criteria as well.
In some instances, operator 360 can also edit the predictive material movement model 860 or the predictive material spill model 861, or both, edit values in the sensor data, as well as input new values, such as based on operator observation. Block 946 shows that edited information can be used as learning trigger criteria.
In some instances, it may also be that operator 360 or user 366 observes that automated control of a controllable subsystem 316, is not what the operator or user desires. In such instances, the operator 360 or user 366 may provide a manual adjustment to the controllable subsystem 316 reflecting that the operator 360 or user 366 desires the controllable subsystem 316 to operate in a different way than is being commanded by control system 314. Thus, manual alteration of a setting by the operator 360 or user 366 can cause one or more of predictive model generator 310 to relearn a model and control system 314 to relearn a control algorithm or to perform machine learning on one or more of the controller components 329 through 337 in control system 314 based upon the adjustment by the operator 360 or user 366, as shown in block 948. Block 949 represents the use of other triggered learning criteria.
In other examples, relearning may be performed periodically or intermittently based, for example, upon a selected time interval such as a discrete time interval or a variable time interval, as indicated by block 950.
If relearning is triggered, whether based upon learning trigger criteria or based upon passage of a time interval, as indicated by block 950, then one or more of the predictive model generator 310 and control system 314 performs machine learning to generate a new predictive model and a new control algorithm, respectively, based upon the learning trigger criteria. The new predictive model and the new control algorithm are generated using any additional data that has been collected since the last learning operation was performed. Performing relearning is indicated by block 952.
If the operation has not been completed, operation moves from block 952 to block 918 such that operation of the mobile machine 100 can be controlled based on the new predictive model or the new control algorithm, or both.
If the operation has been completed, operation moves from block 952 to block 954 where the predictive material movement model 860, the predictive material spill model 871, and the control algorithm(s), are stored. The predictive model 850 may be stored locally on data store 302 or sent to a remote system using communication system 306 for later use.
At block 1000, in-situ sensors 308 generate sensor signals indicative of one or more in-situ data values indicative of characteristics as the mobile machine 100 is operating at a worksite. Material dynamics sensors 179, such as material spill and movement sensors 180, generate sensor signals indicative of one or more in-situ data values indicative of material dynamics characteristics, such as material movement characteristics or material spill characteristics, as indicated by block 1001. Heading/speed sensors 325 generate sensor signals indicative of one or more in-situ data values indicative of speed characteristic(s), such as one or more of travel speed, acceleration, and deceleration, as indicated by block 1002. As described herein, in some examples, geographic position data generated by geographic position sensor 304 may also be used to generate one or more in-situ data values indicative of speed characteristic(s), such as one or more of travel speed, acceleration, and deceleration. Crop moisture sensors 326 generate sensor signals indicative of one or more in-situ data values indicative of moisture of crop material collected by mobile machine 100, as indicated by block 1003. Machine orientation sensors 327 generate sensor signals indicative of machine orientation characteristics, such as one or more of roll, pitch, and yaw of the machine, as indicated by block 1004. Fill level sensors 324 generate sensor signals indicative of a fill level of material receptacle 307, as indicated by block 1005. As described herein, in some examples, material spill and movement sensors 180 may also generate sensor signals indicative of one or more in-situ data values indicative of fill level of material receptacle 307. Material spill and movement sensors 180 can generate sensor signals indicative of one or more in-situ data values indicative of a mass of material in material receptacle 307, as indicated by block 1006. Material spill and movement sensors 180 can generate sensor signals indicative of one or more in-situ data values indicative of a center of mass of material in material receptacle 307, as indicated by block 1007. Various other in-situ sensors can generate sensor signals indicative of various other in-situ data values indicative of various other characteristics, as indicated by block 1008. In some examples, data from in-situ sensors 308 is georeferenced using position, heading, or speed data from geographic position sensor 304.
In one example, at block 1010, predictive model generator 310 controls one or more of the material movement-to-machine orientation model generator 841, the material movement-to-speed model generator 842, the material movement-to-crop moisture model generator 843, the material movement-to-material mass model generator 844, the material movement-to-material center of mass model generator 845, and the material movement-to-fill level model generator 846 to generate a model that models the relationship between one or more of the in-situ machine orientation values, the in-situ speed values, the in-situ crop moisture values, the in-situ mass values, and the in-situ center of mass values, and the in-situ fill level values and in-situ material movement values. Predictive model generator 310 generates a predictive material movement model 860 as indicated by block 1012. Predictive material movement model 860 predicts material movement based on one or more of machine orientation characteristics, speed characteristics, crop moisture, material mass, material center of mass, and fill level.
Alternatively, or additionally, at block 1010, predictive model generator 310 controls one or more of the material spill-to-machine orientation model generator 847, the material spill-to-speed model generator 848, the material spill-to-crop moisture model generator 849, the material spill-to-material mass model generator 850, the material spill-to-material center of mass model generator 851, and the material spill-to-material fill level model generator 852 to generate a model that models the relationship between one or more of the in-situ machine orientation values, the in-situ speed values, the in-situ crop moisture values, the in-situ material mass values, the in-situ material center of mass values, and the in-situ material fill level values and in-situ material spill values. Predictive model generator 310 generates a predictive material spill model 861 as indicated by block 1013. Predictive material spill model 861 predicts material spill based on one or more of machine orientation characteristics, speed characteristics, crop moisture, material mass, material center of mass, and fill level.
At block 1014, agricultural system 300 receives one or more information maps 358. Examples of information maps 358 or receiving information maps 358 are discussed with respect to blocks 1015, 1016, 1017, and 1018. As discussed above, information maps 358 map values of a variable, corresponding to a characteristic, to different locations in the field, as indicated at block 1016. As indicated at block 1015, receiving the information maps 358 may involve selecting one or more of a plurality of possible information maps 358 that are available. For instance, one information map 358 may be a terrain map, such as terrain map 831. Another information map 358 may be a speed map, such as speed map 832. Another information map 358 may be a crop moisture map, such as crop moisture map 833. Another information map may be a fill level map, such as fill level map 835. Various other maps, such as other maps 837, are contemplated herein. The process by which one or more information maps 358 are selected can be manual, semi-automated, or automated. The information maps 358 can be based on data collected prior to a current operation, as indicated by block 1017. For instance, the data may be collected based on aerial images taken during a previous year, or earlier in the current season, or at other times. The data may be based on data detected in ways other than using aerial images. For instance, the data may be collected during a previous operation on the worksite, such an operation during a previous year, or a previous operation earlier in the current season, or at other times. The machines performing those previous operations may be outfitted with one or more sensors that generate sensor data indicative of one or more characteristics. For example, the sensed speed and orientation of mobile machine 100 operating at the worksite in a past year may be used as data to generate the information maps 358. In other examples, and as described above, the information maps 358 may be predictive maps having predictive values, such as a predictive speed map having predictive speed values, a predictive terrain map having predictive terrain values, a predictive crop moisture 14 map having predictive crop moisture values, and a predictive fill level map having predictive fill level values. The predictive information map 358 can be generated by predictive map generator 312 based on a model generated by predictive model generator 310. The data for the information maps 358 or the information maps 358, or both, can be obtained by agricultural system 300 using communication system 306 and stored in data store 302. The data for the information maps 358 can be obtained by agricultural system 300 using communication system 306 in other ways as well, and this is indicated by block 1018 in the flow diagram of
At block 1020, the relationship or model generated by predictive model generator 310 is provided to predictive map generator 312. Predictive map generator 312 controls predictive material dynamics map generator 451, such as predictive material movement map generator 855 or predictive material spill map generator 856, to generate a functional predictive material dynamics map 458, such as a functional predictive material movement map 870 as indicated by block 1021 or functional predictive material spill map 868 as indicated by block 1022. Functional predictive material movement map 870 predicts material movement (or sensor value(s) indicative of material movement) at different geographic locations in a worksite at which mobile machine 100 is operating using the predictive material movement model 860 and one or more of the information maps, such as terrain map 831, speed map 832, crop moisture map 833, and fill level map 835. Functional predictive material spill map 868 predicts material spill (or sensor value(s) indicative of material spill) at different geographic locations in a worksite at which mobile machine 100 is operating using the predictive material spill model 861 and one or more of the information maps, such as terrain map 831, speed map 832, crop moisture map 833, and fill level map 835.
It should be noted that, in some examples, the functional predictive material movement map 870 may include two or more different map layers. Each map layer may represent a different data type, for instance, a functional predictive material movement map 870 that provides two or more of a map layer that provides predictive material movement based on terrain values from terrain map 831, a map layer that provides predictive material movement based on speed values from speed map 832, a map layer that provides predictive material movement based on crop moisture values from crop moisture map 833, and a map layer that provides predictive material movement based on fill level values from fill level map 835. In other examples, functional predictive material movement map 870 can include a map layer that provides predictive material movement based on two or more of terrain values from terrain map 831, speed values from speed map 832, crop moisture values from crop moisture map 638, and fill level values from fill level map 835.
It should be noted that, in some examples, the functional predictive material spill map 868 may include two or more different map layers. Each map layer may represent a different data type, for instance, a functional predictive material spill map 868 that provides two or more of a map layer that provides predictive material spill based on terrain values from terrain map 831, a map layer that provides predictive material spill based on speed values from speed map 832, a map layer that provides predictive material spill based on crop moisture values from crop moisture map 833, and a map layer that provides predictive material spill based on fill level values from fill level map 835. In other examples, functional predictive material spill map 868 can include a map layer that provides predictive material spill based on two or more of terrain values from terrain map 831, speed values from speed map 832, crop moisture values from crop moisture map 638, and fill level values from fill level map 835.
At block 1024, predictive map generator 312 configures the functional predictive material movement map 870 or the functional predictive material spill map 868, or both, so that the functional predictive material movement map 870 or the functional predictive material spill map 868, is actionable (or consumable) by control system 314. Predictive map generator 312 can provide the functional predictive material movement map 870 or the functional predictive material spill map 868, or both, to the control system 314 or to control zone generator 313, or both. Some examples of the different ways in which the functional predictive material movement map 870 or the functional predictive material spill map 868, or both, can be configured or output are described with respect to blocks 1024, 1025, 1026, and 1027. For instance, at block 1024 predictive map generator 312 configures functional predictive material movement map 870 so that functional predictive material movement map 870 includes values that can be read by control system 314 and used as the basis for generating control signals for one or more of the different controllable subsystems 316 of mobile machine 100. Alternatively, or additionally, at block 1024, predictive map generator 312 configures functional predictive material spill map 868 so that functional predictive material spill map 868 includes values that can be read by control system 314 and used as the basis for generating control signals for one or more of the different controllable subsystems 316 of mobile machine 100.
At block 1025, control zone generator can divide the functional predictive material movement map 870 into control zones based on the values on the functional predictive material movement map 870 to generate functional predictive material movement control zone map 873, as an example of a functional predictive material dynamics control zone map 459. Alternatively, or additionally, at block 1025, control zone generator can divide the functional predictive material spill map 868 into control zones based on the values on the functional predictive material spill map 868 to generate functional predictive material spill control zone map 871, as an example of a functional predictive material dynamics control zone map 459. Contiguously-geolocated values that are within a threshold value of one another can be grouped into a control zone. The threshold value can be a default threshold value, or the threshold value can be set based on an operator input, based on an input from an automated system, or based on other criteria. A size of the zones may be based on a responsiveness of the control system 314, the controllable subsystems 316, based on wear considerations, or on other criteria.
At block 1026, predictive map generator 312 configures functional predictive material movement map 870 or functional predictive material spill map 868, or both, for presentation to an operator or other user. Alternatively, or additionally, at block 722, control zone generator 313 can configure functional predictive material movement control zone map 873 or functional predictive material spill control zone map 871, or both, for presentation to an operator or other user. When presented to an operator or other user, the presentation of the functional predictive map(s) 870, 868 or of the functional predictive control zone map(s) 871, 873, or both, may contain one or more of the predictive values on the functional predictive map(s) 870, 868 correlated to geographic location, the control zones of functional predictive control zone map(s) 871, 873 correlated to geographic location, and settings values or control parameters that are used based on the predicted values on predictive map(s) 870, 868 or control zones on predictive control zone map(s) 871, 873. The presentation can, in another example, include more abstracted information or more detailed information. The presentation can also include a confidence level 8 that indicates an accuracy with which the predictive values on predictive map(s) 870, 868 or the control zones on predictive control zone map(s) 871, 873 conform to measured values that may be measured by sensors on mobile machine 100 as mobile machine 100 operates at the worksite. Further where information is presented to more than one location, an authentication and authorization system can be provided to implement authentication and authorization processes. For instance, there may be a hierarchy of individuals that are authorized to view and change maps and other presented information. By way of example, an on-board display device may show the maps in near real time locally on the machine, or the maps may also be generated at one or more remote locations, or both. In some examples, each physical display device at each location may be associated with a person or a user permission level. The user permission level may be used to determine which display elements are visible on the physical display device and which values the corresponding person may change. As an example, a local operator of mobile machine 100 may be unable to see the information corresponding to the predictive map(s) 870, 868 or make any changes to machine operation. A supervisor, such as a supervisor at a remote location, however, may be able to see the predictive map(s) 870, 868 on the display but be prevented from making any changes. A manager, who may be at a separate remote location, may be able to see all of the elements on predictive map(s) 870, 868 and also be able to change the predictive map(s) 870, 868. In some instances, the predictive map(s) 870, 868 are accessible and changeable by a manager located remotely may be used in machine control. This is one example of an authorization hierarchy that may be implemented.
The predictive map 870 or predictive control zone map 873 or both can be configured in other ways as well, as indicated by block 1027. The predictive map 868 or predictive control zone map 871 or both can be configured in other ways as well, as indicated by block 1027.
At block 1028, input from geographic position sensor 304 and other in-situ sensors 308 are received by the control system 314. Particularly, at block 1029, control system 314 detects an input from the geographic position sensors 304 identifying a geographic location of mobile machine 100. Block 1030 represents receipt by the control system 314 of sensor inputs indicative of a trajectory or heading of mobile machine 100, and block 1031 represent receipt by the control system 314 of a speed of agricultural harvester 100. Block 1032 represents receipt by the control system 314 of other information from various in-situ sensors 308.
At block 1033, control system 314 generates control signals to control the controllable subsystems based on the functional predictive material movement map 870 or the functional predictive material movement control zone map 873, or both, and the input from the geographic position sensor 304 and any other in-situ sensors 308, such as heading and or speed inputs from heading/speed sensors 325. Alternatively, or additionally, at block 1033, control system 314 generates control signals to control the controllable subsystems based on the functional predictive material spill map 868 or the functional predictive material spill control zone map 871, or both, and the input from the geographic position sensor 304 and any other in-situ sensors 308, such as heading and or speed inputs from heading/speed sensors 325.
By way of example, propulsion controller 331 of control system 314 can generate control signals to control propulsion subsystem 350 to control one or more propulsion parameters of mobile machine 100, such as one or more of the speed at which the mobile machine travels, the deceleration of mobile machine 100, and the acceleration of mobile machine 100. For instance, functional predictive material movement map 870 or functional predictive material movement control zone map 873 may predict material movement in area(s) of the worksite ahead of or around mobile machine 100, in which case, propulsion controller 331 can generate control signals to control propulsion system 350 to control a propulsion parameter, such as travel speed, acceleration, deceleration, etc., at those area(s). In another example, functional predictive material spill map 868 or functional predictive material spill control zone map 871 may predict material spill in area(s) of the worksite ahead of or around mobile machine 100, in which case, propulsion controller 331 can generate control signals to control propulsion system 350 to control a propulsion parameter, such as travel speed, acceleration, deceleration, etc., at those area(s).
In another example, path planning controller 334 of control system 314 can generate control signals to control steering subsystem 352 to control a route parameter of mobile machine 100, such as one or more of a commanded path at the worksite over which mobile machine 100 travels, and the steering of mobile machine 100. For instance, functional predictive material movement map 870 or functional predictive material movement control zone map 873, or both, may predict material movement in area(s) of the worksite ahead of or around mobile machine 100, in which case, path planning controller 334 can generate control signals to control steering subsystem 352 to cause mobile machine 100 to avoid traveling those area(s) or to travel through them in a different manner, such as with limited steering angles. In another example, functional predictive material spill map 868 or functional predictive material spill control zone map 871, or both, may predict material spill in area(s) of the worksite ahead of or around mobile machine 100, in which case, path planning controller 334 can generate control signals to control steering subsystem 352 to cause mobile machine 100 to avoid traveling those area(s) or to travel through them in a different manner, such as with limited steering angles.
In another example, material transfer controller 335 of control system 314 can generate control signals to control material transfer subsystem 341 to initiate or end a material transfer operation. These are merely some examples. Control system 314 can generate a variety of different control signals to control a variety of different controllable subsystems 316 based on functional predictive material movement map 870 or functional predictive material movement control zone map 873, or both. Alternatively, or additionally, control system 314 can generate a variety of different control signals to control a variety of different controllable subsystem 316 based on functional predictive material spill map 868 or functional predictive material spill control zone map 871, or both.
At block 1036, a determination is made as to whether the operation has been completed. If the operation is not completed, the processing advances to block 1038 where in-situ sensor data from geographic position sensor 304 and in-situ sensors 308 (and perhaps other sensors) continue to be read.
In some examples, at block 1040, agricultural system 300 can also detect learning trigger criteria to perform machine learning on one or more of the functional predictive material movement map 870, functional predictive material movement control zone map 873, predictive material movement model 860, functional predictive material spill map 868, functional predictive material spill control zone map 871, predictive material spill model 861, the zones generated by control zone generator 313, one or more control algorithms implemented by the controllers in the control system 314, and other triggered learning.
The learning trigger criteria can include any of a wide variety of different criteria. Some examples of detecting trigger criteria are discussed with respect to blocks 1042, 1044, 1046, 1048, and 1049. For instance, in some examples, triggered learning can involve recreation of a relationship used to generate a predictive model when a threshold amount of in-situ sensor data are obtained from in-situ sensors 308. In such examples, receipt of an amount of in-situ sensor data from the in-situ sensors 308 that exceeds a threshold triggers or causes the predictive model generator 310 to generate a new predictive model that is used by predictive map generator 312. Thus, as mobile machine 100 continues an operation, receipt of the threshold amount of in-situ sensor data from the in-situ sensors 308 triggers the creation of a new relationship represented by a new predictive material movement model 860 generated by predictive model generator 310 or the creation of a new relationship represented by a new predictive material spill model 861 generated by predictive model generator 310, or both. Further, a new functional predictive material movement map 660, a new functional predictive material movement control zone map 661, or both can be generated using the new predictive material movement model 860. Additionally, or alternatively, a new functional predictive material spill map 868, a new functional predictive material spill control zone map 871, or both can be generated using the new predictive material spill model 861. Block 1042 represents detecting a threshold amount of in-situ sensor data used to trigger creation of a new predictive model.
In other examples, the learning trigger criteria may be based on how much the in-situ sensor data from the in-situ sensors 308 are changing, such as over time or compared to previous values. For example, if variations within the in-situ sensor data (or the relationship between the in-situ sensor data and the information in the one or more information maps 358) are within a selected range or is less than a defined amount, or below a threshold value, then a new predictive model is not generated by the predictive model generator 310. As a result, the predictive map generator 312 does not generate a new functional predictive map (e.g., 870 and/or 868), a new functional predictive control zone map (e.g., 871 and/or 873), or both. However, if variations within the in-situ sensor data are outside of the selected range, are greater than the defined amount, or are above the threshold value, for example, then the predictive model generator 310 generates a new predictive model (e.g., 860 and/or 861) using all or a portion of the newly received in-situ sensor data that the predictive map generator 312 uses to generate a new predictive map (e.g., 870 and/or 868) which can be provided to control zone generator 313 for the creation of a new predictive control zone map (e.g., 871 and/or 873). At block 1044, variations in the in-situ sensor data, such as a magnitude of an amount by which the data exceeds the selected range or a magnitude of the variation of the relationship between the in-situ sensor data and the information in the one or more information maps, can be used as a trigger to cause generation of one or more of a new predictive model (e.g., 860 and/or 861), a new predictive map (e.g., 870 and/or 868), and a new predictive control zone map (e.g., 871 and/or 873). Keeping with the examples described above, the threshold, the range, and the defined amount can be set to default values; set by an operator or user interaction through a user interface; set by an automated system; or set in other ways.
Other learning trigger criteria can also be used. For instance, if predictive model generator 310 switches to a different information map (different from the originally selected information map), then switching to the different information map may trigger re-learning by predictive model generator 310, predictive map generator 312, control zone generator 313, control system 314, or other items. In another example, transitioning of mobile machine 100 to a different topography or to a different control zone may be used as learning trigger criteria as well.
In some instances, operator 360 or user 366 can also edit the functional predictive material map (e.g., 870 and/or 868) or functional predictive control zone map (e.g., 871 and/or 873), or both. The edits can change a value on the functional predictive map, change a size, shape, position, or existence of a control zone on functional predictive control zone map, or both. Block 1046 shows that edited information can be used as learning trigger criteria.
In some instances, it may also be that operator 360 or user 366 observes that automated control of a controllable subsystem 316, is not what the operator or user desires. In such instances, the operator 360 or user 366 may provide a manual adjustment to the controllable subsystem 316 reflecting that the operator 360 or user 366 desires the controllable subsystem 316 to operate in a different way than is being commanded by control system 314. Thus, manual alteration of a setting by the operator 360 or user 366 can cause one or more of predictive model generator 310 to relearn a model, predictive map generator 312 to regenerate a functional predictive map, control zone generator 313 to regenerate one or more control zones on a functional predictive control zone map, and control system 314 to relearn a control algorithm or to perform machine learning on one or more of the controller components 329 through 337 in control system 314 based upon the adjustment by the operator 360 or user 366, as shown in block 1048. Block 1049 represents the use of other triggered learning criteria.
In other examples, relearning may be performed periodically or intermittently based, for example, upon a selected time interval such as a discrete time interval or a variable time interval, as indicated by block 1050.
If relearning is triggered, whether based upon learning trigger criteria or based upon passage of a time interval, as indicated by block 1050, then one or more of the predictive model generator 310, predictive map generator 312, control zone generator 313, and control system 314 performs machine learning to generate a new predictive model, a new predictive map, a new control zone, and a new control algorithm, respectively, based upon the learning trigger criteria. The new predictive model, the new predictive map, the new control zone, and the new control algorithm are generated using any additional data that has been collected since the last learning operation was performed. Performing relearning is indicated by block 1052.
If the operation has not been completed, operation moves from block 1052 to block 1024 such that operation of the mobile machine 100 can be controlled based on the new predictive map, a new control zone, or a new control algorithm.
If the operation has been completed, operation moves from block 1052 to block 1054 where one or more of the functional predictive material movement map 870, functional predictive material movement control zone map 873, predictive material movement model 860, functional predictive material spill map 868, functional predictive material spill control zone map 871, predictive material spill model 861, control zone(s), and control algorithm(s), are store. The functional predictive map(s), functional predictive control zone map(s), predictive model(s), control zone(s), and control algorithm(s) may be stored locally on data store 302 or sent to a remote system using communication system 306 for later use.
The examples herein describe the generation of a predictive model and, in some examples, the generation of a functional predictive map based on the predictive model. The examples described herein are distinguished form other approaches by the use of a model which is at least one of multi-variate or site-specific (i.e., georeferenced, such as map-based). Furthermore, the model is revised as the work machine is performing an operation and while additional in-situ sensor data is collected. The model may also be applied in the future beyond the current worksite. For example, the model may form a baseline (e.g., starting point) for a subsequent operation at a different worksite or at the same worksite at a future time.
The revision of the model in response to new data may employ machine learning methods. Without limitation, machine learning methods may include memory networks, Bayes systems, decisions trees, Cluster Analysis, Eigenvectors, Eigenvalues and Machine Learning, Evolutionary and Genetic Algorithms, Expert Systems/Rules, Support Vector Machines, Engines/Symbolic Reasoning, Generative Adversarial Networks (GANs), Graph Analytics and ML, Linear Regression, Logistic Regression, LSTMs and Recurrent Neural Networks (RNNSs), Convolutional Neural Networks (CNNs), MCMC, Random Forests, Reinforcement Learning or Reward-based machine learning. Learning may be supervised or unsupervised.
Model implementations may be mathematical, making use of mathematical equations, empirical correlations, statistics, tables, matrices, and the like. Other model implementations may rely more on symbols, knowledge bases, and logic such as rule-based systems. Some implementations are hybrid, utilizing both mathematics and logic. Some models may incorporate random, non-deterministic, or unpredictable elements. Some model implementations may make uses of networks of data values such as neural networks. These are just some examples of models.
The predictive paradigm examples described herein differ from non-predictive approaches where an actuator or other machine parameter is fixed at the time the machine, system, or component is designed, set once before the machine enters the worksite, is reactively adjusted manually based on operator perception, or is reactively adjusted based on a sensor value.
The functional predictive map examples described herein also differ from other map-based approaches. In some examples of these other approaches, an a priori control map is used without any modification based on in-situ sensor data or else a difference determined between data from an in-situ sensor and a predictive map are used to calibrate the in-situ sensor. In some examples of the other approaches, sensor data may be mathematically combined with a priori data to generate control signals, but in a location-agnostic way; that is, an adjustment to an a priori, georeferenced predictive setting is applied independent of the location of the work machine at the worksite. The continued use or end of use of the adjustment, in the other approaches, is not dependent on the work machine being in a particular defined location or region within the worksite.
In examples described herein, the functional predictive maps and predictive actuator control rely on obtained maps and in-situ data that are used to generate predictive models. The predictive models are then revised during the operation to generate revised functional predictive maps and revised actuator control. In some examples, the actuator control is provided based on functional predictive control zone maps which are also revised during the operation at the worksite. In some examples, the revisions (e.g., adjustments, calibrations, etc.) are tied to region or zones of the worksite rather than to the whole worksite or some non-georeferenced condition. For example, the adjustments are applied to one or more areas of a worksite to which an adjustment is determined to be relevant (e.g., such as by satisfying one or more conditions which may result in application of an adjustment to one or more locations while not applying the adjustment to one or more other locations), as opposed to applying a change in a blanket way to every location in a non-selective way.
In some examples described herein, the models determine and apply those adjustments to selective portions or zones of the worksite based on a set of a priori data, which, in some instances, is multivariate in nature. For example, adjustments may, without limitation, be tied to defined portions of the worksite based on site-specific factors such as topography, soil type, crop variety, soil moisture, as well as various other factors, alone or in combination. Consequently, the adjustments are applied to the portions of the field in which the site-specific factors satisfy one or more criteria and not to other portions of the field where those site-specific factors do not satisfy the one or more criteria. Thus, in some examples described herein, the model generates a revised functional predictive map for at least the current location or zone, the unworked part of the worksite, or the whole worksite.
As an example, in which the adjustment is applied only to certain areas of the field, consider the following. The system may determine that a detected in-situ characteristic value (e.g., detected material spill or material movement) varies from a predictive value of the characteristic (e.g., predictive material spill or material movement), such as by a threshold amount. This deviation may only be detected in areas of the field where the slope of the worksite is above a certain level. Thus, the revision to the predictive value is only applied to other areas of the worksite having slope above the certain level. In this simpler example, the predictive characteristic value and slope at the point the deviation occurred and the detected characteristic value and slope at the point the deviation occurred are used to generate a linear equation. The linear equation is used to adjust the predictive characteristic value in unharvested areas of the worksite in the functional predictive map as a function of slope and the predicted characteristic value. This results in a revised functional predictive map in which some values are adjusted while others remain unchanged based on selected criteria, e.g., slope as well as threshold deviation. The revised functional map is then used to generate a revised functional control zone map for controlling the mobile machine.
As an example, without limitation, consider an instance of the paradigm described herein which is parameterized as follows.
One or more maps of the field are obtained, such as one or more of a terrain map, a speed map, a crop moisture map, a fill level map, and another type of map;
In-situ material dynamics sensors the generate sensor data indicative of material dynamics values, such as one or more of material spill characteristic values and material movement 11 characteristic values.
A predictive model generator generates one or more predictive models based on the one or more obtained maps and the in-situ sensor data, such as one or more of a predictive material spill model and a predictive material movement model.
A predictive map generator generates one or more functional predictive maps based on a model generated by the predictive model generator and the one or more obtained maps. For example the predictive map generator may generate a functional predictive material spill map that maps predictive material spill characteristic values to one or more locations on the worksite based on a predictive material spill model and the one or more obtained maps. In another example, the predictive map generator may generate a functional predictive material movement map that maps predictive material movement characteristic values to one or more locations on the worksite based on a predictive material movement model and the one or more obtained maps.
Control zones, which include machine settings values, can be incorporated into the functional predictive material spill map or the functional predictive material movement map, or both, to generate a functional predictive material spill control zone map or a functional predictive material movement control zone map.
As the mobile machine continues to operate at the worksite, additional in-situ sensor data is collected. A learning trigger criteria can be detected, such as threshold amount of additional in-situ sensor data being collected, a magnitude of change in a relationship (e.g., the in-situ characteristic values varies to a certain [e.g., threshold] degree from a predictive value of the characteristic), the operator makes edits to the predictive map(s) or to a control algorithm, or both, a certain (e.g., threshold) amount of time elapses, as well as various other learning trigger criteria. The predictive model(s) are then revised based on the additional in-situ sensor data and the values from the obtained maps. The functional predictive maps or the functional predictive control zone maps, or both, are then revised based on the revised model(s) and the values in the obtained maps.
As another example, without limitation, consider an instance of the paradigm described herein which is parameterized as follows.
In-situ material dynamics sensors generate sensor data indicative of material dynamics values, such as one or more of material spill characteristic values and material movement characteristic values.
Other in-situ sensors generate sensor data indicative of other characteristic values, such as one or more of fill level values, speed values, crop moisture values, and machine orientation values.
A predictive model generator generates one or more predictive models based on the in-situ material dynamics sensor data and the other in-situ sensor data, such as one or more of a predictive material spill model and a predictive material movement model.
One or more maps of the field are obtained, such as one or more of a terrain map, a speed map, a crop moisture map, a fill level map, and another type of map.
A predictive map generator generates one or more functional predictive maps based on a model generated by the predictive model generator and the one or more obtained maps. For example, the predictive map generator may generate a functional predictive material spill map that maps predictive material spill characteristic values to one or more locations on the worksite based on a predictive material spill model and the one or more obtained maps. In another example, the predictive map generator may generate a functional predictive material movement map that maps predictive material movement characteristic values to one or more locations on the worksite based on a predictive material movement model and the one or more obtained maps.
Control zones, which include machine settings values, can be incorporated into the functional predictive material spill map or the functional predictive material movement map, or both, to generate a functional predictive material spill control zone map or a functional predictive material movement control zone map.
As the mobile machine continues to operate at the worksite, additional in-situ sensor data is collected. A learning trigger criteria can be detected, such as threshold amount of additional in-situ sensor data being collected, a magnitude of change in a relationship (e.g., the in-situ characteristic values varies to a certain [e.g., threshold] degree from a predictive value of the characteristic), the operator makes edits to the predictive map(s) or to a control algorithm, or both, a certain (e.g., threshold) amount of time elapses, as well as various other learning trigger criteria. The predictive model(s) are then revised based on the additional in-situ sensor data. The functional predictive maps or the functional predictive control zone maps, or both, are then revised based on the revised model(s) and the values in the obtained maps.
The present discussion has mentioned processors and servers. In some examples, the processors and servers include computer processors with associated memory and timing circuitry, not separately shown. They are functional parts of the systems or devices to which they belong and are activated by and facilitate the functionality of the other components or items in those systems.
Also, a number of user interface displays have been discussed. The displays can take a wide variety of different forms and can have a wide variety of different user actuatable operator interface mechanisms disposed thereon. For instance, user actuatable operator interface mechanisms may include text boxes, check boxes, icons, links, drop-down menus, search boxes, etc. The user actuatable operator interface mechanisms can also be actuated in a wide variety of different ways. For instance, they can be actuated using operator interface mechanisms such as a point and click device, such as a track ball or mouse, hardware buttons, switches, a joystick or keyboard, thumb switches or thumb pads, etc., a virtual keyboard or other virtual actuators. In addition, where the screen on which the user actuatable operator interface mechanisms are displayed is a touch sensitive screen, the user actuatable operator interface mechanisms can be actuated using touch gestures. Also, user actuatable operator interface mechanisms can be actuated using speech commands using speech recognition functionality. Speech recognition may be implemented using a speech detection device, such as a microphone, and software that functions to recognize detected speech and execute commands based on the received speech.
A number of data stores have also been discussed. It will be noted the data stores can each be broken into multiple data stores. In some examples, one or more of the data stores may be local to the systems accessing the data stores, one or more of the data stores may all be located remote form a system utilizing the data store, or one or more data stores may be local while others are remote. All of these configurations are contemplated by the present disclosure.
Also, the figures show a number of blocks with functionality ascribed to each block. It will be noted that fewer blocks can be used to illustrate that the functionality ascribed to multiple different blocks is performed by fewer components. Also, more blocks can be used illustrating that the functionality may be distributed among more components. In different examples, some functionality may be added, and some may be removed.
It will be noted that the above discussion has described a variety of different systems, components, logic and interactions. It will be appreciated that any or all of such systems, components, logic and interactions may be implemented by hardware items, such as processors, memory, or other processing components, some of which are described below, that perform the functions associated with those systems, components, or logic, or interactions. In addition, any or all of the systems, components, logic and interactions may be implemented by software that is loaded into a memory and is subsequently executed by a processor or server or other computing component, as described below. Any or all of the systems, components, logic and interactions may also be implemented by different combinations of hardware, software, firmware, etc., some examples of which are described below. These are some examples of different structures that may be used to implement any or all of the systems, components, logic and interactions described above. Other structures may be used as well.
In the example shown in
It will also be noted that the elements of
In some examples, remote server architecture 1002 may include cybersecurity measures. Without limitation, these measures may include encryption of data on storage devices, encryption of data sent between network nodes, authentication of people or processes accessing data, as well as the use of ledgers for recording metadata, data, data transfers, data accesses, and data transformations. In some examples, the ledgers may be distributed and immutable (e.g., implemented as blockchain).
In other examples, applications can be received on a removable Secure Digital (SD) card that is connected to an interface 15. Interface 15 and communication links 13 communicate with a processor 17 (which can also embody processors or servers from other FIGS.) along a bus 19 that is also connected to memory 21 and input/output (I/O) components 23, as well as clock 25 and location system 27.
I/O components 23, in one example, are provided to facilitate input and output operations. I/O components 23 for various examples of the device 16 can include input components such as buttons, touch sensors, optical sensors, microphones, touch screens, proximity sensors, accelerometers, orientation sensors and output components such as a display device, a speaker, and or a printer port. Other I/O components 23 can be used as well.
Clock 25 illustratively comprises a real time clock component that outputs a time and date. It can also, illustratively, provide timing functions for processor 17.
Location system 27 illustratively includes a component that outputs a current geographical location of device 16. This can include, for instance, a global positioning system (GPS) receiver, a LORAN system, a dead reckoning system, a cellular triangulation system, or other positioning system. Location system 27 can also include, for example, mapping software or navigation software that generates desired maps, navigation routes and other geographic functions.
Memory 21 stores operating system 29, network settings 31, applications 33, application configuration settings 35, data store 37, communication drivers 39, and communication configuration settings 41. Memory 21 can include all types of tangible volatile and non-volatile computer-readable memory devices. Memory 21 may also include computer storage media (described below). Memory 21 stores computer readable instructions that, when executed by processor 17, cause the processor to perform computer-implemented steps or functions according to the instructions. Processor 17 may be activated by other components to facilitate their functionality as well.
Note that other forms of the devices 16 are possible.
Computer 1210 typically includes a variety of computer readable media. Computer readable media may be any available media that can be accessed by computer 1210 and includes both volatile and nonvolatile media, removable and non-removable media. By way of example, and not limitation, computer readable media may comprise computer storage media and communication media. Computer storage media is different from, and does not include, a modulated data signal or carrier wave. Computer readable media includes hardware storage media including both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by computer 1210. Communication media may embody computer readable instructions, data structures, program modules or other data in a transport mechanism and includes any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.
The system memory 1230 includes computer storage media in the form of volatile and/or nonvolatile memory or both such as read only memory (ROM) 1231 and random access memory (RAM) 1232. A basic input/output system 1233 (BIOS), containing the basic routines that help to transfer information between elements within computer 1210, such as during start-up, is typically stored in ROM 1231. RAM 1232 typically contains data or program modules or both that are immediately accessible to and/or presently being operated on by processing unit 1220. By way of example, and not limitation,
The computer 1210 may also include other removable/non-removable volatile/nonvolatile computer storage media. By way of example only,
Alternatively, or in addition, the functionality described herein can be performed, at least in part, by one or more hardware logic components. For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Application-specific Integrated Circuits (e.g., ASICs), Application-specific Standard Products (e.g., ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
The drives and their associated computer storage media discussed above and illustrated in
A user may enter commands and information into the computer 1210 through input devices such as a keyboard 1262, a microphone 1263, and a pointing device 1261, such as a mouse, trackball or touch pad. Other input devices (not shown) may include a joystick, game pad, satellite dish, scanner, or the like. These and other input devices are often connected to the processing unit 1220 through a user input interface 1260 that is coupled to the system bus, but may be connected by other interface and bus structures. A visual display 1291 or other type of display device is also connected to the system bus 1221 via an interface, such as a video interface 1290. In addition to the monitor, computers may also include other peripheral output devices such as speakers 1297 and printer 1296, which may be connected through an output peripheral interface 1295.
The computer 1210 is operated in a networked environment using logical connections (such as a controller area network—CAN, local area network—LAN, or wide area network WAN) to one or more remote computers, such as a remote computer 1280.
When used in a LAN networking environment, the computer 1210 is connected to the LAN 1271 through a network interface or adapter 1270. When used in a WAN networking environment, the computer 1210 typically includes a modem 1272 or other means for establishing communications over the WAN 1273, such as the Internet. In a networked environment, program modules may be stored in a remote memory storage device.
It should also be noted that the different examples described herein can be combined in different ways. That is, parts of one or more examples can be combined with parts of one or more other examples. All of this is contemplated herein.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of the claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3568157 | Downing et al. | Mar 1971 | A |
| 3580257 | Teague | May 1971 | A |
| 3599543 | Kerridge | Aug 1971 | A |
| 3775019 | Konig et al. | Nov 1973 | A |
| 3856754 | Habermeier et al. | Dec 1974 | A |
| 4129573 | Bellus et al. | Dec 1978 | A |
| 4166735 | Pilgram et al. | Sep 1979 | A |
| 4183742 | Sasse et al. | Jan 1980 | A |
| 4268679 | Lavanish | May 1981 | A |
| 4349377 | Durr et al. | Sep 1982 | A |
| 4360677 | Doweyko et al. | Nov 1982 | A |
| 4435203 | Funaki et al. | Mar 1984 | A |
| 4493726 | Burdeska et al. | Jan 1985 | A |
| 4527241 | Sheehan et al. | Jul 1985 | A |
| 4566901 | Martin et al. | Jan 1986 | A |
| 4584013 | Brunner | Apr 1986 | A |
| 4687505 | Sylling et al. | Aug 1987 | A |
| 4857101 | Musco et al. | Aug 1989 | A |
| 4911751 | Nyffeler et al. | Mar 1990 | A |
| 5059154 | Reyenga | Oct 1991 | A |
| 5089043 | Hayase et al. | Feb 1992 | A |
| 5246164 | McCann et al. | Sep 1993 | A |
| 5246915 | Lutz et al. | Sep 1993 | A |
| 5250690 | Turner et al. | Oct 1993 | A |
| 5296702 | Beck et al. | Mar 1994 | A |
| 5300477 | Tice | Apr 1994 | A |
| 5416061 | Hewett et al. | May 1995 | A |
| 5477459 | Clegg et al. | Dec 1995 | A |
| 5488817 | Paquet et al. | Feb 1996 | A |
| 5529537 | Johnson | Jun 1996 | A |
| 5563112 | Barnes, III | Oct 1996 | A |
| 5585626 | Beck et al. | Dec 1996 | A |
| 5586033 | Hall et al. | Dec 1996 | A |
| 5592606 | Myers | Jan 1997 | A |
| 5606821 | Sadjadi et al. | Mar 1997 | A |
| 5666793 | Bottinger | Sep 1997 | A |
| 5712782 | Weigelt et al. | Jan 1998 | A |
| 5721679 | Monson | Feb 1998 | A |
| 5767373 | Ward et al. | Jun 1998 | A |
| 5771169 | Wendte | Jun 1998 | A |
| 5789741 | Kinter et al. | Aug 1998 | A |
| 5809440 | Beck et al. | Sep 1998 | A |
| 5841282 | Christy et al. | Nov 1998 | A |
| 5849665 | Gut et al. | Dec 1998 | A |
| 5878821 | Flenker et al. | Mar 1999 | A |
| 5899950 | Milender et al. | May 1999 | A |
| 5902343 | Hale et al. | May 1999 | A |
| 5915492 | Yates et al. | Jun 1999 | A |
| 5957304 | Dawson | Sep 1999 | A |
| 5974348 | Rocks | Oct 1999 | A |
| 5978723 | Hale et al. | Nov 1999 | A |
| 5991687 | Hale et al. | Nov 1999 | A |
| 5991694 | Gudat et al. | Nov 1999 | A |
| 5995859 | Takahashi | Nov 1999 | A |
| 5995894 | Wendte | Nov 1999 | A |
| 5995895 | Watt et al. | Nov 1999 | A |
| 6004076 | Cook et al. | Dec 1999 | A |
| 6016713 | Hale | Jan 2000 | A |
| 6029106 | Hale et al. | Feb 2000 | A |
| 6037901 | Devier et al. | Mar 2000 | A |
| 6041582 | Tiede et al. | Mar 2000 | A |
| 6073070 | Diekhans | Jun 2000 | A |
| 6073428 | Diekhans | Jun 2000 | A |
| 6085135 | Steckel | Jul 2000 | A |
| 6119442 | Hale | Sep 2000 | A |
| 6119531 | Wendte et al. | Sep 2000 | A |
| 6128574 | Diekhans | Oct 2000 | A |
| 6141614 | Janzen et al. | Oct 2000 | A |
| 6178253 | Hendrickson et al. | Jan 2001 | B1 |
| 6185990 | Missotten et al. | Feb 2001 | B1 |
| 6188942 | Corcoran et al. | Feb 2001 | B1 |
| 6199000 | Keller et al. | Mar 2001 | B1 |
| 6204856 | Wood et al. | Mar 2001 | B1 |
| 6205381 | Motz et al. | Mar 2001 | B1 |
| 6205384 | Diekhans | Mar 2001 | B1 |
| 6216071 | Motz | Apr 2001 | B1 |
| 6236924 | Motz et al. | May 2001 | B1 |
| 6272819 | Wendte et al. | Aug 2001 | B1 |
| 6327569 | Reep | Dec 2001 | B1 |
| 6360179 | Reep | Mar 2002 | B1 |
| 6374173 | Ehlbeck et al. | Apr 2002 | B1 |
| 6380745 | Anderson et al. | Apr 2002 | B1 |
| 6431790 | Anderegg et al. | Aug 2002 | B1 |
| 6451733 | Pidskalny et al. | Sep 2002 | B1 |
| 6505146 | Blackmer et al. | Jan 2003 | B1 |
| 6505998 | Bullivant | Jan 2003 | B1 |
| 6539102 | Anderson et al. | Mar 2003 | B1 |
| 6549849 | Lange et al. | Apr 2003 | B2 |
| 6584390 | Beck | Jun 2003 | B2 |
| 6587772 | Behnke | Jul 2003 | B2 |
| 6591145 | Hoskinson et al. | Jul 2003 | B1 |
| 6591591 | Coers et al. | Jul 2003 | B2 |
| 6592453 | Coers et al. | Jul 2003 | B2 |
| 6604432 | Hamblen et al. | Aug 2003 | B1 |
| 6681551 | Sheidler et al. | Jan 2004 | B1 |
| 6682416 | Behnke et al. | Jan 2004 | B2 |
| 6687616 | Peterson et al. | Feb 2004 | B1 |
| 6729189 | Paakkinen | May 2004 | B2 |
| 6735568 | Buckwalter et al. | May 2004 | B1 |
| 6834550 | Upadhyaya et al. | Dec 2004 | B2 |
| 6838564 | Edmunds et al. | Jan 2005 | B2 |
| 6846128 | Sick | Jan 2005 | B2 |
| 6932554 | Isfort et al. | Aug 2005 | B2 |
| 6999877 | Dyer et al. | Feb 2006 | B1 |
| 7073374 | Berkman | Jul 2006 | B2 |
| 7155888 | Diekhans | Jan 2007 | B2 |
| 7167797 | Faivre et al. | Jan 2007 | B2 |
| 7167800 | Faivre et al. | Jan 2007 | B2 |
| 7191062 | Chi et al. | Mar 2007 | B2 |
| 7194965 | Hickey et al. | Mar 2007 | B2 |
| 7211994 | Mergen et al. | May 2007 | B1 |
| 7248968 | Reid | Jul 2007 | B2 |
| 7255016 | Burton | Aug 2007 | B2 |
| 7261632 | Pirro et al. | Aug 2007 | B2 |
| 7302837 | Wendt | Dec 2007 | B2 |
| 7308326 | Maertens et al. | Dec 2007 | B2 |
| 7313478 | Anderson et al. | Dec 2007 | B1 |
| 7318010 | Anderson | Jan 2008 | B2 |
| 7347168 | Reckels et al. | Mar 2008 | B2 |
| 7408145 | Holland | Aug 2008 | B2 |
| 7458428 | Laudick et al. | Dec 2008 | B2 |
| 7480564 | Metzler et al. | Jan 2009 | B2 |
| 7483791 | Anderegg et al. | Jan 2009 | B2 |
| 7537519 | Huster et al. | May 2009 | B2 |
| 7556283 | Fitzner et al. | Jun 2009 | B2 |
| 7557066 | Hills et al. | Jul 2009 | B2 |
| 7562167 | Anderson | Jul 2009 | B2 |
| 7628059 | Scherbring | Dec 2009 | B1 |
| 7665285 | Harada et al. | Feb 2010 | B1 |
| 7687435 | Witschel et al. | Mar 2010 | B2 |
| 7703036 | Satterfield et al. | Apr 2010 | B2 |
| 7725233 | Hendrickson et al. | May 2010 | B2 |
| 7733416 | Gal | Jun 2010 | B2 |
| 7756624 | Diekhans et al. | Jul 2010 | B2 |
| 7798894 | Isfort | Sep 2010 | B2 |
| 7827042 | Jung et al. | Nov 2010 | B2 |
| 7832126 | Koellner et al. | Nov 2010 | B2 |
| 7915200 | Epp et al. | Mar 2011 | B2 |
| 7945364 | Schricker et al. | May 2011 | B2 |
| 7993188 | Ritter | Aug 2011 | B2 |
| 8024074 | Stelford et al. | Sep 2011 | B2 |
| 8060283 | Mott et al. | Nov 2011 | B2 |
| 8107681 | Gaal | Jan 2012 | B2 |
| 8145393 | Foster et al. | Mar 2012 | B2 |
| 8147176 | Coers et al. | Apr 2012 | B2 |
| 8152610 | Harrington | Apr 2012 | B2 |
| 8170756 | Morey et al. | May 2012 | B2 |
| 8190335 | Vik et al. | May 2012 | B2 |
| 8195342 | Anderson | Jun 2012 | B2 |
| 8195358 | Anderson | Jun 2012 | B2 |
| 8213964 | Fitzner et al. | Jul 2012 | B2 |
| 8224500 | Anderson | Jul 2012 | B2 |
| 8229631 | Morey et al. | Jul 2012 | B2 |
| 8252723 | Jakobi et al. | Aug 2012 | B2 |
| 8254351 | Fitzner et al. | Aug 2012 | B2 |
| 8280595 | Foster et al. | Oct 2012 | B2 |
| 8321365 | Anderson | Nov 2012 | B2 |
| 8329717 | Minn et al. | Dec 2012 | B2 |
| 8332105 | Laux et al. | Dec 2012 | B2 |
| 8338332 | Hacker et al. | Dec 2012 | B1 |
| 8340862 | Baumgarten et al. | Dec 2012 | B2 |
| 8407157 | Anderson et al. | Mar 2013 | B2 |
| 8428829 | Brunnert et al. | Apr 2013 | B2 |
| 8437901 | Anderson | May 2013 | B2 |
| 8478493 | Anderson | Jul 2013 | B2 |
| 8488865 | Hausmann et al. | Jul 2013 | B2 |
| 8494727 | Green et al. | Jul 2013 | B2 |
| 8544397 | Bassett et al. | Oct 2013 | B2 |
| 8577561 | Green et al. | Nov 2013 | B2 |
| 8606454 | Wang et al. | Dec 2013 | B2 |
| 8626406 | Schleicher et al. | Jan 2014 | B2 |
| 8635903 | Oetken et al. | Jan 2014 | B2 |
| 8639408 | Anderson | Jan 2014 | B2 |
| 8649940 | Bonefas | Feb 2014 | B2 |
| 8656693 | Madsen et al. | Feb 2014 | B2 |
| 8662972 | Behnke et al. | Mar 2014 | B2 |
| 8671760 | Wallrath et al. | Mar 2014 | B2 |
| 8677724 | Chaney et al. | Mar 2014 | B2 |
| 8738238 | Rekow | May 2014 | B2 |
| 8738244 | Lenz | May 2014 | B2 |
| 8755976 | Peters et al. | Jun 2014 | B2 |
| 8781692 | Kormann | Jul 2014 | B2 |
| 8789563 | Wenzel | Jul 2014 | B2 |
| 8814640 | Behnke et al. | Aug 2014 | B2 |
| 8843269 | Anderson et al. | Sep 2014 | B2 |
| 8868304 | Bonefas | Oct 2014 | B2 |
| 8909389 | Meyer | Dec 2014 | B2 |
| D721740 | Schmaltz et al. | Jan 2015 | S |
| 8942860 | Morselli | Jan 2015 | B2 |
| 8962523 | Rosinger et al. | Feb 2015 | B2 |
| 9002591 | Wang et al. | Apr 2015 | B2 |
| 9008918 | Missotten et al. | Apr 2015 | B2 |
| 9009087 | Mewes et al. | Apr 2015 | B1 |
| 9011222 | Johnson et al. | Apr 2015 | B2 |
| 9014901 | Wang et al. | Apr 2015 | B2 |
| 9043096 | Zielke et al. | May 2015 | B2 |
| 9043129 | Bonefas et al. | May 2015 | B2 |
| 9066465 | Hendrickson et al. | Jun 2015 | B2 |
| 9072227 | Wenzel | Jul 2015 | B2 |
| 9095090 | Casper et al. | Aug 2015 | B2 |
| 9119342 | Bonefas | Sep 2015 | B2 |
| 9127428 | Meier | Sep 2015 | B2 |
| 9131644 | Osborne et al. | Sep 2015 | B2 |
| 9152938 | Lang et al. | Oct 2015 | B2 |
| 9173339 | Sauder et al. | Nov 2015 | B2 |
| 9179599 | Bischoff | Nov 2015 | B2 |
| 9188518 | Snyder et al. | Nov 2015 | B2 |
| 9188986 | Baumann | Nov 2015 | B2 |
| 9226449 | Bischoff | Jan 2016 | B2 |
| 9234317 | Chi | Jan 2016 | B2 |
| 9235214 | Anderson | Jan 2016 | B2 |
| 9301447 | Kormann | Apr 2016 | B2 |
| 9301466 | Kelly | Apr 2016 | B2 |
| 9313951 | Herman et al. | Apr 2016 | B2 |
| 9326443 | Zametzer et al. | May 2016 | B2 |
| 9326444 | Bonefas | May 2016 | B2 |
| 9392746 | Darr et al. | Jul 2016 | B2 |
| 9405039 | Anderson | Aug 2016 | B2 |
| 9410840 | Acheson et al. | Aug 2016 | B2 |
| 9439342 | Pasquier | Sep 2016 | B2 |
| 9457971 | Bonefas et al. | Oct 2016 | B2 |
| 9463939 | Bonefas et al. | Oct 2016 | B2 |
| 9485905 | Jung et al. | Nov 2016 | B2 |
| 9489576 | Johnson et al. | Nov 2016 | B2 |
| 9497898 | Dillon et al. | Nov 2016 | B2 |
| 9510508 | Jung et al. | Dec 2016 | B2 |
| 9511633 | Anderson et al. | Dec 2016 | B2 |
| 9511958 | Bonefas | Dec 2016 | B2 |
| 9516812 | Baumgarten et al. | Dec 2016 | B2 |
| 9521805 | Muench et al. | Dec 2016 | B2 |
| 9522791 | Bonefas et al. | Dec 2016 | B2 |
| 9522792 | Bonefas et al. | Dec 2016 | B2 |
| 9523180 | Deines | Dec 2016 | B2 |
| 9529364 | Foster et al. | Dec 2016 | B2 |
| 9532504 | Herman et al. | Jan 2017 | B2 |
| 9538714 | Anderson | Jan 2017 | B2 |
| 9563492 | Bell et al. | Feb 2017 | B2 |
| 9563848 | Hunt et al. | Feb 2017 | B1 |
| 9563852 | Wiles et al. | Feb 2017 | B1 |
| 9578808 | Dybro et al. | Feb 2017 | B2 |
| 9605583 | Goering et al. | Mar 2017 | B2 |
| 9605994 | Jensen | Mar 2017 | B2 |
| 9629308 | Schøler et al. | Apr 2017 | B2 |
| 9631964 | Gelinske et al. | Apr 2017 | B2 |
| 9642305 | Nykamp et al. | May 2017 | B2 |
| 9648807 | Escher et al. | May 2017 | B2 |
| 9675008 | Rusciolelli et al. | Jun 2017 | B1 |
| 9681605 | Noonan et al. | Jun 2017 | B2 |
| 9694712 | Healy et al. | Jul 2017 | B2 |
| 9696162 | Anderson | Jul 2017 | B2 |
| 9699967 | Palla et al. | Jul 2017 | B2 |
| 9714856 | Myers | Jul 2017 | B2 |
| 9717178 | Sauder et al. | Aug 2017 | B1 |
| 9721181 | Guan et al. | Aug 2017 | B2 |
| 9723790 | Berry et al. | Aug 2017 | B2 |
| 9740208 | Sugumaran et al. | Aug 2017 | B2 |
| 9767521 | Stuber et al. | Sep 2017 | B2 |
| 9772625 | Gilmore et al. | Sep 2017 | B2 |
| 9807934 | Rusciolelli et al. | Nov 2017 | B2 |
| 9807940 | Roell et al. | Nov 2017 | B2 |
| 9810679 | Kimmel | Nov 2017 | B2 |
| 9829364 | Wilson et al. | Nov 2017 | B2 |
| 9848528 | Werner et al. | Dec 2017 | B2 |
| 9856609 | Dehmel | Jan 2018 | B2 |
| 9856612 | Oetken | Jan 2018 | B2 |
| 9861040 | Bonefas | Jan 2018 | B2 |
| 9872433 | Acheson et al. | Jan 2018 | B2 |
| 9903077 | Rio | Feb 2018 | B2 |
| 9903979 | Dybro et al. | Feb 2018 | B2 |
| 9904963 | Rupp et al. | Feb 2018 | B2 |
| 9915952 | Dollinger et al. | Mar 2018 | B2 |
| 9922405 | Sauder et al. | Mar 2018 | B2 |
| 9924636 | Lisouski et al. | Mar 2018 | B2 |
| 9928584 | Jens et al. | Mar 2018 | B2 |
| 9933787 | Story | Apr 2018 | B2 |
| 9974226 | Rupp et al. | May 2018 | B2 |
| 9982397 | Korb et al. | May 2018 | B2 |
| 9984455 | Fox et al. | May 2018 | B1 |
| 9992931 | Bonefas et al. | Jun 2018 | B2 |
| 9992932 | Bonefas et al. | Jun 2018 | B2 |
| 10004176 | Mayerle | Jun 2018 | B2 |
| 10015928 | Nykamp et al. | Jul 2018 | B2 |
| 10019018 | Hulin | Jul 2018 | B2 |
| 10019790 | Bonefas et al. | Jul 2018 | B2 |
| 10025983 | Guan et al. | Jul 2018 | B2 |
| 10028435 | Anderson et al. | Jul 2018 | B2 |
| 10028451 | Rowan et al. | Jul 2018 | B2 |
| 10034427 | Krause et al. | Jul 2018 | B2 |
| 10039231 | Anderson et al. | Aug 2018 | B2 |
| 10064331 | Bradley | Sep 2018 | B2 |
| 10064335 | Byttebier et al. | Sep 2018 | B2 |
| 10078890 | Tagestad et al. | Sep 2018 | B1 |
| 10085372 | Noyer et al. | Oct 2018 | B2 |
| 10091925 | Aharoni et al. | Oct 2018 | B2 |
| 10126153 | Bischoff et al. | Nov 2018 | B2 |
| 10126282 | Anderson et al. | Nov 2018 | B2 |
| 10129528 | Bonefas et al. | Nov 2018 | B2 |
| 10143132 | Inoue et al. | Dec 2018 | B2 |
| 10152035 | Reid et al. | Dec 2018 | B2 |
| 10154624 | Guan et al. | Dec 2018 | B2 |
| 10165725 | Sugumaran et al. | Jan 2019 | B2 |
| 10178823 | Kovach et al. | Jan 2019 | B2 |
| 10183667 | Anderson et al. | Jan 2019 | B2 |
| 10188037 | Bruns et al. | Jan 2019 | B2 |
| 10201121 | Wilson | Feb 2019 | B1 |
| 10209179 | Hollstein | Feb 2019 | B2 |
| 10231371 | Dillon | Mar 2019 | B2 |
| 10254147 | Vermue et al. | Apr 2019 | B2 |
| 10254765 | Rekow et al. | Apr 2019 | B2 |
| 10255670 | Wu et al. | Apr 2019 | B1 |
| 10275550 | Lee | Apr 2019 | B2 |
| 10295703 | Dybro et al. | May 2019 | B2 |
| 10310455 | Blank et al. | Jun 2019 | B2 |
| 10314232 | Isaac et al. | Jun 2019 | B2 |
| 10315655 | Blank et al. | Jun 2019 | B2 |
| 10317272 | Bhavsar et al. | Jun 2019 | B2 |
| 10351364 | Green et al. | Jul 2019 | B2 |
| 10368488 | Becker et al. | Aug 2019 | B2 |
| 10398084 | Ray et al. | Sep 2019 | B2 |
| 10408545 | Blank et al. | Sep 2019 | B2 |
| 10412889 | Palla et al. | Sep 2019 | B2 |
| 10426086 | Van de Wege et al. | Oct 2019 | B2 |
| 10437243 | Blank et al. | Oct 2019 | B2 |
| 10477756 | Richt et al. | Nov 2019 | B1 |
| 10485178 | Mayerle | Nov 2019 | B2 |
| 10521526 | Haaland et al. | Dec 2019 | B2 |
| 10537061 | Farley et al. | Jan 2020 | B2 |
| 10631462 | Bonefas | Apr 2020 | B2 |
| 10677637 | Von Muenster | Jun 2020 | B1 |
| 10681872 | Viaene et al. | Jun 2020 | B2 |
| 10701861 | Gould et al. | Jul 2020 | B2 |
| 10703277 | Schroeder | Jul 2020 | B1 |
| 10729067 | Hammer et al. | Aug 2020 | B2 |
| 10733752 | Hageman et al. | Aug 2020 | B2 |
| 10740703 | Story | Aug 2020 | B2 |
| 10745868 | Laugwitz et al. | Aug 2020 | B2 |
| 10760946 | Meier et al. | Sep 2020 | B2 |
| 10809118 | Von Muenster | Oct 2020 | B1 |
| 10830634 | Blank et al. | Nov 2020 | B2 |
| 10890922 | Ramm et al. | Jan 2021 | B2 |
| 10912249 | Wilson | Feb 2021 | B1 |
| 10966071 | Suleman et al. | Mar 2021 | B2 |
| 11008177 | Banthia et al. | May 2021 | B2 |
| 11112262 | Anderson | Sep 2021 | B2 |
| 11308738 | Thomas et al. | Apr 2022 | B2 |
| 20020011061 | Lucand et al. | Jan 2002 | A1 |
| 20020083695 | Behnke et al. | Jul 2002 | A1 |
| 20020091458 | Moore | Jul 2002 | A1 |
| 20020099471 | Benneweis | Jul 2002 | A1 |
| 20020133309 | Hardt | Sep 2002 | A1 |
| 20020173893 | Blackmore et al. | Nov 2002 | A1 |
| 20020193928 | Beck | Dec 2002 | A1 |
| 20020193929 | Beck | Dec 2002 | A1 |
| 20020198654 | Lange et al. | Dec 2002 | A1 |
| 20030004630 | Beck | Jan 2003 | A1 |
| 20030014171 | Ma et al. | Jan 2003 | A1 |
| 20030015351 | Goldman et al. | Jan 2003 | A1 |
| 20030024450 | Juptner | Feb 2003 | A1 |
| 20030060245 | Coers et al. | Mar 2003 | A1 |
| 20030069680 | Cohen et al. | Apr 2003 | A1 |
| 20030075145 | Sheidler et al. | Apr 2003 | A1 |
| 20030174207 | Alexia et al. | Sep 2003 | A1 |
| 20030182144 | Pickett et al. | Sep 2003 | A1 |
| 20030187560 | Keller et al. | Oct 2003 | A1 |
| 20030216158 | Bischoff | Nov 2003 | A1 |
| 20030229432 | Ho et al. | Dec 2003 | A1 |
| 20030229433 | van den Berg et al. | Dec 2003 | A1 |
| 20030229435 | Van der Lely | Dec 2003 | A1 |
| 20040004544 | William Knutson | Jan 2004 | A1 |
| 20040054457 | Kormann | Mar 2004 | A1 |
| 20040073468 | Vyas et al. | Apr 2004 | A1 |
| 20040116169 | Isfort et al. | Jun 2004 | A1 |
| 20040193348 | Gray et al. | Sep 2004 | A1 |
| 20050059445 | Niermann et al. | Mar 2005 | A1 |
| 20050066738 | Moore | Mar 2005 | A1 |
| 20050150202 | Quick | Jul 2005 | A1 |
| 20050197175 | Anderson | Sep 2005 | A1 |
| 20050241285 | Maertens et al. | Nov 2005 | A1 |
| 20050279070 | Pirro et al. | Dec 2005 | A1 |
| 20050283314 | Hall | Dec 2005 | A1 |
| 20050284119 | Brunnert | Dec 2005 | A1 |
| 20060014489 | Fitzner et al. | Jan 2006 | A1 |
| 20060014643 | Hacker et al. | Jan 2006 | A1 |
| 20060047377 | Ferguson et al. | Mar 2006 | A1 |
| 20060047418 | Metzler et al. | Mar 2006 | A1 |
| 20060058896 | Pokorny et al. | Mar 2006 | A1 |
| 20060074560 | Dyer et al. | Apr 2006 | A1 |
| 20060094487 | Huster et al. | May 2006 | A1 |
| 20060155449 | Dammann | Jul 2006 | A1 |
| 20060162631 | Hickey et al. | Jul 2006 | A1 |
| 20060196158 | Faivre et al. | Sep 2006 | A1 |
| 20060200334 | Faivre et al. | Sep 2006 | A1 |
| 20070005209 | Fitzner et al. | Jan 2007 | A1 |
| 20070021948 | Anderson | Jan 2007 | A1 |
| 20070037621 | Isfort | Feb 2007 | A1 |
| 20070056258 | Behnke | Mar 2007 | A1 |
| 20070068238 | Wendte | Mar 2007 | A1 |
| 20070073700 | Wippersteg et al. | Mar 2007 | A1 |
| 20070089390 | Hendrickson et al. | Apr 2007 | A1 |
| 20070135190 | Diekhans et al. | Jun 2007 | A1 |
| 20070185749 | Anderson et al. | Aug 2007 | A1 |
| 20070199903 | Denny | Aug 2007 | A1 |
| 20070208510 | Anderson et al. | Sep 2007 | A1 |
| 20070233348 | Diekhans et al. | Oct 2007 | A1 |
| 20070233374 | Diekhans et al. | Oct 2007 | A1 |
| 20070239337 | Anderson | Oct 2007 | A1 |
| 20070282523 | Diekhans et al. | Dec 2007 | A1 |
| 20070298744 | Fitzner et al. | Dec 2007 | A1 |
| 20080005938 | Aebischer et al. | Jan 2008 | A1 |
| 20080030320 | Wilcox et al. | Feb 2008 | A1 |
| 20080098035 | Wippersteg et al. | Apr 2008 | A1 |
| 20080140431 | Anderson et al. | Jun 2008 | A1 |
| 20080162004 | Price et al. | Jul 2008 | A1 |
| 20080177449 | Pickett et al. | Jul 2008 | A1 |
| 20080245042 | Brunnert et al. | Oct 2008 | A1 |
| 20080248843 | Birrell et al. | Oct 2008 | A1 |
| 20080268927 | Farley et al. | Oct 2008 | A1 |
| 20080269052 | Rosinger et al. | Oct 2008 | A1 |
| 20080289308 | Brubaker | Nov 2008 | A1 |
| 20080312085 | Kordes et al. | Dec 2008 | A1 |
| 20090002187 | Kriel et al. | Jan 2009 | A1 |
| 20090044505 | Huster et al. | Feb 2009 | A1 |
| 20090074243 | Missotten et al. | Mar 2009 | A1 |
| 20090084173 | Gudat et al. | Apr 2009 | A1 |
| 20090143941 | Tarasinski et al. | Jun 2009 | A1 |
| 20090192654 | Wendte et al. | Jul 2009 | A1 |
| 20090216410 | Allen et al. | Aug 2009 | A1 |
| 20090226036 | Gaal | Sep 2009 | A1 |
| 20090259483 | Hendrickson et al. | Oct 2009 | A1 |
| 20090265098 | Dix | Oct 2009 | A1 |
| 20090306835 | Ellermann et al. | Dec 2009 | A1 |
| 20090311084 | Coers et al. | Dec 2009 | A1 |
| 20090312919 | Foster et al. | Dec 2009 | A1 |
| 20090312920 | Boenig et al. | Dec 2009 | A1 |
| 20090325658 | Phelan et al. | Dec 2009 | A1 |
| 20100036696 | Lang et al. | Feb 2010 | A1 |
| 20100042297 | Foster et al. | Feb 2010 | A1 |
| 20100063626 | Anderson | Mar 2010 | A1 |
| 20100063648 | Anderson | Mar 2010 | A1 |
| 20100063651 | Anderson | Mar 2010 | A1 |
| 20100063664 | Anderson | Mar 2010 | A1 |
| 20100063954 | Anderson | Mar 2010 | A1 |
| 20100070145 | Foster et al. | Mar 2010 | A1 |
| 20100071329 | Hindryckx et al. | Mar 2010 | A1 |
| 20100094481 | Anderson | Apr 2010 | A1 |
| 20100121541 | Behnke et al. | May 2010 | A1 |
| 20100137373 | Hungenberg et al. | Jun 2010 | A1 |
| 20100145572 | Steckel et al. | Jun 2010 | A1 |
| 20100152270 | Suty-Heinze et al. | Jun 2010 | A1 |
| 20100152943 | Matthews | Jun 2010 | A1 |
| 20100217474 | Baumgarten et al. | Aug 2010 | A1 |
| 20100245542 | Kim et al. | Sep 2010 | A1 |
| 20100268562 | Anderson | Oct 2010 | A1 |
| 20100268679 | Anderson | Oct 2010 | A1 |
| 20100285964 | Waldraff et al. | Nov 2010 | A1 |
| 20100317517 | Rosinger et al. | Dec 2010 | A1 |
| 20100319941 | Peterson | Dec 2010 | A1 |
| 20100332051 | Kormann | Dec 2010 | A1 |
| 20110022273 | Peters et al. | Jan 2011 | A1 |
| 20110059782 | Harrington | Mar 2011 | A1 |
| 20110061762 | Madsen et al. | Mar 2011 | A1 |
| 20110066337 | Kormann | Mar 2011 | A1 |
| 20110072773 | Schroeder et al. | Mar 2011 | A1 |
| 20110084851 | Peterson et al. | Apr 2011 | A1 |
| 20110086684 | Luellen et al. | Apr 2011 | A1 |
| 20110160961 | Wollenhaupt | Jun 2011 | A1 |
| 20110205084 | Morselli et al. | Aug 2011 | A1 |
| 20110213531 | Farley et al. | Sep 2011 | A1 |
| 20110224873 | Reeve et al. | Sep 2011 | A1 |
| 20110227745 | Kikuchi et al. | Sep 2011 | A1 |
| 20110257850 | Reeve et al. | Oct 2011 | A1 |
| 20110270495 | Knapp | Nov 2011 | A1 |
| 20110295460 | Hunt et al. | Dec 2011 | A1 |
| 20110307149 | Pighi et al. | Dec 2011 | A1 |
| 20120004813 | Baumgarten et al. | Jan 2012 | A1 |
| 20120029732 | Meyer et al. | Feb 2012 | A1 |
| 20120085458 | Wenzel | Apr 2012 | A1 |
| 20120087771 | Wenzel | Apr 2012 | A1 |
| 20120096827 | Chaney et al. | Apr 2012 | A1 |
| 20120136542 | Upcroft et al. | May 2012 | A1 |
| 20120143642 | O'Neil et al. | Jun 2012 | A1 |
| 20120215378 | Sprock et al. | Aug 2012 | A1 |
| 20120215379 | Sprock et al. | Aug 2012 | A1 |
| 20120215394 | Wang et al. | Aug 2012 | A1 |
| 20120215409 | Wang et al. | Aug 2012 | A1 |
| 20120253611 | Zielke et al. | Oct 2012 | A1 |
| 20120263560 | Diekhans et al. | Oct 2012 | A1 |
| 20120265412 | Diekhans et al. | Oct 2012 | A1 |
| 20120271489 | Roberts et al. | Oct 2012 | A1 |
| 20120302299 | Behnke et al. | Nov 2012 | A1 |
| 20120323452 | Green et al. | Dec 2012 | A1 |
| 20130019580 | Anderson et al. | Jan 2013 | A1 |
| 20130022430 | Anderson et al. | Jan 2013 | A1 |
| 20130046419 | Anderson et al. | Feb 2013 | A1 |
| 20130046439 | Anderson et al. | Feb 2013 | A1 |
| 20130046525 | Ali et al. | Feb 2013 | A1 |
| 20130103269 | Hellgen et al. | Apr 2013 | A1 |
| 20130124239 | Rosa et al. | May 2013 | A1 |
| 20130184944 | Missotten et al. | Jul 2013 | A1 |
| 20130197767 | Lenz | Aug 2013 | A1 |
| 20130205733 | Peters et al. | Aug 2013 | A1 |
| 20130210505 | Bischoff | Aug 2013 | A1 |
| 20130211675 | Bonefas | Aug 2013 | A1 |
| 20130213518 | Bonefas | Aug 2013 | A1 |
| 20130227922 | Zametzer et al. | Sep 2013 | A1 |
| 20130231823 | Wang et al. | Sep 2013 | A1 |
| 20130319941 | Schneider | Dec 2013 | A1 |
| 20130325242 | Cavender-Bares et al. | Dec 2013 | A1 |
| 20130329012 | Bartos et al. | Dec 2013 | A1 |
| 20130332003 | Murray et al. | Dec 2013 | A1 |
| 20130346127 | Jensen | Dec 2013 | A1 |
| 20140002489 | Sauder et al. | Jan 2014 | A1 |
| 20140019017 | Wilken et al. | Jan 2014 | A1 |
| 20140021598 | Sutardja | Jan 2014 | A1 |
| 20140050364 | Brueckner et al. | Feb 2014 | A1 |
| 20140067745 | Avey | Mar 2014 | A1 |
| 20140121882 | Gilmore | May 2014 | A1 |
| 20140129048 | Baumgarten et al. | May 2014 | A1 |
| 20140172222 | Nickel | Jun 2014 | A1 |
| 20140172224 | Matthews et al. | Jun 2014 | A1 |
| 20140172225 | Matthews et al. | Jun 2014 | A1 |
| 20140208870 | Quaderer et al. | Jul 2014 | A1 |
| 20140215984 | Bischoff | Aug 2014 | A1 |
| 20140224377 | Bonefas | Aug 2014 | A1 |
| 20140230391 | Hendrickson et al. | Aug 2014 | A1 |
| 20140230392 | Dybro et al. | Aug 2014 | A1 |
| 20140236381 | Anderson et al. | Aug 2014 | A1 |
| 20140236431 | Hendrickson et al. | Aug 2014 | A1 |
| 20140257911 | Anderson | Sep 2014 | A1 |
| 20140262547 | Acheson et al. | Sep 2014 | A1 |
| 20140277960 | Blank et al. | Sep 2014 | A1 |
| 20140297242 | Sauder et al. | Oct 2014 | A1 |
| 20140303814 | Burema et al. | Oct 2014 | A1 |
| 20140311113 | Bonefas | Oct 2014 | A1 |
| 20140324272 | Madsen et al. | Oct 2014 | A1 |
| 20140331631 | Sauder et al. | Nov 2014 | A1 |
| 20140338298 | Jung et al. | Nov 2014 | A1 |
| 20140350802 | Biggerstaff et al. | Nov 2014 | A1 |
| 20140360148 | Wienker et al. | Dec 2014 | A1 |
| 20150049088 | Snyder et al. | Feb 2015 | A1 |
| 20150088785 | Chi | Mar 2015 | A1 |
| 20150094944 | Baumann | Apr 2015 | A1 |
| 20150095830 | Massoumi et al. | Apr 2015 | A1 |
| 20150101519 | Blackwell et al. | Apr 2015 | A1 |
| 20150105984 | Birrell et al. | Apr 2015 | A1 |
| 20150124054 | Darr et al. | May 2015 | A1 |
| 20150168187 | Myers | Jun 2015 | A1 |
| 20150189832 | Krause et al. | Jul 2015 | A1 |
| 20150211199 | Corcoran et al. | Jul 2015 | A1 |
| 20150230403 | Jung et al. | Aug 2015 | A1 |
| 20150242799 | Seki et al. | Aug 2015 | A1 |
| 20150243114 | Tanabe et al. | Aug 2015 | A1 |
| 20150264863 | Muench | Sep 2015 | A1 |
| 20150276794 | Pistrol et al. | Oct 2015 | A1 |
| 20150278640 | Johnson et al. | Oct 2015 | A1 |
| 20150285647 | Meyer zu Helligen et al. | Oct 2015 | A1 |
| 20150293029 | Acheson et al. | Oct 2015 | A1 |
| 20150302305 | Rupp et al. | Oct 2015 | A1 |
| 20150305238 | Klausmann et al. | Oct 2015 | A1 |
| 20150305239 | Jung | Oct 2015 | A1 |
| 20150327425 | Dillon | Nov 2015 | A1 |
| 20150327440 | Dybro et al. | Nov 2015 | A1 |
| 20150351320 | Takahara et al. | Dec 2015 | A1 |
| 20150370935 | Starr | Dec 2015 | A1 |
| 20150373902 | Pasquier | Dec 2015 | A1 |
| 20150379785 | Brown, Jr. et al. | Dec 2015 | A1 |
| 20160025531 | Bischoff et al. | Jan 2016 | A1 |
| 20160029558 | Dybro et al. | Feb 2016 | A1 |
| 20160052525 | Tuncer et al. | Feb 2016 | A1 |
| 20160057922 | Freiberg et al. | Mar 2016 | A1 |
| 20160059854 | Ohsugi et al. | Mar 2016 | A1 |
| 20160061643 | Wilson et al. | Mar 2016 | A1 |
| 20160066505 | Bakke et al. | Mar 2016 | A1 |
| 20160073573 | Ethington et al. | Mar 2016 | A1 |
| 20160078375 | Ethington et al. | Mar 2016 | A1 |
| 20160078570 | Ethington et al. | Mar 2016 | A1 |
| 20160088794 | Baumgarten et al. | Mar 2016 | A1 |
| 20160106038 | Boyd et al. | Apr 2016 | A1 |
| 20160084813 | Anderson et al. | May 2016 | A1 |
| 20160146611 | Matthews | May 2016 | A1 |
| 20160183463 | Herman et al. | Jun 2016 | A1 |
| 20160202227 | Mathur et al. | Jul 2016 | A1 |
| 20160203657 | Bell et al. | Jul 2016 | A1 |
| 20160212939 | Ouchida et al. | Jul 2016 | A1 |
| 20160215994 | Mewes et al. | Jul 2016 | A1 |
| 20160232621 | Ethington et al. | Aug 2016 | A1 |
| 20160247075 | Mewes et al. | Aug 2016 | A1 |
| 20160247082 | Stehling | Aug 2016 | A1 |
| 20160252384 | Wilson et al. | Sep 2016 | A1 |
| 20160260021 | Marek | Sep 2016 | A1 |
| 20160270294 | Viaene et al. | Sep 2016 | A1 |
| 20160286720 | Heitmann et al. | Oct 2016 | A1 |
| 20160286721 | Heitmann et al. | Oct 2016 | A1 |
| 20160286722 | Heitmann et al. | Oct 2016 | A1 |
| 20160309656 | Wilken et al. | Oct 2016 | A1 |
| 20160327535 | Cotton et al. | Nov 2016 | A1 |
| 20160330906 | Acheson et al. | Nov 2016 | A1 |
| 20160338267 | Anderson et al. | Nov 2016 | A1 |
| 20160342915 | Humphrey | Nov 2016 | A1 |
| 20160345485 | Acheson et al. | Dec 2016 | A1 |
| 20160360697 | Diaz | Dec 2016 | A1 |
| 20170013773 | Kirk et al. | Jan 2017 | A1 |
| 20170031365 | Sugumaran et al. | Feb 2017 | A1 |
| 20170034997 | Mayerle | Feb 2017 | A1 |
| 20170042088 | Nykamp et al. | Feb 2017 | A1 |
| 20170049045 | Wilken et al. | Feb 2017 | A1 |
| 20170049053 | Bonefas et al. | Feb 2017 | A1 |
| 20170055433 | Jamison | Mar 2017 | A1 |
| 20170055446 | Nykamp et al. | Mar 2017 | A1 |
| 20170082442 | Anderson | Mar 2017 | A1 |
| 20170083024 | Reijersen Van Buuren | Mar 2017 | A1 |
| 20170086378 | Lisouski et al. | Mar 2017 | A1 |
| 20170086381 | Roell et al. | Mar 2017 | A1 |
| 20170089741 | Takahashi et al. | Mar 2017 | A1 |
| 20170089742 | Bruns et al. | Mar 2017 | A1 |
| 20170090068 | Xiang et al. | Mar 2017 | A1 |
| 20170105331 | Herlitzius et al. | Apr 2017 | A1 |
| 20170105335 | Xu et al. | Apr 2017 | A1 |
| 20170112049 | Weisberg et al. | Apr 2017 | A1 |
| 20170112061 | Meyer | Apr 2017 | A1 |
| 20170118915 | Middelberg et al. | May 2017 | A1 |
| 20170124463 | Chen et al. | May 2017 | A1 |
| 20170127606 | Horton | May 2017 | A1 |
| 20170160916 | Baumgarten et al. | Jun 2017 | A1 |
| 20170161627 | Xu et al. | Jun 2017 | A1 |
| 20170185086 | Sauder et al. | Jun 2017 | A1 |
| 20170188515 | Baumgarten et al. | Jul 2017 | A1 |
| 20170192431 | Foster et al. | Jul 2017 | A1 |
| 20170206645 | Bonefas et al. | Jul 2017 | A1 |
| 20170208742 | Ingibergsson et al. | Jul 2017 | A1 |
| 20170213141 | Xu et al. | Jul 2017 | A1 |
| 20170215330 | Missotten et al. | Aug 2017 | A1 |
| 20170227969 | Murray et al. | Aug 2017 | A1 |
| 20170235471 | Scholer et al. | Aug 2017 | A1 |
| 20170245434 | Jung et al. | Aug 2017 | A1 |
| 20170245435 | Rusciolelli et al. | Aug 2017 | A1 |
| 20170251600 | Anderson et al. | Sep 2017 | A1 |
| 20170270446 | Starr et al. | Sep 2017 | A1 |
| 20170270616 | Basso | Sep 2017 | A1 |
| 20170276534 | Vermue et al. | Sep 2017 | A1 |
| 20170316692 | Rusciolelli et al. | Nov 2017 | A1 |
| 20170318743 | Sauder et al. | Nov 2017 | A1 |
| 20170322550 | Yokoyama | Nov 2017 | A1 |
| 20170328030 | Yamada et al. | Nov 2017 | A1 |
| 20170332551 | Todd et al. | Nov 2017 | A1 |
| 20170336787 | Pichlmaier et al. | Nov 2017 | A1 |
| 20170370765 | Meier et al. | Dec 2017 | A1 |
| 20180000011 | Schleusner et al. | Jan 2018 | A1 |
| 20180014452 | Starr | Jan 2018 | A1 |
| 20180022559 | Knutson | Jan 2018 | A1 |
| 20180024549 | Hurd | Jan 2018 | A1 |
| 20180035603 | Kremmer et al. | Feb 2018 | A1 |
| 20180035622 | Gresch et al. | Feb 2018 | A1 |
| 20180037424 | Green et al. | Feb 2018 | A1 |
| 20180054955 | Oliver | Mar 2018 | A1 |
| 20180060975 | Hassanzadeh | Mar 2018 | A1 |
| 20180070534 | Mayerle | Mar 2018 | A1 |
| 20180077865 | Gallmeier | Mar 2018 | A1 |
| 20180084709 | Wieckhorst et al. | Mar 2018 | A1 |
| 20180084722 | Wieckhorst et al. | Mar 2018 | A1 |
| 20180092301 | Vandike et al. | Apr 2018 | A1 |
| 20180092302 | Vandike et al. | Apr 2018 | A1 |
| 20180108123 | Baurer et al. | Apr 2018 | A1 |
| 20180120098 | Matsuo et al. | May 2018 | A1 |
| 20180120133 | Blank et al. | May 2018 | A1 |
| 20180121821 | Parsons et al. | May 2018 | A1 |
| 20180124992 | Koch et al. | May 2018 | A1 |
| 20180128933 | Koch et al. | May 2018 | A1 |
| 20180132422 | Hassanzadeh et al. | May 2018 | A1 |
| 20180136664 | Tomita et al. | May 2018 | A1 |
| 20180146612 | Sauder et al. | May 2018 | A1 |
| 20180146624 | Chen et al. | May 2018 | A1 |
| 20180153084 | Calleija et al. | Jun 2018 | A1 |
| 20180174291 | Asada et al. | Jun 2018 | A1 |
| 20180177125 | Takahara et al. | Jun 2018 | A1 |
| 20180181893 | Basso | Jun 2018 | A1 |
| 20180196438 | Newlin et al. | Jul 2018 | A1 |
| 20180196441 | Muench et al. | Jul 2018 | A1 |
| 20180211156 | Guan et al. | Jul 2018 | A1 |
| 20180232674 | Bilde | Aug 2018 | A1 |
| 20180242523 | Kirchbeck et al. | Aug 2018 | A1 |
| 20180245308 | Ready-Campbell et al. | Aug 2018 | A1 |
| 20180249641 | Lewis et al. | Sep 2018 | A1 |
| 20180257657 | Blank et al. | Sep 2018 | A1 |
| 20180271015 | Redden et al. | Sep 2018 | A1 |
| 20180279599 | Struve | Oct 2018 | A1 |
| 20180295771 | Peters | Oct 2018 | A1 |
| 20180310474 | Posselius et al. | Nov 2018 | A1 |
| 20180317381 | Bassett | Nov 2018 | A1 |
| 20180317385 | Wellensiek et al. | Nov 2018 | A1 |
| 20180325012 | Ferrari et al. | Nov 2018 | A1 |
| 20180325014 | Debbaut | Nov 2018 | A1 |
| 20180332767 | Muench et al. | Nov 2018 | A1 |
| 20180338422 | Brubaker | Nov 2018 | A1 |
| 20180340316 | Izumikawa | Nov 2018 | A1 |
| 20180340845 | Rhodes et al. | Nov 2018 | A1 |
| 20180359917 | Blank et al. | Dec 2018 | A1 |
| 20180359919 | Blank et al. | Dec 2018 | A1 |
| 20180364726 | Foster et al. | Dec 2018 | A1 |
| 20190021226 | Dima et al. | Jan 2019 | A1 |
| 20190025175 | Laugwitz | Jan 2019 | A1 |
| 20190026914 | Hageman et al. | Jan 2019 | A1 |
| 20190041813 | Horn et al. | Feb 2019 | A1 |
| 20190050948 | Perry et al. | Feb 2019 | A1 |
| 20190057460 | Sakaguchi et al. | Feb 2019 | A1 |
| 20190066234 | Bedoya et al. | Feb 2019 | A1 |
| 20190069470 | Pfeiffer et al. | Mar 2019 | A1 |
| 20190075727 | Duke et al. | Mar 2019 | A1 |
| 20190085785 | Abolt | Mar 2019 | A1 |
| 20190090423 | Escher et al. | Mar 2019 | A1 |
| 20190098825 | Neitemeier et al. | Apr 2019 | A1 |
| 20190104722 | Slaughter et al. | Apr 2019 | A1 |
| 20190108413 | Chen et al. | Apr 2019 | A1 |
| 20190124819 | Madsen et al. | May 2019 | A1 |
| 20190136491 | Martin | May 2019 | A1 |
| 20190138962 | Ehlmann et al. | May 2019 | A1 |
| 20190147094 | Zhan et al. | May 2019 | A1 |
| 20190147249 | Kiepe et al. | May 2019 | A1 |
| 20190156255 | Carroll | May 2019 | A1 |
| 20190166761 | Conrad et al. | Jun 2019 | A1 |
| 20190174667 | Gresch et al. | Jun 2019 | A1 |
| 20190183047 | Dybro et al. | Jun 2019 | A1 |
| 20190200522 | Hansen et al. | Jul 2019 | A1 |
| 20190230855 | Reed et al. | Aug 2019 | A1 |
| 20190239416 | Green et al. | Aug 2019 | A1 |
| 20190261550 | Damme et al. | Aug 2019 | A1 |
| 20190261559 | Heitmann et al. | Aug 2019 | A1 |
| 20190261560 | Jelenkavic | Aug 2019 | A1 |
| 20190313570 | Owechko | Oct 2019 | A1 |
| 20190327889 | Borgstadt | Oct 2019 | A1 |
| 20190327892 | Fries et al. | Oct 2019 | A1 |
| 20190335662 | Good et al. | Nov 2019 | A1 |
| 20190335674 | Basso | Nov 2019 | A1 |
| 20190343035 | Smith et al. | Nov 2019 | A1 |
| 20190343043 | Bormann et al. | Nov 2019 | A1 |
| 20190343044 | Bormann et al. | Nov 2019 | A1 |
| 20190343048 | Farley et al. | Nov 2019 | A1 |
| 20190351765 | Rabusic | Nov 2019 | A1 |
| 20190354081 | Blank et al. | Nov 2019 | A1 |
| 20190364733 | Laugen et al. | Dec 2019 | A1 |
| 20190364734 | Kriebel | Dec 2019 | A1 |
| 20200000006 | McDonald et al. | Jan 2020 | A1 |
| 20200008351 | Zielke et al. | Jan 2020 | A1 |
| 20200015416 | Barther et al. | Jan 2020 | A1 |
| 20200019159 | Kocer et al. | Jan 2020 | A1 |
| 20200024102 | Brill et al. | Jan 2020 | A1 |
| 20200029488 | Bertucci et al. | Jan 2020 | A1 |
| 20200034759 | Dumstorff et al. | Jan 2020 | A1 |
| 20200037491 | Schoeny et al. | Feb 2020 | A1 |
| 20200053961 | Dix et al. | Feb 2020 | A1 |
| 20200064144 | Tomita et al. | Feb 2020 | A1 |
| 20200064863 | Tomita et al. | Feb 2020 | A1 |
| 20200074023 | Nizami et al. | Mar 2020 | A1 |
| 20200084963 | Gururajan et al. | Mar 2020 | A1 |
| 20200084966 | Corban et al. | Mar 2020 | A1 |
| 20200090094 | Blank | Mar 2020 | A1 |
| 20200097851 | Alvarez et al. | Mar 2020 | A1 |
| 20200113142 | Coleman et al. | Apr 2020 | A1 |
| 20200125822 | Yang et al. | Apr 2020 | A1 |
| 20200128732 | Chaney | Apr 2020 | A1 |
| 20200128733 | Vandike et al. | Apr 2020 | A1 |
| 20200128734 | Brammeier et al. | Apr 2020 | A1 |
| 20200128735 | Bonefas et al. | Apr 2020 | A1 |
| 20200128737 | Anderson et al. | Apr 2020 | A1 |
| 20200128738 | Suleman | Apr 2020 | A1 |
| 20200128740 | Suleman | Apr 2020 | A1 |
| 20200133262 | Suleman et al. | Apr 2020 | A1 |
| 20200137957 | Friesen | May 2020 | A1 |
| 20200141784 | Lange et al. | May 2020 | A1 |
| 20200146203 | Deng | May 2020 | A1 |
| 20200150631 | Frieberg et al. | May 2020 | A1 |
| 20200154639 | Takahara et al. | May 2020 | A1 |
| 20200163277 | Cooksey et al. | May 2020 | A1 |
| 20200180493 | Thompson | Jun 2020 | A1 |
| 20200183406 | Borgstadt | Jun 2020 | A1 |
| 20200187409 | Meyer Zu Helligen | Jun 2020 | A1 |
| 20200196526 | Koch et al. | Jun 2020 | A1 |
| 20200202596 | Kitahara et al. | Jun 2020 | A1 |
| 20200221632 | Strnad et al. | Jul 2020 | A1 |
| 20200221635 | Hendrickson et al. | Jul 2020 | A1 |
| 20200221636 | Boydens et al. | Jul 2020 | A1 |
| 20200265527 | Rose et al. | Aug 2020 | A1 |
| 20200278680 | Schultz et al. | Sep 2020 | A1 |
| 20200317114 | Hoff | Oct 2020 | A1 |
| 20200319632 | Desai et al. | Oct 2020 | A1 |
| 20200319655 | Desai et al. | Oct 2020 | A1 |
| 20200323133 | Anderson et al. | Oct 2020 | A1 |
| 20200323134 | Darr et al. | Oct 2020 | A1 |
| 20200325653 | Hageman et al. | Oct 2020 | A1 |
| 20200325655 | Hageman et al. | Oct 2020 | A1 |
| 20200326674 | Palla et al. | Oct 2020 | A1 |
| 20200326727 | Palla | Oct 2020 | A1 |
| 20200333278 | Locken et al. | Oct 2020 | A1 |
| 20200337232 | Blank et al. | Oct 2020 | A1 |
| 20200352099 | Meier et al. | Nov 2020 | A1 |
| 20200359547 | Sakaguchi et al. | Nov 2020 | A1 |
| 20200359549 | Sakaguchi et al. | Nov 2020 | A1 |
| 20200363256 | Meier et al. | Nov 2020 | A1 |
| 20200375083 | Anderson et al. | Dec 2020 | A1 |
| 20200375084 | Sakaguchi et al. | Dec 2020 | A1 |
| 20200378088 | Anderson | Dec 2020 | A1 |
| 20200404842 | Dugas et al. | Dec 2020 | A1 |
| 20210015041 | Bormann et al. | Jan 2021 | A1 |
| 20210022283 | Vandike et al. | Jan 2021 | A1 |
| 20210026362 | Wilson et al. | Jan 2021 | A1 |
| 20210029877 | Vandike | Feb 2021 | A1 |
| 20210099522 | Chiocco et al. | Apr 2021 | A1 |
| 20210129853 | Appleton et al. | May 2021 | A1 |
| 20210176916 | Sidon et al. | Jun 2021 | A1 |
| 20210176918 | Franzen et al. | Jun 2021 | A1 |
| 20210195840 | Puryk et al. | Jul 2021 | A1 |
| 20210289687 | Heinold et al. | Sep 2021 | A1 |
| 20210294337 | Van Mill et al. | Sep 2021 | A1 |
| 20210321567 | Sidon et al. | Oct 2021 | A1 |
| 20210333802 | Singh et al. | Oct 2021 | A1 |
| 20210337729 | O'Connor et al. | Nov 2021 | A1 |
| 20210339729 | O'Connor et al. | Nov 2021 | A1 |
| 20220015288 | Christiansen et al. | Jan 2022 | A1 |
| 20220019238 | Christiansen et al. | Jan 2022 | A1 |
| 20220019239 | Christiansen et al. | Jan 2022 | A1 |
| 20220019240 | Christiansen et al. | Jan 2022 | A1 |
| 20220019241 | Christiansen et al. | Jan 2022 | A1 |
| 20220039318 | Raste et al. | Feb 2022 | A1 |
| 20220095539 | Faust et al. | Mar 2022 | A1 |
| 20220110238 | Vandike et al. | Apr 2022 | A1 |
| 20220110246 | Vandike et al. | Apr 2022 | A1 |
| 20220110253 | Anderson et al. | Apr 2022 | A1 |
| 20220110262 | Vandike et al. | Apr 2022 | A1 |
| 20220113727 | Vandike et al. | Apr 2022 | A1 |
| 20220117143 | Kraus et al. | Apr 2022 | A1 |
| 20220122197 | Hanrieder | Apr 2022 | A1 |
| 20220132736 | Meyers et al. | May 2022 | A1 |
| 20220132737 | Anderson et al. | May 2022 | A1 |
| 20220176818 | Piscopo | Jun 2022 | A1 |
| 20220197302 | McClelland et al. | Jun 2022 | A1 |
| 20220204051 | Rands et al. | Jun 2022 | A1 |
| 20220204052 | Creaby et al. | Jun 2022 | A1 |
| 20220206512 | Rands et al. | Jun 2022 | A1 |
| 20220207852 | Sharma | Jun 2022 | A1 |
| 20220225568 | Töniges | Jul 2022 | A1 |
| 20220232763 | Palla et al. | Jul 2022 | A1 |
| 20220232770 | Yanke et al. | Jul 2022 | A1 |
| 20220256770 | Faust et al. | Aug 2022 | A1 |
| 20220304227 | Hill et al. | Sep 2022 | A1 |
| 20220304231 | Faust et al. | Sep 2022 | A1 |
| 20220374026 | Vougioukas et al. | Nov 2022 | A1 |
| 20220382277 | Nishii et al. | Dec 2022 | A1 |
| 20220408645 | O'Connor et al. | Dec 2022 | A1 |
| 20220410704 | O'Connor et al. | Dec 2022 | A1 |
| 20230027697 | Rush | Jan 2023 | A1 |
| 20230031013 | Faust et al. | Feb 2023 | A1 |
| 20230035890 | McKinney et al. | Feb 2023 | A1 |
| 20230041687 | Kormann | Feb 2023 | A1 |
| 20230232739 | Vandike et al. | Jul 2023 | A1 |
| 20230247939 | Vandike et al. | Aug 2023 | A1 |
| Number | Date | Country |
|---|---|---|
| 108898 | Oct 2018 | AR |
| 20100224431 | Apr 2011 | AU |
| 6800140 | Dec 1989 | BR |
| PI0502658 | Feb 2007 | BR |
| PI0802384 | Mar 2010 | BR |
| PI1100258 | Mar 2014 | BR |
| 102014007178 | Aug 2016 | BR |
| 102019000830 | Jul 2020 | BR |
| 1165300 | Apr 1984 | CA |
| 2283767 | Mar 2001 | CA |
| 2330979 | Aug 2001 | CA |
| 2629555 | Nov 2009 | CA |
| 135611 | May 2011 | CA |
| 3081383 | May 2019 | CA |
| 2451633 | Oct 2001 | CN |
| 101236188 | Aug 2008 | CN |
| 100416590 | Sep 2008 | CN |
| 101303338 | Nov 2008 | CN |
| 101363833 | Feb 2009 | CN |
| 201218789 | Apr 2009 | CN |
| 201528545 | Jul 2010 | CN |
| 101839906 | Sep 2010 | CN |
| 101929166 | Dec 2010 | CN |
| 102080373 | Jun 2011 | CN |
| 102138383 | Aug 2011 | CN |
| 202110103 | Jan 2012 | CN |
| 202119772 | Jan 2012 | CN |
| 102564528 | Jul 2012 | CN |
| 202340435 | Jul 2012 | CN |
| 103088807 | May 2013 | CN |
| 103181263 | Jul 2013 | CN |
| 203053961 | Jul 2013 | CN |
| 203055121 | Jul 2013 | CN |
| 203206739 | Sep 2013 | CN |
| 102277867 | Oct 2013 | CN |
| 203275401 | Nov 2013 | CN |
| 203613525 | May 2014 | CN |
| 203658201 | Jun 2014 | CN |
| 103954738 | Jul 2014 | CN |
| 203741803 | Jul 2014 | CN |
| 104247607 | Dec 2014 | CN |
| 204000818 | Dec 2014 | CN |
| 204435344 | Jul 2015 | CN |
| 204475304 | Jul 2015 | CN |
| 105205248 | Dec 2015 | CN |
| 204989174 | Jan 2016 | CN |
| 105432228 | Mar 2016 | CN |
| 105741180 | Jul 2016 | CN |
| 106053330 | Oct 2016 | CN |
| 106198879 | Dec 2016 | CN |
| 106226470 | Dec 2016 | CN |
| 106248873 | Dec 2016 | CN |
| 1061106198877 | Dec 2016 | CN |
| 106290800 | Jan 2017 | CN |
| 106327349 | Jan 2017 | CN |
| 106644663 | May 2017 | CN |
| 206330815 | Jul 2017 | CN |
| 206515118 | Sep 2017 | CN |
| 206515119 | Sep 2017 | CN |
| 206616118 | Nov 2017 | CN |
| 206696107 | Dec 2017 | CN |
| 206696107 | Dec 2017 | CN |
| 107576674 | Jan 2018 | CN |
| 107576674 | Jan 2018 | CN |
| 206906093 | Jan 2018 | CN |
| 206941558 | Jan 2018 | CN |
| 206941558 | Jan 2018 | CN |
| 107736088 | Feb 2018 | CN |
| 107795095 | Mar 2018 | CN |
| 207079558 | Mar 2018 | CN |
| 107941286 | Apr 2018 | CN |
| 107957408 | Apr 2018 | CN |
| 108009542 | May 2018 | CN |
| 108304796 | Jul 2018 | CN |
| 207567744 | Jul 2018 | CN |
| 108614089 | Oct 2018 | CN |
| 208013131 | Oct 2018 | CN |
| 108881825 | Nov 2018 | CN |
| 208047351 | Nov 2018 | CN |
| 109392426 | Jan 2019 | CN |
| 109357804 | Feb 2019 | CN |
| 109485353 | Mar 2019 | CN |
| 109633127 | Apr 2019 | CN |
| 109763476 | May 2019 | CN |
| 109961024 | Jul 2019 | CN |
| 110262287 | Sep 2019 | CN |
| 110720302 | Jan 2020 | CN |
| 111201879 | May 2020 | CN |
| 210585958 | May 2020 | CN |
| 111406505 | Jul 2020 | CN |
| 114846991 | Aug 2022 | CN |
| 114915637 | Aug 2022 | CN |
| 115067061 | Sep 2022 | CN |
| 115280960 | Nov 2022 | CN |
| 115643895 | Jan 2023 | CN |
| 247426 | Dec 1986 | CS |
| 248318 | Feb 1987 | CS |
| 17266 | Feb 2007 | CZ |
| 20252 | Nov 2009 | CZ |
| 441597 | Mar 1927 | DE |
| 504035 | Jul 1930 | DE |
| 2354828 | May 1975 | DE |
| 152380 | Nov 1981 | DE |
| 3728669 | Mar 1989 | DE |
| 4431824 | May 1996 | DE |
| 19509496 | Sep 1996 | DE |
| 19528663 | Feb 1997 | DE |
| 19718455 | Nov 1997 | DE |
| 19705842 | Aug 1998 | DE |
| 19828355 | Jan 2000 | DE |
| 10050224 | Apr 2002 | DE |
| 10120173 | Oct 2002 | DE |
| 202004015141 | Dec 2004 | DE |
| 102005000770 | Jul 2006 | DE |
| 102005000771 | Aug 2006 | DE |
| 102008021785 | Nov 2009 | DE |
| 102009041646 | Mar 2011 | DE |
| 102010004648 | Jul 2011 | DE |
| 102010038661 | Feb 2012 | DE |
| 102011005400 | Sep 2012 | DE |
| 202012103730 | Oct 2012 | DE |
| 102011052688 | Feb 2013 | DE |
| 102012211001 | Jan 2014 | DE |
| 102012220109 | May 2014 | DE |
| 102012223768 | Jun 2014 | DE |
| 102013212151 | Dec 2014 | DE |
| 102013012027 | Jan 2015 | DE |
| 102013019098 | Jan 2015 | DE |
| 102014108449 | Feb 2015 | DE |
| 102011005400 | May 2015 | DE |
| 2014201203 | Jul 2015 | DE |
| 102014208068 | Oct 2015 | DE |
| 102015006398 | May 2016 | DE |
| 102015109799 | Dec 2016 | DE |
| 112015002194 | Jan 2017 | DE |
| 102017204511 | Sep 2018 | DE |
| 102019206734 | Nov 2020 | DE |
| 102019114872 | Dec 2020 | DE |
| 102011052688 | Feb 2021 | DE |
| 102020124508 | Mar 2022 | DE |
| 0070219 | Oct 1984 | EP |
| 0355049 | Feb 1990 | EP |
| 845198 | Jun 1998 | EP |
| 0532146 | Aug 1998 | EP |
| 1219153 | Jul 2002 | EP |
| 1219159 | Jul 2002 | EP |
| 1444879 | Aug 2004 | EP |
| 1219159 | Jun 2005 | EP |
| 1219153 | Feb 2006 | EP |
| 1674324 | Jun 2006 | EP |
| 1692928 | Aug 2006 | EP |
| 1574122 | Feb 2008 | EP |
| 1609349 | May 2008 | EP |
| 1943877 | Jul 2008 | EP |
| 2057884 | May 2009 | EP |
| 1598586 | Sep 2009 | EP |
| 1731983 | Sep 2009 | EP |
| 2146307 | Jan 2010 | EP |
| 2174537 | Apr 2010 | EP |
| 2186389 | May 2010 | EP |
| 2267566 | Dec 2010 | EP |
| 3491192 | Dec 2010 | EP |
| 2057884 | Jan 2011 | EP |
| 2359674 | Aug 2011 | EP |
| 2301318 | Nov 2011 | EP |
| 2146307 | May 2012 | EP |
| 2446732 | May 2012 | EP |
| 2508057 | Oct 2012 | EP |
| 2524586 | Nov 2012 | EP |
| 2529610 | Dec 2012 | EP |
| 2243353 | Mar 2013 | EP |
| 2174537 | May 2013 | EP |
| 2592919 | May 2013 | EP |
| 2621258 | Aug 2013 | EP |
| 1674324 | May 2014 | EP |
| 2759829 | Jul 2014 | EP |
| 2764764 | Aug 2014 | EP |
| 2786325 | Oct 2014 | EP |
| 2267566 | Dec 2014 | EP |
| 2829171 | Jan 2015 | EP |
| 2191439 | Mar 2015 | EP |
| 2586286 | Mar 2015 | EP |
| 2592919 | Sep 2015 | EP |
| 2921042 | Sep 2015 | EP |
| 2944725 | Nov 2015 | EP |
| 2510777 | Mar 2016 | EP |
| 2997805 | Mar 2016 | EP |
| 3000302 | Mar 2016 | EP |
| 2868806 | Jul 2016 | EP |
| 3085221 | Oct 2016 | EP |
| 3095310 | Nov 2016 | EP |
| 3097759 | Nov 2016 | EP |
| 2452551 | May 2017 | EP |
| 3175691 | Jun 2017 | EP |
| 3195719 | Jul 2017 | EP |
| 3195720 | Jul 2017 | EP |
| 2995191 | Oct 2017 | EP |
| 3259976 | Dec 2017 | EP |
| 3262934 | Jan 2018 | EP |
| 3491192 | Jan 2018 | EP |
| 3287007 | Feb 2018 | EP |
| 3298876 | Mar 2018 | EP |
| 3300579 | Apr 2018 | EP |
| 3315005 | May 2018 | EP |
| 3315007 | May 2018 | EP |
| 3316006 | May 2018 | EP |
| 3316208 | May 2018 | EP |
| 3316218 | May 2018 | EP |
| 2829171 | Jun 2018 | EP |
| 2508057 | Jul 2018 | EP |
| 2508057 | Jul 2018 | EP |
| 3378298 | Sep 2018 | EP |
| 3378299 | Sep 2018 | EP |
| 3384754 | Oct 2018 | EP |
| 3289853 81 | Mar 2019 | EP |
| 3289853 | Mar 2019 | EP |
| 3448145 | Mar 2019 | EP |
| 3456167 | Mar 2019 | EP |
| 3466239 | Apr 2019 | EP |
| 3289852 81 | Jun 2019 | EP |
| 3289852 | Jun 2019 | EP |
| 3494770 | Jun 2019 | EP |
| 3498074 | Jun 2019 | EP |
| 3000302 | Aug 2019 | EP |
| 3533314 | Sep 2019 | EP |
| 3569049 | Nov 2019 | EP |
| 3000307 81 | Dec 2019 | EP |
| 3000307 | Dec 2019 | EP |
| 3586592 | Jan 2020 | EP |
| 3593613 | Jan 2020 | EP |
| 3593620 | Jan 2020 | EP |
| 3448144 | Feb 2020 | EP |
| 3613272 | Feb 2020 | EP |
| 3243374 | Mar 2020 | EP |
| 3626038 | Mar 2020 | EP |
| 3259976 | Apr 2020 | EP |
| 3635647 | Apr 2020 | EP |
| 3378298 | May 2020 | EP |
| 3646699 | May 2020 | EP |
| 3662741 | Jun 2020 | EP |
| 3685648 | Jul 2020 | EP |
| 2995191 | Oct 2020 | EP |
| 2812267 | Feb 2021 | EP |
| 3847878 | Jul 2021 | EP |
| 3706544 | Dec 2021 | EP |
| 3970471 | Mar 2022 | EP |
| 3315005 | Apr 2022 | EP |
| 2116215 | Jul 1998 | ES |
| 2311322 | Feb 2009 | ES |
| 5533 | Nov 1913 | FI |
| 1451480 | Jan 1966 | FR |
| 2817344 | May 2002 | FR |
| 2901291 | Nov 2007 | FR |
| 2901291 | Nov 2007 | FR |
| 901081 | Jul 1962 | GB |
| 2517049 | Feb 2015 | GB |
| 201519517 | May 2017 | GB |
| 01632 | Aug 2016 | IN |
| 1632DE2014 | Aug 2016 | IN |
| 1632DE2014 | Aug 2016 | IN |
| 201641027017 | Oct 2016 | IN |
| 202041039250 | Sep 2020 | IN |
| 202241032643 | Jun 2022 | IN |
| 7079681 | Nov 1982 | JP |
| S60253617 | Dec 1985 | JP |
| S63308110 | Dec 1988 | JP |
| H02196960 | Aug 1990 | JP |
| H02215311 | Aug 1990 | JP |
| H0779681 | Mar 1995 | JP |
| H1066436 | Mar 1998 | JP |
| H10191762 | Jul 1998 | JP |
| 2000352044 | Dec 2000 | JP |
| 2001057809 | Mar 2001 | JP |
| 2002186348 | Jul 2002 | JP |
| 2005227233 | Aug 2005 | JP |
| 2006166871 | Jun 2006 | JP |
| 2011205967 | Oct 2011 | JP |
| 2015070812 | Apr 2015 | JP |
| 2015084667 | May 2015 | JP |
| 2015151826 | Aug 2015 | JP |
| 2015219651 | Dec 2015 | JP |
| 2016071726 | May 2016 | JP |
| 2016160808 | Sep 2016 | JP |
| 6087258 | Mar 2017 | JP |
| 2017136035 | Aug 2017 | JP |
| 2017137729 | Aug 2017 | JP |
| 2017195804 | Nov 2017 | JP |
| 2018068284 | May 2018 | JP |
| 2018102154 | Jul 2018 | JP |
| 2018151388 | Sep 2018 | JP |
| 2019004796 | Jan 2019 | JP |
| 2019129744 | Aug 2019 | JP |
| 2019146506 | Sep 2019 | JP |
| 2019216744 | Dec 2019 | JP |
| 2020018255 | Feb 2020 | JP |
| 2020031607 | Mar 2020 | JP |
| 2020113062 | Jul 2020 | JP |
| 2020127405 | Aug 2020 | JP |
| 2021069341 | May 2021 | JP |
| 2022141070 | Sep 2022 | JP |
| 100974892 | Aug 2010 | KR |
| 100974892 | Aug 2010 | KR |
| 20110018582 | Feb 2011 | KR |
| 101067576 | Sep 2011 | KR |
| 101067576 | Sep 2011 | KR |
| 101134075 | Apr 2012 | KR |
| 101447197 | Oct 2014 | KR |
| 101653750 | Sep 2016 | KR |
| 20170041377 | Apr 2017 | KR |
| 200485051 | Nov 2017 | KR |
| 200485051 | Nov 2017 | KR |
| 101873657 | Aug 2018 | KR |
| GT06000012 | Jan 2008 | MX |
| 178299 | Apr 2000 | PL |
| 130713 | Nov 2015 | RO |
| 1791767 | Jan 1993 | RU |
| 2005102554 | Jul 2006 | RU |
| 2421744 | Jun 2011 | RU |
| 2421744 | Jun 2011 | RU |
| 2447640 | Apr 2012 | RU |
| 2502047 | Dec 2013 | RU |
| 2502047 | Dec 2013 | RU |
| 2529905 | Oct 2014 | RU |
| 164128 | Aug 2016 | RU |
| 2017114139 | Oct 2018 | RU |
| 2017114139 | May 2019 | RU |
| 834514 | May 1981 | SU |
| 887717 | Dec 1981 | SU |
| 1052940 | Nov 1983 | SU |
| 1134669 | Jan 1985 | SU |
| 1526588 | Dec 1989 | SU |
| 1540053 | Jan 1991 | SU |
| 1761864 | Sep 1992 | SU |
| 90246 | Apr 2010 | UA |
| 1986005353 | Sep 1986 | WO |
| 2001052160 | Jul 2001 | WO |
| 2002015673 | Feb 2002 | WO |
| 2003005803 | Jan 2003 | WO |
| 2007050192 | May 2007 | WO |
| 2009156542 | Dec 2009 | WO |
| 2010003421 | Jan 2010 | WO |
| 2011104085 | Sep 2011 | WO |
| 2012041621 | Apr 2012 | WO |
| 2012110508 | Aug 2012 | WO |
| 2012110544 | Aug 2012 | WO |
| 2013063106 | May 2013 | WO |
| 2013079247 | Jun 2013 | WO |
| 2013086351 | Jun 2013 | WO |
| WO2013087275 | Jun 2013 | WO |
| 2013120079 | Aug 2013 | WO |
| 2013141975 | Sep 2013 | WO |
| 2013151619 | Oct 2013 | WO |
| 2013162673 | Oct 2013 | WO |
| 2013184177 | Dec 2013 | WO |
| 2013184178 | Dec 2013 | WO |
| 2014046685 | Mar 2014 | WO |
| 2014093814 | Jun 2014 | WO |
| 2014116819 | Jul 2014 | WO |
| 2014195302 | Dec 2014 | WO |
| 2015038751 | Mar 2015 | WO |
| 2015153809 | Oct 2015 | WO |
| 16020595 | Feb 2016 | WO |
| 2016020595 | Feb 2016 | WO |
| 2016118686 | Jul 2016 | WO |
| 2017008161 | Jan 2017 | WO |
| 2017060168 | Apr 2017 | WO |
| 2017077113 | May 2017 | WO |
| 2017096489 | Jun 2017 | WO |
| 2017099570 | Jun 2017 | WO |
| 2017116913 | Jul 2017 | WO |
| 2017170507 | Oct 2017 | WO |
| 2017205406 | Nov 2017 | WO |
| 2017205410 | Nov 2017 | WO |
| WO2018043336 | Mar 2018 | WO |
| 2018073060 | Apr 2018 | WO |
| 2018081759 | May 2018 | WO |
| 2018112615 | Jun 2018 | WO |
| WO2018112615 | Jun 2018 | WO |
| WO2018116772 | Jun 2018 | WO |
| 2018142768 | Aug 2018 | WO |
| 2018185522 | Oct 2018 | WO |
| 2018200870 | Nov 2018 | WO |
| 2018206587 | Nov 2018 | WO |
| 2018220159 | Dec 2018 | WO |
| 2018226139 | Dec 2018 | WO |
| 2018235486 | Dec 2018 | WO |
| 2018235942 | Dec 2018 | WO |
| WO18235486 | Dec 2018 | WO |
| 2019034213 | Feb 2019 | WO |
| 2019079205 | Apr 2019 | WO |
| 2019081349 | May 2019 | WO |
| 2019091535 | May 2019 | WO |
| 2019109191 | Jun 2019 | WO |
| 2019124174 | Jun 2019 | WO |
| 2019124217 | Jun 2019 | WO |
| 2019124273 | Jun 2019 | WO |
| WO2019124174 | Jun 2019 | WO |
| WO2019124225 | Jun 2019 | WO |
| 2019129333 | Jul 2019 | WO |
| 2019129334 | Jul 2019 | WO |
| 2019129335 | Jul 2019 | WO |
| 2019215185 | Nov 2019 | WO |
| 2019230358 | Dec 2019 | WO |
| 2020039312 | Feb 2020 | WO |
| WO2020026578 | Feb 2020 | WO |
| WO2020026650 | Feb 2020 | WO |
| WO2020026651 | Feb 2020 | WO |
| WO2020031473 | Feb 2020 | WO |
| WO2020038810 | Feb 2020 | WO |
| WO2020039671 | Feb 2020 | WO |
| WO2020044726 | Mar 2020 | WO |
| 2020082182 | Apr 2020 | WO |
| WO202082182 | Apr 2020 | WO |
| WO2020100810 | May 2020 | WO |
| WO2020110920 | Jun 2020 | WO |
| 2020195007 | Oct 2020 | WO |
| 2020206941 | Oct 2020 | WO |
| 2020206942 | Oct 2020 | WO |
| 2020210607 | Oct 2020 | WO |
| WVO2020206942 | Oct 2020 | WO |
| 2020221981 | Nov 2020 | WO |
| WO21049098 | Mar 2021 | WO |
| 2021262500 | Dec 2021 | WO |
| WO21246384 | Dec 2021 | WO |
| 2022118572 | Jun 2022 | WO |
| WO22130038 | Jun 2022 | WO |
| WO22130039 | Jun 2022 | WO |
| WO22268779 | Dec 2022 | WO |
| Entry |
|---|
| Notice of Allowance for U.S. Appl. No. 16/384,425 dated Dec. 13, 2022, 9 pages. |
| Application and Drawings for U.S. Appl. No. 16/171,978, filed Oct. 26, 2018, 53 pages. |
| Notice of Allowance for U.S. Appl. No. 16/432,557 dated Mar. 22, 2021, 9 pages. |
| Zhao, L., Yang, J., Li, P. and Zhang, L., 2014. Characteristics analysis and classification of crop harvest patterns by exploiting high-frequency multipolarization SAR data. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 7(9), pp. 3773-3783. |
| Feng-jie, X., Er-da, W. and Feng-yuan, X., Crop area yield risk evaluation and premium rates calculation—Based on nonparametric kernel density estimation. In 2009 International Conference on Management Science and Engineering, 7 pages. |
| Liu, R. and Bai, X., 2014, May. Random fuzzy production and distribution plan of agricultural products and its PSO algorithm. In 2014 IEEE International Conference on Progress in Informatics and Computing (pp. 32-36). IEEE. |
| Notice of Allowance for U.S. Appl. No. 16/171,978 dated Mar. 31, 2021, 6 pages. |
| Application and Drawings for U.S. Appl. No. 17/067,383, filed Oct. 9, 2020, 61 pages. |
| Final Office Action for U.S. Appl. No. 16/384,425 dated Oct. 21, 2022, 11 pages. |
| 7 Combine Tweaks to Boost Speed (https://www.agriculture.com/machinery/harvest-equipment/7-combine-tweaks-to-boost-speed_203-ar33059) 8 pages, Aug. 19, 2018. |
| Managing corn harvest this fall with variable corn conditions (https://www.ocj.com/2019/10/managing-corn-harvest-this-fall-with-variable-corn-conditions/), 4 pages, Oct. 10, 2019. |
| Reducing Aflatoxin in Corn During Harvest and Storage (https://extension.uga.edu/publications/detail.html?number=B1231&title=Reducing%20Aflatoxin%20in%20Corn%20During%20Harvest%20and%20Storage), 9 pages, Published with Full Review on Apr. 19, 2017. |
| Variable Rate Applications to Optimize Inputs (https://www.cotton.org/tech/physiology/cpt/miscpubs/upload/CPT-v9No2-98-REPOP.pdf), 8 pages, Nov. 2, 1998. |
| Robin Booker, Video: Canadian cage mill teams up with JD (https://www.producer.com/2019/12/video-canadian-cage-mill-teams-up-with-jd/) , 6 pages, Dec. 19, 2019. |
| Jarnevich, et al. “Forecasting Weed Distributions using Climate Data: A GIS Early Warning Tool”, Invasive Plant Science and Management, 11 pages, Jan. 20, 2017. |
| Burks, “Classification of Weed Species Using Color Texture Features and Discriminant Analysis” (http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.468.5833&rep=rep1&type=pdf), 8 pages, 2000. |
| John Deere, https://www.youtube.com/watch?v=1Gq77CfdGI4&list=PL1KGsSJ4CWk4rShNb3-sTMOliL8meHBL5 (last accessed Jul. 14, 2020), Jun. 15, 2020, 5 pages. |
| Combine Adjustments (http://corn.agronomy.wisc.edu/Management/L036.aspx), 2 pages, Originally written Feb. 1, 2006; last updated Oct. 18, 2018. |
| Ardekani, “Off- and on-ground GPR techniques for field-scale soil moisture mapping” Jun. 2013, 13 pages. |
| Does an Adaptive Gearbox Really Learn How You Drive? (https://practicalmotoring.com.au/voices/does-an-adaptive-gearbox-really-learn-how-you-drive/), Oct. 30, 2019, 8 pages. |
| https://www.researchgate.net/publication/222527694_Energy_Requirement_Model_for_a_Combine_Harvester_Part_I_Development_of_Component Models, Abstract Only, Jan. 2005. |
| http://canola.okstate.edu/cropproduction/harvesting, 8 pages, Aug. 2011. |
| “Tips and Tricks of Harvesting High Moisture Grain”, https://www.koenigequipment.com/blog/tips-and-tricks-of-harvesting-highmoisture-grain, 5 pages, last accessed Feb. 11, 2021. |
| Hoff, Combine Adjustements, Mar. 1943, 8 pages. |
| Haung et al., “Accurate Weed Mapping and Prescription Map Generation Based onFully Convolutional Networks Using UAV Imagery”, 14 pages, Oct. 1, 2018. |
| Thompson, “Morning glory can make it impossible to harvest corn”, Feb. 19, 2015, 4 pages. |
| Martin et al. Breakage Susceptibiltly and Hardness of Corn Kernels of Various Sizes and Shapes, vol. 3(): May 1087, 10 pages. https://pdfs.semanticscholar.org/e579/1b5363b6a78efd44adfb9775530cdd1417ca.pdf. |
| Hoff, “Combine Adjustments” (https.//smallfarmersjournal.com/combine-adjustments/), Mar. 1943, 9 pages. |
| Optimizing Crop Profit Across Multiple Grain Attributes and Stover, Electronic Publication Date May 26, 2009, 17 pages. |
| Unglesbee. Soybean Pod Shatter—Bad Enough to Scout Before Harvest—DTN, Oct. 17, 2018, 11 pages. Susceptibility to shatter (https://agfax.com/2018/10/17/soybean-pod-shaffer-bad-enough-to-scout-before-harvest-dtn/). |
| GIS Maps for Agricultural, accessed on May 10, 2022, 7 pages. https://www.satimagingcorp.com/service/geographic-information-systems/gis-maps-agriculture-mapping. |
| https://wingtra.com/drone-mapping-applications/use-of-drones-in-agriculture, accessed on May 10, 2022, 19 pages. |
| Energy Requirement Model for a Combine Harvester: Part 1: Development of Component Models, Published online Dec. 22, 2004, 17 pages. |
| Energy Requirement Model for a Combine Harvester, Part 2: Integration of Component Models, Published online Jan. 18, 2005, 11 pages. |
| Pioneer on reducing soybean harvest losses including combine adjustments (last accessed Jul. 23, 2020) (https://www.pioneer.com/us/agronomy/reducing_harvest_losses_in_soybeans.html), 5 pages. |
| Prosecution History for U.S. Appl. No. 16/384,425 including: Non-Final Office Action dated May 11, 2022, and Application and Drawings filed Apr. 15, 2019, 64 pages. |
| Application and Drawings for U.S. Appl. No. 16/175,993, filed Oct. 31, 2018, 28 pages. |
| Application and Drawings for U.S. Appl. No. 16/380,623, filed Apr. 10, 2019, 36 pages. |
| Application and Drawings for U.S. Appl. No. 16/783,511, filed Feb. 6, 2020, 55 pages. |
| “Automated Weed Detection With Drones” dated May 25, 2017, retrieved at: <<https://www.precisionhawk.com/blog/media/topic/automated-weed-identification-with-drones>>, retrieved on Jan. 21, 2020, 4 pages. |
| F. Forcella, “Estimating the Timing of Weed Emergence”, Site-Specific Management Guidelines, retrieved at: <<http://www.ipni.net/publication/ssmg.nsf/0/D26EC9A906F9B8C9852579E500773936/$FILE/SSMG-20.pdf>>, retrieved on Jan. 21, 2020, 4 pages. |
| Chauhan et al., “Emerging Challenges and Opportunities for Education and Research in Weed Science”, frontiers in Plant Science. Published online Sep. 5, 2017, 22 pages. |
| Apan, A., Wells , N., Reardon-Smith, K, Richardson, L, McDougall, K, and Basnet, B.B., 2008. Predictive mapping of blackberry in the Condamine Catchment using logistic regression and spatial analysis. In Proceedings of the 2008 Queensland Spatial Conference: Global Warning: What's Happening in Paradise. Spatial Sciences Institute, 11 pages. |
| Jarnevich, C.S., Holcombe, T.R., Barnett, D.T., Stohlgren, T.J. and Kartesz, J.T., 2010. Forecasting weed distributions using climate data: a GIS early warning tool. Invasive Plant Science and Management. 3(4), pp. 365-375. |
| Sa et al., “WeedMap: A Large-Scale Semantic Weed Mapping Framework Using Aerial Multispectral Imaging and Deep Neural Network for Precision Farming”, Sep. 6, 2018, 25 pages. |
| Pflanz et al., “Weed Mapping with UAS Imagery and a Bag of Visual Words Based Image Classifier”, Published Sep. 24, 2018, 28 pages. |
| Provisional Application and Drawings for U.S. Appl. No. 62/928,964, filed Oct. 31, 2019, 14 pages. |
| Application and Drawings for U.S. Appl. No. 16/783,475, filed Feb. 6, 2020, 55 pages. |
| U.S. Appl. No. 17/067,483 Application and Drawings as filed on Oct. 9, 2020, 63 pages. |
| U.S. Appl. No. 17/066,442 Application and Drawings as filed on Oct. 8, 2020, 65 pages. |
| U.S. Appl. No. 16/380,550, filed Apr. 10, 2019, Application and Drawings, 47 pages. |
| U.S. Appl. No. 17/066,999 Application and Drawings as filed on Oct. 9, 2020, 67 pages. |
| U.S. Appl. No. 17/066,444 Application and Drawings as filed on Oct. 8, 2020, 102 pages. |
| Extended Search Report for European Patent Application No. 20167930.5 dated Sep. 15, 2020, 8 pages. |
| Extended Search Report for European Patent Application No. 19205901.2 dated Mar. 17, 2020, 6 pages. |
| Notice of Allowance for U.S. Appl. No. 16/171,978, dated Dec. 15, 2020, 21 pages. |
| Zhigen et al., “Research of the Combine Harvester Load Feedback Control System Using Multi-Signal Fusion Method and Fuzzy Algorithm,” 2010, Publisher: IEEE, 5 pages. |
| Dan et al., “On-the-go Throughput Prediction in a Combine Harvester Using Sensor Fusion,” 2017, Publisher: IEEE, 6 pages. |
| Fernandez-Quintanilla et al., “Is the current state of the art of weed monitoring sutible for site-specific weed management in arable crops?”, First Published May 1, 2018, 4 pages. |
| Dionysis Bochtis et al. “Field Operations Planning for Agricultural Vehicles: A Hierarchical Modeling Framework.” Agricultural Engineering International: the CIGR Ejournal. Manuscript PM 06 021. Vol. IX. Feb. 2007, pp. 1-11. |
| U.S. Appl. No. 16/432,557, filed Jun. 5, 2019, 61 pages. |
| European Search Report issued in counterpart European Patent Application No. 19205142.3 dated Feb. 28, 2020 (6 pages). |
| Mei-Ju et al., “Two paradigms in cellular Internet-of-Things access for energy-harvesting machine-to-machine devices: push-based versus pull-based,” 2016, vol. 6, 9 pages. |
| Yi et al., “An Efficient MAC Protocol With Adaptive Energy Harvesting for Machine-to-Machine Networks,” 2015, vol. 3, Publisher: IEEE, 10 pages. |
| European Search Report issued in European Patent Application No. 19203883.4 dated Mar. 23, 2020 (10 pages). |
| Notice of Allowance for U.S. Appl. No. 16/171,978 dated Oct. 28, 2020, 5 pages. |
| Notice of Allowance for U.S. Appl. No. 16/171,978, dated Aug. 7, 2020, 9 pages. |
| K.R. Manjunath et al., “Developing Spectral Library of Major Plant Species of Western Himalayas Using Ground Observations”, J. Indian Soc Remote Sen (Mar. 2014) 42(a):201-216, 17 pages. |
| U.S. Appl. No. 16/380,564 Application and Drawings as filed on Apr. 10, 2019, 55 pages. |
| S. Veenadhari et al., “Machine Learning Approach For Forecasting Crop Yield Based on Climatic Parameters”, 2014 International Conference on Computer Communication and Informatics (ICCCI-2014) Jan. 3-6, 2014, Coimbatore, India, 5 pages. |
| Non-Final Office Action for U.S. Appl. No. 16/380,531 dated Oct. 21, 2020, 10 pages. |
| U.S. Appl. No. 16/380,531 Application and Drawings as filed on Apr. 10, 2019, 46 pages. |
| Leu et al., Grazing Corn Residue Using Resources and Reducing Costs, Aug. 2009, 4 pages. |
| “No-Till Soils”, Soil Heath Brochure, 2 pages, last accessed Jul. 14, 2020. |
| Strickland et al., “Nitrate Toxicity in Livestock” Oklahoma State University, Feb. 2017, 2 pages. |
| Strickland et al., “Nitrate Toxicity in Livestock” Oklahoma State University, 8 pages, Feb. 2017. |
| Brownlee, “Neural Networks are Function Approximation Algorithms”, Mar. 18, 2020, 13 pages. |
| Thompson, “Morning glory can make it impossible to harvest corn”, Feb. 19, 2015, 3 pages. |
| Tumlison, “Monitoring Growth Development and Yield Estimation of Maize Using Very High-Resolution Uavimages in Gronau, Germany”, Feb. 2017, 63 pages. |
| Hunt, “Mapping Weed Infestations Using Remote Sensing”, 8 pages, Jul. 19, 2005. |
| Wright, et al., “Managing Grain Protein in Wheat Using Remote Sensing”, 12 pages, 2003. |
| “Malting Barley in Pennsylvania”, Agronomy Facts 77, 6 pages, Code EE0179 Jun. 2016. |
| “Green stem syndrome in soybeans”, Agronomy eUpdate Issue 478 Oct. 10, 2014, 3 pages. |
| “Keep Weed Seed Out of Your Harvest”, Aug. 8, 2019, 1 pages. |
| Hodrius et al., “The Impact of Multi-Sensor Data Assimilation on Plant Parameter Retrieval and Yield Estimation for Sugar Beet”, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, vol. XL-7/W3, 2015, 36th International Symposium on Remote Sensing of Environment, May 11-15, 2015, Berlin, Germany, 7 pages. |
| Fernandez-Quintanilla et al., “Is the current state of the art of weed monitoring suitable for site-specific weed management in arable crops?”, Feb. 2018, 35 pages. |
| Anonymously, “Improved System and Method for Controlling Agricultural Vehicle Operation Using Historical Data”, Dec. 16, 2009, 8 pages. |
| Anonymously, “System and Method for Controlling Agricultural Vehicle Operation Using Historical Data”, Jun. 30, 2009, 8 pages. |
| “Leafsnap, a new mobile app that identifies plants by leaf shape, is launched by Smithsonian and collaborators”, May 2, 2011, 5 pages. |
| Insect Gallery, Department of Entomology, Kansas State University, Oct. 19, 2020, 8 pages. |
| Licht, “Influence of Corn Seeding Rate, Soil Attributes, and Topographic Characteristics on Grain Yield, Yield Components, and Grain Composition”, 2015, 107 pages,. |
| “Notice of Retraction Virtual simulation of plant with individual stem based on crop growth model”, Mar. 5, 2017, 7 pages. |
| Ma et al., “Identification of Fusarium Head Blight in Winter Wheat Ears Using Continuous Wavelet Analysis”, Dec. 19, 2019, 15 pages. |
| Leland, “Who Did that? Identifying Insect Damage”, Apr. 1, 2015, 4 pages. |
| “How to improve maize protein content” https://www.yara.co.uk/crop-nutrition/forage-maize/improving-maize-protein-content, Sep. 30, 2020, 10 pages. |
| Hafemeister, “Weed control at harvest, combines are ideal vehicles for spreading weed seeds”, Sep. 25, 2019, 3 pages. |
| “Harvesting Tips”, Northern Pulse Growers Association, 9 pages, Jan. 31, 2001. |
| Wortmann et al., “Harvesting Crop Residues”, Aug. 10, 2020, 8 pages. |
| “Harvesting”, Oklahoma State University, Canola Swathing Guide, 2010, 9 pages, last accessed Jul. 14, 2020. |
| Hanna, “Harvest Tips for Lodged Corn”, Sep. 6, 2011, 3 pages. |
| “Green Weeds Complicate Harvest”, Crops, Slider, Sep. 26, 2012, 2 pages. |
| “Agrowatch Green Vegetation Index”, Retrieved Dec. 11, 2020, 4 pages. |
| “Grazing Corn Residues” (http://www.ca.uky.edu), 3 pages, Aug. 24, 2009. |
| Jarnevich et al., Forecasting Weed Distributions Using Climate Data: A GIS Early Warning Tool, Downloaded on Jul. 13, 2020, 12 pages. |
| Combine Cutting and Feeding Mechanisms in the Southeast, By J-K Park, Agricultural Research Service, U.S. Dept. of Agriculture, 1963, 1 page. |
| Hartzler, “Fate of weed seeds in the soil”, 4 pages, Jan. 31, 2001. |
| Digman, “Combine Considerations for a Wet Corn Harvest”, Extension SpecialistUW—Madison, 3 pages, Oct. 29, 2009. |
| S-Series Combine and Front End Equipment Optimization, John Deere Harvester Works, 20 pages Date: Oct. 9, 2017. |
| Determining yield monitoring system delay time with geostatistical and data segmentation approaches (https://www.ars.usda.gov/ARSUserFiles/50701000/cswq-0036-128359.pdf) Jul. 2002, 13 pages. |
| Precision Agriculture: Yield Monitors (dated Nov. 1998—metadata; last accessed Jul. 16, 2020) (https://extensiondata.missouri.edu/pub/pdf/envqual/wq0451.pdf) 4 pages. |
| Paul et al., “Effect of soil water status and strength on trafficability” (1979) (https://www.nrcresearchpress.com/doi/pdfplus/10.4141/cjss79-035), 12 pages, Apr. 23, 1979. |
| Sick, “Better understanding corn hybrid characteristics and properties can impact your seed decisions” (https://emergence.fbn.com/agronomy/corn-hybrid-characteristics-and-properties-impact-seed-decisions) By Steve Sick, FBN Breeding Project Lead | Sep. 21, 2018, 8 pages. |
| Robertson et al., “Maize Stalk Lodging: Morphological Determinants of Stalk Strength” Mar. 2017, 10 pages. |
| Martin, et al., “Breakage Susceptibility and Hardness of Corn Kernels of Various Sizes and Shapes”, May 1987, 10 Pages. |
| Lamsal et al. “Sugarcane Harvest Logistics in Brazil” Iowa Research Online, Sep. 11, 2013, 27 pages. |
| Jensen, “Algorithms for Operational Planning of Agricultural Field Operations”, Mechanical Engineering Technical Report ME-TR-3, Nov. 9, 2012, 23 pages. |
| Chauhan, “Remote Sensing of Crop Lodging”, Nov. 16, 2020, 16 pages. |
| Extended European Search Report and Written Opinion issued in European Patent Application No. 20208171.7, dated May 11, 2021, in 05 pages. |
| Cordoba, M.A., Bruno, C.I. Costa, J.L. Peralta, N.R. and Balzarini, M.G., 2016, Protocol for multivariate homegeneous zone delineation in precision agriculture, biosystems engineering, 143, pp. 95-107. |
| Pioneer Estimator, “Corn Yield Estimator” accessed on Feb. 13, 2018, 1 page. retrieved from: https://www.pioneer.com/home/site/us/tools-apps/growing-tools/corn-yield-estimator/. |
| Guindin, N. “Estimating Maize Grain Yield From Crop Biophysical Parameters Using Remote Sensing”, Nov. 4, 2013, 19 pages. |
| EP Application No. 19203883.4-1004 Office Action dated May 3, 2021, 4 pages. |
| Iowa State University Extension and Outreach, “Harvest Weed Seed Control”, Dec. 13, 2018, 6 pages. https://crops.extension.iastate.edu/blog/bob-hartzler/harvest-weed-seed-control. |
| Getting Rid Of WeedsThrough Integrated Weed Management, accessed on Jun. 25, 2021, 10 pages. https://integratedweedmanagement.org/index.php/iwm-toolbox/the-harrington-seed-destructor. |
| The Importance of Reducing Weed Seeds, Jul. 2018, 2 pages. https://www.aphis.usda.gov/plant_health/soybeans/soybean-handouts.pdf. |
| Alternative Crop Guide, Published by the Jefferson Institute, “Buckwheat”, Revised Jul. 2002. 4 pages. |
| Martin et al., “Breakage Susceptibility and Harness of Corn Kernels of Various Sizes and Shapes”, May 1987, 10 pages. |
| Jones et al., “Brief history of agricultural systems modeling” Jun. 21, 2016, 15 pages. |
| Dan Anderson, “Brief history of agricultural systems modeling” 1 pages. Aug. 13, 2019. |
| A.Y. Şeflek, “Determining the Physico-Mechanical Characteristics of Maize Stalks Fordesigning Harvester”, The Journal of Animal & Plant Sciences, 27(3): 2017, p. 855-860 ISSN: 1018-7081, Jun. 1, 2017. |
| Carmody, Paul, “Windrowing and harvesting”, 8 pages Date: Feb. 3, 2010. |
| Dabney, et al., “Forage Harvest Representation in RUSLE2”, Published Nov. 15, 2013, 17 pages. |
| John Deere S-Series Combines S760, S770, S780, S790 Brochure, 44 pages, Nov. 15, 2017. |
| Sekhon et al., “Stalk Bending Strength is Strongly Assoicated with Maize Stalk Lodging Incidence Across Multiple Environments”, Jun. 20, 2019, 23 pages. |
| Thomison et al. “Abnormal Corn Ears”, Apr. 28, 2015, 1 page. |
| Anderson, “Adjust your Combine to Reduce Damage to High Moisture Corn”, Aug. 13, 2019, 11 pages. |
| Sumner et al., “Reducing Aflatoxin in Corn During Harvest and Storage”, Reviewed by John Worley, Apr. 2017, 6 pages. |
| Sick, “Better understanding corn hybrid characteristics and properties can impact your seed decisions”, 8 pages, Sep. 21, 2018. |
| TraCI/Change Vehicle State—SUMO Documentation, 10 pages, Retrieved Dec. 11, 2020. |
| Arnold, et al., Chapter 8. “Plant Growth Component”, Jul. 1995, 41 pages. |
| Humburg, Chapter: 37 “Combine Adjustments to Reduce Harvest Losses”, 2019, South Dakota Board of Regents, 8 pages. |
| Hoff, “Combine Adjustments”, Cornell Extension Bulletin 591, Mar. 1943, 10 pages. |
| University of Wisconsin, Corn Agronomy, Originally written Feb. 1, 2006 | Last updated Oct. 18, 2018, 2 pages. |
| University of Nebraska-Lincoln, “Combine Adjustments for Downed Corn—Crop Watch”, Oct. 27, 2017, 5 pages. |
| “Combine Cleaning: Quick Guide To Removing Resistant Weed Seeds (Among Other Things)”, Nov. 2006, 5 pages. |
| Dekalb, “Corn Drydown Rates”, 7 pages, Aug. 4, 2020. |
| Mahmoud et al. Iowa State University, “Corn Ear Orientation Effects on Mechanical Damage and Forces on Concave”, 1975, 6 pages. |
| Sindelar et al., Kansas State University, “Corn Growth & Development” Jul. 17, 2017, 9 pages. |
| Pannar, “Manage the Growth Stages of the Maize Plant With Pannar”, Nov. 14, 2016, 7 pages. |
| He et al., “Crop residue harvest impacts wind erodibility and simulated soil loss in the Central Great Plains”, Sep. 27, 2017, 14 pages. |
| Blanken, “Designing a Living Snow Fence for Snow Drift Control”, Jan. 17, 2018, 9 pages. |
| Jean, “Drones give aerial boost to ag producers”, Mar. 21, 2019, 4 pages. |
| Zhao et al., “Dynamics modeling for sugarcane sucrose estimation using time series satellite imagery”, Jul. 27, 2017, 11 pages. |
| Brady, “Effects of Cropland Conservation Practices on Fish and Wildlife Habitat”, Sep. 1, 2007, 15 pages. |
| Jasa, et al., “Equipment Adjustments for Harvesting Soybeans at 13%-15% Moisture”, Sep. 15, 2017, 2 pages. |
| Bendig et al., “Estimating Biomass of Barley Using Crop Surface Models (CSMs) Derived from UAV-Based RGB Imaging”, Oct. 21, 2014, 18 pages. |
| Robertson, et al., “Maize Stalk Lodging: Morphological Determinants of Stalk Strength”, Mar. 3, 2017, 10 pages. |
| MacGowan et al. Purdue University, Corn and Soybean Crop Deprediation by Wildlife, Jun. 2006, 14 pages. |
| Martinez-Feria et al., Iowa State University, “Corn Grain Dry Down in Field From Maturity to Harvest”, Sep. 20, 2017, 3 pages. |
| Wrona, “Precision Agriculture's Value” Cotton Physiology Today, vol. 9, No. 2, 1998, 8 pages. |
| Zhang et al., “Design of an Optical Weed Sensor Using Plant Spectral Characteristics” Sep. 2000, 12 pages. |
| Hunt, et al., “What Weeds Can Be Remotely Sensed?”, 5 pages, May 2016. |
| Pepper, “Does An Adaptive Gearbox Really Learn How You Drive?”, Oct. 30, 2019, 8 pages. |
| Eggerl, “Optimization of Combine Processes Using Expert Knowledge and Methods of Artificial Intelligence”, Oct. 7, 1982, 143 pages. |
| Sheely et al., “Image-Based, Variable Rate Plant Growth Regulator Application in Cotton at Sheely Farms in California”, Jan. 6-10, 2003 Beltwide Cotton Conferences, Nashville, TN, 17 pages. |
| Kovacs et al., “Physical characteristics and mechanical behaviour of maize stalks for machine development”, Apr. 23, 2019, 1-pages. |
| Anonymously, “Optimizing Crop Profit Across Multiple Grain Attributes and Stover”, ip.com, May 26, 2009, 17 pages. |
| Breen, “Plant Identification: Examining Leaves”, Oregon State University, 2020, 8 pages. |
| Caglayan et al., A Plant Recognition Approach Using Shape and Color Features in Leaf Images, Sep. 2013, 11 pages. |
| Casady et al., “Precision Agriculture” Yield Monitors University of Missouri-System, 4 pages, 1998. |
| Apan et al., “Predicting Grain Protein Content in Wheat Using Hyperspectral Sensing of In-season Crop Canopies and Partial Least Squares Regression” 18 pages, 2006. |
| Xu et al., “Prediction of Wheat Grain Protein by Coupling Multisource Remote Sensing Imagery and ECMWF Data”, Apr. 24, 2020, 21 pages. |
| Day, “Probability Distributions of Field Crop Yields,” American Journal of Agricultural Economics, vol. 47, Issue 3, Aug. 1965, Abstract Only, 1 page. |
| Butzen, “Reducing Harvest Losses in Soybeans”, Pioneer, Jul. 23, 2020, 3 pages. |
| Martin et al., “Relationship between secondary variables and soybean oil and protein concentration”, Abstract Only, 1 page., 2007. |
| Torres, “Precision Planting of Maize” Dec. 2012, 123 pages. |
| Apan et al., “Predictive Mapping of Blackberry in the Condamine Catchment Using Logistic Regressiona dn Spatial Analysis”, Jan. 2008, 12 pages. |
| Robson, “Remote Sensing Applications for the Determination of Yield, Maturity and Aflatoxin Contamination in Peanut”, Oct. 2007, 275 pages. |
| Bhattarai et al., “Remote Sensing Data to Detect Hessian Fly Infestation in Commercial Wheat Fields”, Apr. 16, 2019, 8 pages. |
| Towery, et al., “Remote Sensing of Crop Hail Damage”, Jul. 21, 1975, 31 pages. |
| Sa et al., “WeedMap: A Large-Scale Semantic Weed Mapping Framework Using Aerial Multispectral Imaging and Deep Neural Network for Precision Farming”, Sep. 7, 2018, 25 pages. |
| Mathyam et al., “Remote Sensing of Biotic Stress in Crop Plants and Its Applications for Pest Management”, Dec. 2011, 30 pages. |
| Martinez-Feria et al., “Evaluating Maize and Soybean Grain Dry-Down In The Field With Predictive Algorithms and Genotype-by-Environmental Analysis”, May 9, 2019, 13 pages. |
| “GIS Maps for Agriculture”, Precision Agricultural Mapping, Retrieved Dec. 11, 2020, 6 pages. |
| Paul, “Scabby Wheat Grain? Increasing Your Fan Speed May Help”, https://agcrops.osu.edu/newsletter/corn-newsletter/2015-20/scabby-wheat-grain-increasing-yourfan-speed-may-help, C.O.R.N Newsletter//2015-20, 3 pages. |
| Clay et al., “Scouting for Weeds”, SSMG-15, 4 page, 2002. |
| Taylor et al., “Sensor-Based Variable Rate Application for Cotton”, 8 pages, 2010. |
| Christiansen et al., “Designing and Testing a UAV Mapping System for Agricultural Field Surveying”, Nov. 23, 2017, 19 pages. |
| Haung et al., “AccurateWeed Mapping and Prescription Map Generation Based on Fully Convolutional Networks Using UAV Imagery”, Oct. 1, 2018, 12 pages. |
| Morrison, “Should You Use Tillage to Control Resistant Weeds”, Aug. 29, 2014, 9 pages. |
| Morrison, “Snow Trapping Snars Water”, Oct. 13, 2005, 3 pages. |
| “Soil Zone Index”, https://www.satimagingcorp.com/applications/natural-resources/agricultu . . . , Retrieved Dec. 11, 2020, 5 pages. |
| Malvic, “Soybean Cyst Nematode”, University of Minnesota Extension, Oct. 19, 2020, 3 pages. |
| Unglesbee, “Soybean Pod Shatter—Bad Enough to Scout Before Harvest?—DTN”, Oct. 17, 2018, 4 pages. |
| Tao, “Standing Crop Residue Can Reduce Snow Drifting and Increase Soil Moisture”, 2 pages, last accessed Jul. 14, 2020. |
| Berglund, et al., “Swathing and Harvesting Canola”, Jul. 2019, 8 pages. |
| Bell et al., “Synthetic Aperture Radar and Optical Remote Sensing of Crop Damage Attributed To Severe Weather in the Central United States”, Jul. 25, 2018, 1 page. |
| Rosencrance, “Tabletop Grapes in India to Be Picked by Virginia Tech Robots”, Jul. 23, 2020, 8 pages. |
| Lofton, et al., The Potential of Grazing Grain Sorghum Residue Following Harvest, May 13, 2020, 11 pages. |
| Beal et al., “Time Shift Evaluation to Improve Yield Map Quality”, Published in Applied Engineering in Agriculture vol. 17(3): 385-390 (@ 2001 American Society of Agricultural Engineers ), 9 pages. |
| “Tips and Tricks of Harvesting High Moisture Grain”, https://www.koenigequipment.com/blog/tips-and-tricks-of-harvesting-highmoisture-grain, 7 pages, last accessed Jul. 14, 2020. |
| Ransom, “Tips for Planting Winter Wheat and Winter Rye (for Grain) (Aug. 15, 2019)”, 2017, 3 pages. |
| AgroWatch Tree Grading Maps, “The Grading Maps and Plant Count Reports”, https://www.satimagingcorp.com/applications/natural-resources/agricultu . . . , Retrieved Dec. 11, 2020, 4 pages. |
| Ackley, “Troubleshooting Abnormal Corn Ears”, Jul. 23, 2020, 25 pages. |
| Smith, “Understanding Ear Flex”, Feb. 25, 2019, 17 pages. |
| Carroll et al., “Use of Spectral Vegetation Indicies Derived from Airborne Hyperspectral Imagery For Detection of European Corn Borer Infestation in lowa Corn Plots”, Nov. 2008, 11 pages. |
| Agriculture, “Using drones in agriculture and capturing actionable data”, Retrieved Dec. 11, 2020, 18 pages. |
| Bentley et al., “Using Landsat to Identify Thunderstorm Damage in Agricultural Regions”, Aug. 28, 2001, 14 pages. |
| Duane Grant and the Idaho Wheat Commission, “Using Remote Sensing to Manage Wheat Grain Protein”, Jan. 2, 2003, 13 pages. |
| Zhang et al., “Using satellite multispectral imagery for damage mapping of armyworm (Spodoptera frugiperda) in maize at a regional scale”, Apr. 10, 2015, 14 pages. |
| Booker, “Video: Canadian cage mill teams up with JD”, Dec. 19, 2019, 6 pages. |
| AgTalk Home, “Best Combine to Handle Weeds”, Posted Nov. 23, 2018, 9 pages. |
| “Volunteer corn can be costly for soybeans”, Jun. 2, 2016, 1 page. |
| Pflanz, et al., “Weed Mapping with UAS Imagery and a Bag of Visual Words Based Image Classifier”, Published Sep. 24, 2018, 17 pages. |
| Hartzler, “Weed seed predation in agricultural fields”, 9 pages, 2009. |
| Sa et al., “Weedmap: A Large-Scale Sematnic Weed Mapping Framework Using Aerial Multispectral Imaging and Deep Neural Netowrk for Precision Farming”, Sep. 6, 2018, 25 pages. |
| Nagelkirk, Michigan State University-Extension, “Wheat Harvest: Minimizing the Risk of Fusarium Head Scab Losses”, Jul. 11, 2013, 4 pages. |
| Saskatchewan, “Wheat: Winter Wheat”, (https://www.saskatchewan.ca/business/agriculture-natural-resources-and-industry/agribusiness-farmers-and-ranchers/crops-and-irrigation/field-crops/cereals-barley-wheat-oats-triticale/wheat-winter-wheat) 5 pages, last accessed Jul. 14, 2020. |
| Quora, “Why would I ever use sport mode in my automatic transmission car? Will this incrase fuel efficiency or isit simply a feature that makes form more fun when driving?”, Aug. 10, 2020, 5 pages. |
| Wade, “Using a Drone's Surface Model to Estimate Crop Yields & Assess Plant Health”, Oct. 19, 2015, 14 pages. |
| Mathyam et al., “Remote Sensing of Biotic Stress in Crop Plants and Its Applications for Pest Stress”, Dec. 2011, 30 pages. |
| “Four Helpful Weed-Management Tips for Harvest Time”, 2 pages, Sep. 4, 2019. |
| Franz et al., “The role of topography, soil, and remotely sensed vegetation condition towards predicting crop yield”, University of Nebraska—Lincoln, Mar. 23, 2020, 44 pages. |
| Peiffer et al., The Genetic Architecture of Maize Stalk Strength:, Jun. 20, 2013, 14 pages. |
| Extended European Search Report and Written Opinion issued in European Patent Application No. 22198062.6 (which is foreign counterpart application to U.S. Appl. No. 17/584,966), dated Jun. 27, 2023, in 06 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20230255143 A1 | Aug 2023 | US |
