Patent Grant
10198806
This invention relates to inspecting plants for contamination, and more particularly to preventing contact between contaminated plants and harvesting equipment.
Harvesting leafy vegetable plants typically involves moving a harvester (e.g., an automated harvester) across a field under the guidance of an operator steering the harvester and visually inspecting plants that are harvested from the field for contamination by other personnel riding on the harvester. Sunlight and warm temperatures during the daytime can cause some leafy vegetable plants (e.g., baby green vegetable plants) to become tender and pliable and to wilt, rendering a stature of the plants as less than optimal for cutting by a harvester. Accordingly, harvesting operations are often performed on some leafy vegetable plants at night, when the plants are stiffer and tend to stand taller and more erect. As a harvester moves along a field during a harvesting operation, an operator looks ahead of the harvester to scan the field for contaminated plants. In some cases, personnel may walk the field before the harvesting operation (e.g., during the daytime) to search for contaminated plants. The personnel may mark the contaminated plants (e.g., with flags or other markers) so that the operator is alerted to the contaminated plants during a subsequent harvesting operation.
If the operator of the harvester recognizes contaminated plants, then the operator may attempt to avoid harvesting the contaminated plants by steering the harvester around the contaminated plants to prevent the cutter from contacting the contaminated plants. Limited visibility in the dark can sometimes result in contaminated plants being overlooked or in contaminated plants being identified too slowly, such that the contaminated plants are cut by the harvester. If the operator or the personnel riding on the harvester discover that contaminated plants have been harvested, then the contaminated plants are discarded and the harvester has to be shut down and decontaminated (e.g., disinfected or sterilized) before harvesting can resume. Similarly, if contaminated plants are discovered at a processing plant (e.g., post-harvest), then the contaminated plants and all in-process plants are discarded and a processing line has to be shut down and decontaminated. Such contamination incidents occurring at a harvester or at a processing line can result in costly expenses and significant dangers to food safety.
The invention involves a realization that improvements in inspecting plants (e.g., leafy vegetable plants) for contamination in an automated manner can improve a yield of a harvesting operation by preventing downtime of harvesting machinery and reducing product rejection (e.g., preventing contamination of previously harvested plants). Such automated inspections advantageously provide large amounts of accurate information that successfully identifies contaminated plants in real time based on image analyses. A plant inspection system configured to perform such inspections can allow a field to be harvested efficiently and during periods of limited visibility (e.g., at night) while avoiding dangerous contamination of previously harvested plants such that food safety is maintained and while avoiding costly shutdowns and decontaminations of a harvester or of a downstream (e.g., post-harvest) processing line (e.g., at a processing plant).
One aspect of the invention features a method of inspecting plants for contamination. The method includes generating a first series of images of a plant disposed along a planting bed using a camera mounted to a frame being moved along the planting bed by a harvester, identifying a region of interest displayed in the first series of images from feature boundary data defined by color regions associated with the first series of images, comparing a color parameter of the region of interest to a color criterion associated with a type of contamination, comparing a morphological parameter of the region of interest to a reference parameter associated with the type of contamination, and upon determining that the color parameter meets the color criterion and that the morphological parameter sufficiently matches the reference parameter, identifying the region of interest as a region of contamination on a surface of the plant. The method further includes transmitting data including an instruction to lift a cutter of the harvester up from the planting bed to avoid harvesting the plant in response to identifying the region of interest as the region of contamination, and while the cutter of the harvester is being lifted up from the planting bed, generating a second series of images of an additional plant disposed along the planting bed using the camera as the frame continues to be moved along the planting bed by the harvester.
In some embodiments, the camera is a portion of a machine vision system that is operable to analyze the first and second series of images.
In certain embodiments, the machine vision system is operable to generate first and second series of machine vision views respectively associated with the first and second series of images.
In some embodiments, the camera is located forward of the harvester such that the camera generates images of plants disposed along the planting bed before the plants can be severed from the planting bed by the cutter.
In certain embodiments, the method further includes maintaining a predetermined vertical distance between the camera and the planting bed.
In some embodiments, the method further includes illuminating the plant using one or more lights associated with the camera while the first and second series of images are generated.
In certain embodiments, the method further includes blocking environmental light from impinging upon the plant and the additional plant while the first and second series of images are generated, respectively.
In some embodiments, the method further includes identifying the region of interest using a blob analysis.
In certain embodiments, the blob analysis identifies portions of the region of interest that share one or more colors and determines a border around the portions.
In some embodiments, the morphological parameter is a shape of the region of interest.
In certain embodiments, the morphological parameter is a size of the region of interest.
In some embodiments, the type of contamination includes fecal matter, disease, rodents, insects, or foreign matter.
In certain embodiments, the plant is a leafy vegetable plant.
In some embodiments, the leafy vegetable plant is a baby green vegetable plant.
In certain embodiments, the leafy vegetable plant is a mature green vegetable plant.
In some embodiments, the method further includes storing a first series of machine vision views associated with the first series of images in association with position coordinates of the region of contamination in response to identifying the region of interest as the region of contamination.
In certain embodiments, the method further includes storing a record of a lifting action taken by the cutter to avoid harvesting the plant.
In some embodiments, the method further includes generating a report including information related to an identification of the region of interest as the region of contamination.
In certain embodiments, the method further includes determining whether or not the cutter contacted the plant based on a height to which the cutter was lifted and an amount of time elapsed between the data being transmitted and the cutter being lifted.
In some embodiments, the data is transmitted wirelessly.
In certain embodiments, the instruction includes an actuation of a light that can alert an operator of the harvester.
In some embodiments, the instruction includes a message displayed on a computing device associated with the harvester.
In certain embodiments, the instruction includes a control signal to automatically lift the cutter.
In some embodiments, the method further includes transmitting data including a subsequent instruction to lower the cutter of the harvester towards the planting bed to continue harvesting plants from the planting bed after the cutter has been lifted up from the planting bed.
In certain embodiments, the method further includes transmitting data including a subsequent instruction to cease a harvesting operation and to decontaminate one or more portions of the harvester.
In some embodiments, the method further includes moving the plant to expose hidden regions of contamination on the surface of the plant while the first and second series of images are generated.
In certain embodiments, moving the plant includes moving the plant forward and backward with a flexible comb.
In some embodiments, moving the plant includes blowing air toward the plant.
In certain embodiments, the method further includes generating a third series of images of the additional plant using an additional camera mounted to the harvester after the additional plant has been harvested by the cutter and transported to a conveyor belt.
Another aspect of the invention features a plant inspection system that includes a camera that is configured to generate a first series images of a plant disposed along a planting bed, a processor that is associated with the camera, and a controller that is communicably coupled to the processor. The camera is mounted to a frame being moved along the planting bed by a harvester. The processor is configured to identify a region of interest displayed in the first series of images from feature boundary data defined by color regions associated with the first series of images, compare a color parameter of the region of interest to a color criterion associated with a type of contamination, compare a morphological parameter of the region of interest to a reference parameter associated with the type of contamination, and upon determining that the color parameter meets the color criterion and that the morphological parameter sufficiently matches the reference parameter, identify the region of interest as a region of contamination on a surface of the plant. The controller is configured to transmit data including an instruction to lift a cutter of the harvester up from the planting bed to avoid harvesting the plant in response to the processor identifying the region of interest as the region of contamination while the camera generates a second series of images of an additional plant disposed along the planting bed as the frame continues to be moved along the planting bed by the harvester.
Another aspect of the invention features a method of inspecting plants for contamination. The method includes generating a first series of images of a plant disposed along a planting bed using a camera mounted to a frame being moved along the planting bed by a harvester, identifying a region of interest displayed in the first series of images as a region of contamination on the plant based on a color criterion and a morphological criterion applied to the region of interest, and transmitting data including an instruction to increase a vertical distance between the plant and a cutter of the harvester to avoid harvesting the plant in response to identifying the region of interest as the region of contamination. The method further includes generating a second series of images of an additional plant disposed along the planting bed using the camera as the frame continues to be moved along the planting bed by the harvester while the vertical distance between the plant and the cutter is being increased.
In some embodiments, the camera is a portion of a machine vision system that is operable to analyze the first and second series of images.
In certain embodiments, the machine vision system is operable to generate first and second series of machine vision views respectively associated with the first and second series of images.
In some embodiments, the camera is located forward of the harvester such that the camera generates images of plants disposed along the planting bed before the plants can be severed from the planting bed by the cutter.
In certain embodiments, the method further includes maintaining a predetermined vertical distance between the camera and the planting bed.
In some embodiments, the method further includes illuminating the plant using one or more lights associated with the camera while the first and second series of images are generated.
In certain embodiments, the method further includes blocking environmental light from impinging upon the plant and the additional plant while the first and second series of images are generated, respectively.
In some embodiments, identifying the region of interest includes performing a blob analysis that identifies portions of the region of interest that share one or more colors and that determines a border around the portions.
In certain embodiments, the morphological criterion is related to a shape or a size of the region of interest.
In some embodiments, the type of contamination includes fecal matter, disease, rodents, insects, or foreign matter.
In certain embodiments, the plant is a leafy vegetable plant.
In some embodiments, the leafy vegetable plant is a baby green vegetable plant or a mature green vegetable plant.
In certain embodiments, the method further includes storing a first series of machine vision views associated with the first series of images in association with position coordinates of the region of contamination in response to identifying the region of interest as the region of contamination.
In some embodiments, the method further includes selectively actuating a deflection plate of multiple deflection plates to force the plant down towards the planting bed based on the instruction.
In certain embodiments, selectively actuating the deflection plate includes rotating the deflection plate down towards the plant.
In some embodiments, the method further includes flattening the plant atop the planting bed.
In certain embodiments, the method further includes pushing the plant down into the planting bed.
In some embodiments, the method further includes harvesting a non-contaminated plant adjacent to the plant while the vertical distance between the plant and the cutter is being increased.
In certain embodiments, the plant is a first contaminated plant, and the method further includes increasing a vertical distance between the cutter and a second contaminated plant while the vertical distance is being increased between the cutter and the first contaminated plant.
In some embodiments, the instruction indicates that the cutter should be lifted up from the planting bed.
In certain embodiments, the method further includes storing a record of an action taken to increase the distance between the plant and the cutter.
In some embodiments, the method further includes generating a report including information related to an identification of the region of interest as the region of contamination.
In certain embodiments, the method further includes determining whether or not the cutter contacted the plant.
In some embodiments, the data is transmitted wirelessly.
In certain embodiments, the instruction is associated with an actuation of a light that can alert an operator of the harvester.
In some embodiments, the instruction includes a message displayed on a computing device associated with the harvester.
In certain embodiments, the instruction includes a control signal to automatically perform an action to increase the distance between the plant and the cutter.
In some embodiments, the method further includes transmitting data including a subsequent instruction to cease a harvesting operation and to decontaminate one or more portions of the harvester.
In certain embodiments, the method further includes moving the plant to expose hidden regions of contamination on the plant while the first and second series of images are generated.
Another aspect of the invention features a plant inspection system that includes a camera that is configured to generate a first series images of a plant disposed along a planting bed, where the camera us mounted to a frame being moved along the planting bed by a harvester. The plant inspection system further includes a processor that is associated with the camera and configured to identifying a region of interest displayed in the first series of images as a region of contamination on the plant based on a color criterion and a morphological criterion applied to the region of interest. The plant inspection system further includes a controller that is communicably coupled to the processor and configured to transmit data including an instruction to increase a vertical distance between the plant and a cutter of the harvester to avoid harvesting the plant in response to the processor identifying the region of interest as the region of contamination, while the camera generates a second series of images of an additional plant disposed along the planting bed as the frame continues to be moved along the planting bed by the harvester.
Another aspect of the invention features a method of inspecting plants for contamination. The method includes generating a first series of images of a plant disposed along a planting bed using a camera mounted to a frame being moved along the planting bed by a vehicle, identifying a region of interest displayed in the first series of images as a region of contamination on the plant based on a color criterion and a morphological criterion applied to the region of interest, and selectively actuating a plant eradication device of multiple plant eradication devices to remove the plant from a surface of the planting bed in response to identifying the region of interest as the region of contamination. The method further includes generating a second series of images of an additional plant disposed along the planting bed using the camera as the frame continues to be moved along the planting bed by the vehicle while the plant eradication device is being selectively actuated.
In some embodiments, the plant eradication device includes a deflection plate.
In certain embodiments, selectively actuating the plant eradication device includes rotating the deflection plate down towards the plant to force the plant down into the planting bed based on the instruction.
In some embodiments, the plant eradication device includes a cutting blade and a rotating carrier.
In certain embodiments, the rotating carrier includes a paddle wheel.
In some embodiments, the rotating carrier includes a conveyor.
In certain embodiments, selectively actuating the plant eradication device includes severing the plant with the cutting blade and transporting the plant away from a location at which the plant is growing with the rotating carrier.
In some embodiments, the method further includes conveying the plant away from the planting bed in a first direction perpendicular to a second direction in which the frame is being moved along the planting bed by the vehicle.
In certain embodiments, the method further includes harvesting a non-contaminated plant adjacent to the plant while the plant is being removed from the surface of the planting bed.
In some embodiments, the plant is a first contaminated plant, the plant eradication device is a first plant eradication device, and the method further includes selectively actuating a second plant eradication device of the multiple plant eradication devices to remove the second contaminated plant from the surface of the planting bed while the first plant eradication device is being selectively actuated.
Another aspect of the invention features a plant inspection system that includes a camera that is configured to generate a first series images of a plant disposed along a planting bed, where the camera mounted to a frame being moved along the planting bed by a vehicle. The plant inspection system further includes a processor that is associated with the camera and configured to identifying a region of interest displayed in the first series of images as a region of contamination on the plant based on a color criterion and a morphological criterion applied to the region of interest. The plant inspection system further includes a plant eradication device of multiple plant eradication devices that is mounted to the frame and that can be selectively actuated to remove the plant from a surface of the planting bed in response to the processor identifying the region of interest as the region of contamination, while the camera generates a second series of images of an additional plant disposed along the planting bed as the frame continues to be moved along the planting bed by the vehicle.
The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, aspects, and advantages of the invention will be apparent from the description, drawings, and claims.
This patent or patent application publication contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
Like reference symbols in the various figures indicate like elements.
Plant inspection systems for identifying contaminated plants during harvesting operations and pre-harvest operations are described below. The described plant inspection systems includes a manipulation device, a machine vision system, and associated control elements that allow the plant inspection systems to identify contaminated regions on plant components in an automated and real-time manner, thereby improving a yield of a harvesting operation as compared to that which would be achieved using conventional harvesting techniques. Plants that may be examined by the plant inspection systems include leafy vegetable plants (e.g., baby green vegetable plants and mature green vegetable plants) and other vegetable plants grown in fields.
The plant inspection system 100 is mounted to a frame 115 located along a front side of the harvester, which moves in a forward direction 101 along a planting bed 107 (e.g., a bed of soil) in which multiple rows (e.g., parallel rows) of leafy vegetable plants 103 are planted. (For illustration purposes, the multiple rows of leafy vegetable plants 103 are represented by a single block 103 in
The plant inspection system 100 is positioned forward of a cutting blade 109 (e.g., a bandsaw) and a conveyor 111 (e.g., a belt conveyor) of the harvester. The conveyor 111 is operable to move severed plants 103 rearward towards a collection container and personnel riding on the harvester who place the plants 103 into the collection containers. The cutting blade 109 and the conveyor 111 can be lowered to harvest plants 103 from the planting bed 107 (e.g., to sever plants 103 from the planting bed 107 and to convey the severed plants 103) and raised to avoid harvesting plants 103 from the planting bed 107 (e.g., to avoid contact between the cutting blade 109 and the conveyor 111 and the plants 103). During a harvesting operation, the cutting blade 109 and the conveyor 111 are maintained at a predetermined cutting distance above the planting bed 107, which can vary in elevation according to an elevation of the field in which the planting bed 107 is arranged. Accordingly, the cutting blade 109 and the conveyor 111 float about 5 cm to about 10 cm (e.g., about 8 cm) above the planting bed 107.
The cutting blade 109 and the conveyor 111 typically have a cutting width that is at least as large as the width of the planting bed 107. Therefore, in some examples, the cutting blade 109 and the conveyor 111 have a cutting width of about 150 cm to about 170 cm (e.g., about 160 cm). For illustration purposes, only components of the plant inspection system 100 and of the harvester that are significant to the description of plant inspection are shown. However, it will be understood that other standard electrical and mechanical components will be included. For example, the plant inspection system 100 and/or the harvester may include a generator and/or batteries for powering the electrical components.
The plant inspection system 100 includes a machine vision system 102 that generates images of the plants 103, two connection arms 104 by which the machine vision system 102 is attached to the harvester, two frames 106 that support the machine vision system 102, four wheels 108 attached (e.g., welded at hubs) to the frames 106, an enclosure 110 that houses various electrical components, and a cutter position sensor 148 (e.g., an ultrasonic sensor or laser position sensor) that detects a height of the cutting blade 109. The plant inspection system 100 also includes a programmable controller (PC) 118 that is electrically coupled to the machine vision system 102, a GPS system 120 that provides field locations to the PC 118, a cellular system 146 for remote communication and reporting, and a WiFi system 122 that wirelessly transmits data from the PC to a computing device (e.g., a laptop computer, a tablet computer, a smartphone) that is used by an operator of the harvester. The PC 118, the GPS system 120, cellular system 146, and the WiFi system 122 are housed within the enclosure 110.
The connection arms 104 include multiple segments. The connection arms 104 are attached at first ends 112 to the frame 115 along the front side of the harvester and are attached at second ends 114 to the machine vision system 102. The connection arms 104 are pivotable with respect to the frame 115 of the harvester at first ends 112 and, to a lesser extent, pivotable with respect to the machine vision system 102. The connection arms 104 position the machine vision system 102 at a distance of about 100 cm to about 200 cm (e.g., about 150 cm) forward of the harvester. The connection arms 104 are also operable to raise and lower the machine vision system 102 and other components attached directly or indirectly thereto while the harvester is maneuvered (e.g., turned) from one planting bed 107 to a next planting bed 107 when the harvester reaches an end of a planting bed 107. In this regard, the plant inspection system 100 includes two hydraulic cylinders 116 that are operable to raise and lower a height of the connection arms 104.
The plant inspection system 100 is configured such that the wheels 108 are positioned within the furrows 105 extending along the planting bed 107. The wheels 108, together with the connection arms 104, allow the harvester to push the plant inspection system 100 in the forward direction 101 in a manner such that the machine vision system 102 floats above the plants 103. Movements of the cutting blade 109 and the conveyor 111 of the harvester are independent of movements of the plant inspection system 100, such that the wheels 108 of the plant inspection system 100 remain on the ground irrespective of the height of the cutting blade 109 and the conveyor 111 above the plants 103. The frames 106, to which the wheels 108 are attached, are vertically adjustable relative to the machine vision system 102 to accommodate plants 103, planting beds 107, and fields of variable heights and elevations.
Referring to
The cameras 124 and the light arrays 126 are centrally mounted to respective panels 128 such that the cameras 124 and the light arrays 126 are spaced about evenly apart across the hood 130. The light arrays 126 may be provided as ring light arrays that surround the cameras 124 or as bar light arrays that are otherwise positioned in association with the cameras 124. In some examples, the ring light arrays may provide more uniform illumination than the bar light arrays. The light arrays 126 include multiple LEDs that have filters for sufficient illumination and desired image characteristics. The hood 130 is adapted to block (e.g., reduce the amount of) sunlight, other types of light, and precipitation that may otherwise impinge upon components of the plant inspection system 100 or upon the plants 103 in the fields of view during the harvesting operation. The plant inspection system 100 also includes a skirt 136 (e.g., a neoprene skirt) that covers a gap between the lower rail 134 and a top surface of plants 103. Like the hood 130, the skirt 136 is adapted to block light and precipitation that may otherwise impinge upon components of the plant inspection system 100 and upon the plants 103 in the fields of view. The skirt 136 can be mounted at various vertical positions along the lower rail 134 depending on the height of the plants 103.
The cameras 124 are oriented (e.g., horizontally) and positioned to image respective fields of view along the top surface of the plants 103. The cameras 124 may be standard resolution, color video graphics array (VGA) cameras known to a person skilled in the art. For example, the cameras 124 may have a pixel count of 480×640 and image a 27 cm×36 cm field of view. The camera resolution (e.g., pixel dimension) of such a field of view may be 0.056 cm, which is adequate for identifying contamination (e.g., feces, disease, or foreign matter) on components of the plants 103. The focal length selected for the cameras 124 is 6 mm. A working distance between lenses of the cameras 124 and the top surface of the plants is 45 cm. The cameras 124 can acquire images every 100 ms, allowing the cameras 124 to acquire three images of the same plant 103 while the inspection system 100 moves at a predetermined speed (e.g., about 65 cm/s) in the field 107. The images acquired by each camera 124 overlap the images acquired by a neighboring camera 124 by about 2.5 cm, such that no gaps exist between images acquired of adjacent fields of view. The fields of view (e.g., when aligned serially) cover a width of about 180 cm.
Referring to
Referring particularly to
Referring particularly to
The processor 144 then combines the pixels meeting a particular color criteria (e.g., pixels that are adjacent or sufficiently close to each other) into a blob (e.g., as illustrated by the blobs 206, 208) and draws a border around the blob, thereby defining a pattern (e.g., a shape) of the blob. In some examples, bird feces appear as spots or splotches on plants 103. In some examples, fungal diseases appear as chlorotic lesions or small circular lesions on plants 103. In some examples, leaf tip burn is exhibited by withering of leaf tips. The camera processor 144 further determines a size of the blob (e.g., a length and/or a width of the respective pattern). The processor 144 compares the pattern, size, and color of the blob to known (e.g., stored) patterns, sizes, and colors of bird feces (or other animal feces), fungal diseases, leaf tip burn, plant viruses, other diseases, rodents, insects, and foreign matter. Blobs with patterns that sufficiently match known patterns of contamination, that meet a minimum size threshold (e.g., a stored threshold value) for the contamination, and that fall within a color range associated with a particular type of contamination can be identified accordingly and marked with a crosshair 210 (e.g., the blue crosshair faintly shown in
In some examples, the camera processor 144 varies the Red Green Blue (RGB) data of the pixels in the image 200 prior to performing the blob analysis, in order to brighten the image 200, to enhance the pixel colors in the image 200, and/or to increase contrast among the colors in the image 200, thereby producing an enhanced image. Such processing can improve the detection of contamination in dim light (e.g., at night or when the contamination is shadowed by leaves or other plant material) or in cases where the contamination is affected by moisture or other environmental contaminants.
Once the camera processor 144 identifies a blob (e.g., the blob 206 or the blob 208) corresponding to a particular type of contamination, the camera processor 144 runs an algorithm to determine digital image position coordinates of the blob (e.g., xy position coordinates of a centroid of the blob in a digital coordinate system of the digital image). The processor 144 then provides the digital image position coordinates and machine vision views 202, 204 with identified regions of contamination to the PC 118. The PC 118 downloads GPS coordinates of the camera 124 from the GPS system 120 and determines field position coordinates of the regions of contamination based on the digital image position coordinates and the GPS coordinates. The PCL 118 stores the machine vision views 202, 204, the field position coordinates of the regions of contamination, the type of contamination identified, and a summary of an evasive action taken to avoid harvesting the contaminated plants 103, which will be discussed in more detail below. In some examples, such parameters are stored in association with one more of a date, a time, and other field location parameters. The camera processor 144 sends the machine vision views 202, 204 showing identified regions of contamination and associated digital image position coordinates to the PC 118 in a continuous manner as regions of contamination are identified.
Using the above-described image analyses, the plant inspection system 100 can quickly process large amounts of information to recognize colors and patterns to identify regions of contamination on surfaces of the plants 103. In contrast, conventional, manual inspections performed by the operator of the harvester or by field personnel cannot achieve such an extent of inspection detail in a feasible manner (e.g., in a reasonable amount of time, in a manner that systematically identifies hidden contamination, or performing the harvesting operation during daylight hours, when visibility is optimal but when climate and plant conditions are sub-optimal). Accordingly, as compared to such conventional analyses, the above-described image analyses are more accurate, provide more information, and are more successful in correctly identifying contaminated plants 103.
When the PC 118 receives an indication (e.g., machine vision views 202, 204 showing an identified region of contamination along with digital image position coordinates) that a region of contamination has been identified, the PC 118 sends a control signal to a light stand (omitted for clarity) on the plant inspection system 100 to actuate a red warning light within the light stand. The PC 118 also sends a control signal via the WiFi system 122 to the computing device used by the operator of the harvester to display a message on the computing device instructing the operator to raise the cutting blade 109. The red warning light and the message can alert the operator to the contaminated plant 103. In response to the red warning light and/or to the message displayed on the computing device, the operator will take actions to raise the cutting blade 109 (and the associated conveyor 111) to avoid harvesting the contaminated plant 103 from the planting bed 107 (e.g., to prevent the cutting blade 109 and the conveyor 111 from contacting the contaminated plant 103).
According to a distance between the cutting blade 109 and the plant inspection system 100 (e.g., about 150 cm), the operator will have about 1 s to about 3 s (e.g., about 2 s) to raise the cutting blade 109 to prevent the cutting blade 109 from contacting the contaminated plant 103 from the time that the operator perceives the red warning light or the message. In some examples, the operator raises the cutting blade 109 by about 4 cm to about 8 cm above the plants 103 to clear (e.g., avoid contact with) the contaminated plant 103. The cutter position sensor 148 records a height of the cutting blade 109 (e.g., relative to the top surface of the planting bed 107) achieved by the cutting blade 109 during lifting. Based on the height achieved by the cutting blade 109 and the forward movement of the harvester, the PC 118 determines whether or not the cutting blade 109 successfully avoided contact with the contaminated plant 103. If the cutting blade 109 successfully bypasses the contaminated plant 103, then the PC 118 sends a control signal to the light stand to actuate a green light (e.g., an “all clear” light) in the light stand. The PC 118 also sends a control signal to the computing device used by the operator of the harvester to display a message indicating that the cutting blade 109 can be lowered. In response to the green light and/or to the message displayed on the computing device, the operator will take actions to lower the cutting blade 109 (and the associated conveyor 111) to continue harvesting plants 103 from the planting bed 107.
If, on the other hand, the PC 118 determines that the cutting blade 109 was not raised fast enough or high enough to avoid contact with the contaminated plant 103, then the PC 118 will send a control signal to the light stand to actuate a blinking red light. The PC 118 will also send a control signal via the WiFi system 122 to the computing device used by the operator of the harvester to display a message indicating that the contaminated plant 103 was harvested, that the contaminated plant 103 should be removed from the harvester, and that, in some cases (e.g., if the contamination is identified as bird feces), that the harvester should be decontaminated (e.g., disinfected or sterilized) where contact was made with the contaminated plant 103.
As mentioned above, the PC 118 is programmed to store a log of all machine vision views 202, 204 showing regions of contamination, field position coordinates of the regions of contamination, types of contamination identified, successful and failed attempts to avoid harvesting contaminated plants 103, and dates and times at which the attempts occurred. Using the information in the log, the PC 118 can generate a field report at a predetermined time (e.g., at an end of a harvesting operation or at a certain time of the day). The PC 118 may send the field report to the computing device used by the operator or to another computing device remote from the harvester and the plant inspection system 100 via the cellular system 146 so that actions can be taken by growers of the field. Such actions may include authorizing shipment of plant products or holding plant products for manual inspection.
Owing to capabilities of the plant inspection system 100 to illuminate a planting bed, to identify contaminated plants, and to alert an operator of a harvester to the presence of contaminated plants in real time, the plant inspection system 100 can allow a field to be harvested efficiently and during periods of limited visibility (e.g., at night) while avoiding dangerous contamination (e.g., salmonella poisoning resulting from bird feces) of previously harvested plants such that food safety is maintained and while avoiding costly shutdowns and decontaminations of the harvester or of a downstream (e.g., post-harvest) processing line (e.g., at a processing plant).
A processor associated with the camera then identifies a region of interest displayed in the first series of images from feature boundary data defined by color regions associated with the first series of images (304). For example, the processor performs a blob analysis on the images by combining pixels that are adjacent or sufficiently close to each other and within a certain range of colors into a blob. The processor then determines a border around the blob, thereby defining a pattern (e.g., a shape) of the blob. The processor further generates a series of machine vision views associated with the first series of images and displaying the blob. Next, the processor compares a color parameter of the region of interest to a color criterion associated with a type of contamination (306) and compares a morphological parameter (e.g., a size or a shape) of the region of interest to a reference parameter associated with the type of contamination (308).
Upon determining that the color parameter meets the color criterion and that the morphological parameter sufficiently matches the reference parameter, the processor identifies the region of interest as a region of contamination (e.g., feces, disease, rodents, insects, or foreign matter) on a surface of the plant (310). In response to the processor identifying the region of interest as the region of contamination, a PC communicably coupled to the processor transmits data including an instruction to lift a cutter of the harvester up from the planting bed to avoid harvesting (e.g., contacting and further severing) the plant (312). In some examples, the instruction is a signal to actuate a light that can alert an operator of the harvester to the region of contamination. In some examples, the instruction is a message displayed on a computing device associated with the harvester (e.g., a computing device in proximity to the operator of the harvester or a computing device remote from the harvester). In other cases, the instruction is a control signal to automatically lift the cutter.
In response to the processor identifying the region of interest as the region of contamination, the PC also stores a series of machine vision views associated with the first series of images in association with position coordinates of the region of contamination (e.g., based on GPS coordinates received from a GPS system) and stores a record of a lifting action taken by the cutter to avoid harvesting the plant. The PC can generate a report including the series of machine vision views, the position coordinates of the region of contamination, the record of the lifting action, and other information related to an identification of the region of interest as the region of contamination. Based on a height to which the cutter was lifted and an amount of time elapsed between the data being transmitted and the cutter being lifted, the PC can determine whether or not the cutter contacted the plant. If the cutter cleared the plant, then the PC transmits data including an instruction to lower the cutter of the harvester towards the planting bed to continue harvesting plants from the planting bed. If the cutter did not clear the plant, then the PC transmits data including an instruction to cease the harvesting operation and to decontaminate one or more portions (e.g., the cutter and a conveyor) of the harvester.
While the cutter of the harvester is being lifted up from the planting bed, the camera generates a second series of images of an additional plant disposed along the planting bed as the frame continues to be moved along the planting bed by the harvester (314). In some embodiments, an additional camera mounted to the harvester and downstream of the cutter generates a third series of images of the additional plant after the additional plant has been harvested by the cutter and transported to a conveyor belt, as will be discussed in more detail below with respect to
While the plant inspection system 100 has been described as providing a signal that instructs an operator of a harvester to manually raise and lower a cutting blade of the harvester, in some embodiments, the PC 118 of the plant inspection system 100 is programmed to send a control signal to the harvester that automatically causes the cutting blade to be raised and lowered, which can shorten a response time to avoid contaminated plants, as compared to an amount of time required for the operator to manually takes actions to raise and lower the cutting blade.
While the plant inspection system 100 has been described and illustrated as being attached to a frontal frame of a harvester, in some embodiments, a plant inspection system may be located above a conveyor belt for inspecting plants that have been cut from a planting bed. For example,
While the plant inspection system 100 has been described and illustrated as preventing the harvest of contaminated plants by lifting a cutting blade up from a planting bed to avoid contact between the cutting blade and the contaminated plants, in some embodiments, a plant inspection system can prevent the harvest of contaminated plants by forcing the contaminated plants downward beneath a cutting blade to prevent contact between the cutting blade and the contaminated plants. For example,
The plant inspection system 500 includes multiple components of the plant inspection system 100 that are constructed and operable as described above with respect to
The deflection plates 550 are mounted to supporting structural components (e.g., rods or other components) and are individually actuated by pneumatic or hydraulic cylinders that are connected to respective, individual solenoid valves (omitted from
One or more deflection plates 550 can be selectively actuated (e.g., swung downward about the deflection axis 558) simultaneously or at different times to push one or more respective, contaminated plants 103 down toward the planting bed 107. Owing to the selective, individual activation of the deflection plates 550, contaminated plants 103 are prevented from coming into contact with the cutting blade 109 of the harvester, while non-contaminated or otherwise healthy plants 103 disposed along the same cross-machine position (e.g., along a same lateral position across the width of the planting bed 107) as the contaminated plants 103 are maintained intact and therefore harvested. Accordingly, use of the deflection plates 550 during a harvesting operation can increase a yield of the plants 103, as compared to lifting the cutting blade 109 (e.g., on the plant inspection system 100) to avoid an entire row including both contaminated and non-contaminated plants 103. The deflection plates 550 can be cleaned (e.g., using a high pressure wash) in place. In some examples, portions (e.g., the machine vision system 102) of the plant inspection system 500 can be moved (e.g., wheeled away) to facilitate cleaning of the deflection plates 550.
In some examples, the push plates 554 of the deflection plates 550 have a width of about 10 cm to about 20 cm (e.g., about 15 cm), a length of about 20 cm to about 40 cm (e.g., about 30 cm), and a thickness of about 0.2 cm to about 0.8 cm (e.g., about 0.5 cm). In some examples, the flanges 556 of the deflection plates 550 have a width of about 10 cm to about 20 cm (e.g., about 15 cm), a length of about 2 cm to about 8 cm (e.g., about 5 cm), and a thickness of about 0.2 cm to about 0.8 cm (e.g., about 0.5 cm). In some examples, the flanges 556 extend from the push plates 554 at an angle of about 30 degrees to about 60 degrees (e.g., about 45 degrees). In some examples, a height of the deflection axis 558 (e.g., defining a non-actuated height of the deflection plates 550) is located at a vertical distance of about 6 cm to about 18 cm (e.g., about 12 cm) above a center height of the wheels 108. In some examples, the deflection plates 550 are pivotable downward from an orientation approximately parallel to the planting bed 107 through a deflection angle of about 30 degrees to about 60 degrees (e.g., 45 degrees) to push contaminated plants 103 down toward the planting bed 107. The deflection angle (e.g., a pre-set parameter) can be input to the computing device by the operator prior to the harvesting operation. In some examples, the deflection plates 550 are pivotable at an angular velocity of about 300 degrees per second to about 600 degrees per second (e.g., about 450 degrees per second). In some examples, the deflection plates 550 are operable to smash contaminated plants 103 down to a height within a range of about 0.5 cm to about 1.0 cm above the top surface of the planting bed 107 or to push contaminated plants 103 downward a depth in a range of about 0.5 cm to about 2.0 cm beneath the top surface of the planting bed 107. All or a portion of the contaminated plants 103 may be pushed into the planting bed 107, depending on the depth. Example materials from which the deflection plates 550 may be made include tool steel, stainless steel, and other materials.
Using the image analyses described above with respect to the plant inspection system 100, the plant inspection system 500 can quickly process large amounts of information to recognize colors and patterns to identify regions of contamination on surfaces of plants 103. When the PC 118 receives an indication (e.g., machine vision views 202, 204 showing one or more identified regions of contamination along with digital image position coordinates) that one or more regions of contamination have been identified as the plant inspection system 500 moves along the planting bed 107, the PC 118 sends a control signal to a light stand (omitted from
In response to identification of the one or more contaminated plants 103, the PC 118 also sends one or more control signals to one or more respective actuation cylinders associated with respective deflection plates 550 disposed above the respective one or more contaminated plants 103. According to the one or more control signals, the one or more deflection plates 550 are lowered (e.g., swung downward about the deflection axis 558) to push the one or more contaminated plants 103 downward to smash the one or more contaminated plants 103 atop the planting bed 107 or to force the one or more contaminated plants 103 into the planting bed 107, thereby increasing a space (e.g., a vertical distance) between the one or more contaminated plants 103 and the cutting blade 109 of the harvester. Meanwhile, non-contaminated plants 103 located along the same cross-machine position as the one or more contaminated plants 103 are severed by the cutting blade 109.
Based on the forward movement of the harvester and the time at which the one or more deflection plates 550 are actuated, the PC 118 determines whether or not contact was prevented between the cutting blade 109 and the one or more contaminated plants 103. If contact was successfully prevented, then the PC 118 sends a control signal to the light stand to actuate a green light (e.g., an “all clear” light) in the light stand. The PC 118 also sends a control signal to the computing device used by the operator of the harvester to display a message indicating that contact was prevented. The green light and/or the message displayed on the computing device alert the operator to the successful avoidance of the one or more contaminated plants 103. If, on the other hand, the PC 118 determines that contact was made between the cutting blade 109 and one or more contaminated plants 103, then the PC 118 sends a control signal to the light stand to actuate a blinking red light. The PC 118 also sends a control signal via the WiFi system 122 to the computing device used by the operator of the harvester to display a message indicating that one or more of the contaminated plants 103 were harvested, that the one or more contaminated plants 103 should be removed from the harvester, and that, in some cases (e.g., if the contamination is identified as bird feces), that the harvester should be decontaminated (e.g., disinfected or sterilized) where contact was made with the one or more contaminated plants 103.
As discussed above with respect to the plant inspection system 100, the PC 118 is programmed to store a log of all machine vision views 202, 204 showing regions of contamination, field position coordinates of the regions of contamination, and types of contamination identified. In the plant inspection system 500, the PC 118 is additionally programmed to store a log of successful and failed attempts to push contaminated plants 103 down toward the planting bed 107 to prevent contact between the cutting blade 109 and the contaminated plants 103 and dates and times at which the attempts occurred. Using the information in the log, the PC 118 can generate a field report at a predetermined time (e.g., at an end of a harvesting operation or at a certain time of the day). The PC 118 may send the field report to the computing device used by the operator or to another computing device remote from the harvester and the plant inspection system 500 via the cellular system 146 so that actions can be taken by growers of the field. Such actions may include authorizing shipment of plant products or holding plant products for manual inspection.
Owing to capabilities of the plant inspection system 500 to illuminate a planting bed, to identify contaminated plants, to prevent contact between a cutting blade and contaminated plants in real time, and to alert an operator of a harvester to an occurrence of harvesting a contaminated plant in real time, the plant inspection system 500 can allow a field to be harvested efficiently and during periods of limited visibility (e.g., at night) while avoiding dangerous contamination (e.g., salmonella poisoning resulting from bird feces) of previously harvested plants such that food safety is maintained and while avoiding costly shutdowns and decontaminations of the harvester or of a downstream (e.g., post-harvest) processing line (e.g., at a processing plant).
A processor associated with the camera then identifies a region of interest displayed in the first series of images as a region of contamination (e.g., feces, disease, rodents, insects, or foreign matter) on the plant based on a color criterion and a morphological criterion applied to the region of interest (604). For example, the processor performs a blob analysis on the images that identifies portions of the region of interest that share one or more colors by combining pixels that are adjacent or sufficiently close to each other and within a certain range of colors into a blob. The processor then determines a border around the blob, thereby defining a pattern (e.g., a shape) of the blob. The processor further generates a series of machine vision views associated with the first series of images and displaying the blob. The morphological criterion may be related to a shape and/or a size of the region of interest.
In response to the processor identifying the region of interest as the region of contamination, a PC communicably coupled to the processor transmits data including an instruction to increase a vertical distance between the plant and a cutter of the harvester to avoid harvesting (e.g., contacting and further severing) the plant (606). In some implementations, the instruction indicates that the cutter should be lifted up from the planting bed. In some implementations, a deflection plate of multiple deflection plates is selectively actuated to force the plant down towards the planting bed based on the instruction. In some examples, the deflection plate is selectively actuated by rotating the deflection plate down towards the plant. In some cases, the plant is flattened atop the planting bed. In other cases, the plant is pushed down into the planting bed. In some examples, a non-contaminated or otherwise healthy plant adjacent to the plant is harvested while the vertical distance between the plant and the cutter is being increased by selectively actuating a second deflection plate of the multiple deflection plates. In some examples, a vertical distance between the cutter and a second contaminated plant is increased (e.g., by selectively actuating a second deflection plate of the multiple deflection plates) while the vertical distance is being increased between the cutter and the plant.
In some examples, the instruction is associated with an actuation of a light that can alert an operator of the harvester to the region of contamination. In some examples, the instruction includes a message displayed on a computing device associated with the harvester (e.g., a computing device in proximity to the operator of the harvester or a computing device remote from the harvester). In other cases, the instruction is a control signal to automatically perform an action (e.g., selectively actuating a deflection plate or lifting the cutter) to increase the distance between the plant and the cutter.
In response to the processor identifying the region of interest as the region of contamination, the PC also stores a series of machine vision views associated with the first series of images in association with position coordinates of the region of contamination (e.g., based on GPS coordinates received from a GPS system) and stores a record of an action taken to increase the distance between the plant and the cutter (e.g., a lifting action taken by the cutter or a selective rotation of a deflection device). The PC can generate a report including the series of machine vision views, the position coordinates of the region of contamination, the record of the action, and other information related to an identification of the region of interest as the region of contamination. Based on an amount of time elapsed between the data being transmitted and the action being taken, the PC can determine whether or not the cutter contacted the plant. If the cutter cleared the plant, then the PC transmits data indicating that it is safe to continue harvesting plants from the planting bed. If the cutter did not clear the plant, then the PC transmits data including an instruction to cease the harvesting operation and to decontaminate one or more portions (e.g., the cutter and a conveyor) of the harvester.
While the vertical distance between the plant and cutter is being increased, the camera generates a second series of images of an additional plant disposed along the planting bed as the frame continues to be moved along the planting bed by the harvester (608). In some embodiments, an additional camera mounted to the harvester and downstream of the cutter generates a third series of images of the additional plant after the additional plant has been harvested by the cutter and transported to a conveyor belt, as discussed above with respect to
While the plant inspection system 500 has been described and illustrated as attachable to a front side of a harvester for use during a harvesting operation, in some embodiments, a plant inspection system may be attached to a rear side of a tractor to inspect plants prior to a harvesting operation. For example, a pre-harvest operation can be carried out by a plant inspection system that is substantially similar in construction and function to the plant inspection system 500, except that the plant inspection system includes structural features for a standard three-point hitch instead of the connection arms 104. Accordingly, such a plant inspection system also includes the machine vision system 102 (e.g., including the cameras 124, the camera processors 144, the light arrays 126, the panels 128, the hood 130, the upper rail 132, and the lower rail 134), the skirt 136, the frame 106, the wheels 108, the enclosure 110 (e.g., housing the PC 118, the GPS system 120, the WiFi system 122, and the cellular system 146), the deflection plates 550, the manipulation device, the light stand, and the lights supported thereon, as described with respect to the plant inspection system 500 and
The plant inspection system is attached to a rear side of a tractor via the three-point hitch, such that during a pre-harvest operation, the plant inspection system is pulled along a planting bed 107 in the forward direction 101 by the tractor, with the machine vision system 102 located behind the tractor and the deflection plates 550 located behind the machine vision system 102. In such cases, the deflection angle and/or a stiffness of the deflection plates 550 may be selected such that the deflection plates 550 are actuated to push the contaminated plants 103 downward beneath the top surface of the planting bed 107 to prevent the contaminated plants 103 from erecting above the top surface of the planting bed 107 prior to a subsequent harvesting operation that may occur 1 hour to 24 hours later.
During a pre-harvest operation, the PC 118 of the plant inspection system determines whether or not one or more contaminated plants 103 were successfully pushed down towards or down into the planting bed 107 based on the forward movement of the tractor and the time at which the one or more deflection plates 550 are actuated. If the one or more contaminated plants 103 were not successfully pushed down, then the PC 118 sends a control signal to the light stand to actuate the blinking red light. The PC 118 also sends a control signal via the WiFi system 122 to a computing device used by an operator of the tractor to display a message indicating that the one or more contaminated plants 103 remain and should be manually removed from the planting bed 107 or avoided in real-time during a subsequent harvesting operation (e.g., using the plant inspection system 100).
During or after a pre-harvest operation, the PC 118 may send a field report to the computing device used by the operator of the tractor or to another computing device remote from the tractor and the plant inspection system so that actions can be taken by growers of the field. Such actions may include authorizing a subsequent harvesting operation, authorizing shipment of plant products, or holding plant products for manual inspection. Using such a plant inspection system during a pre-harvest operation can allow the field to be subsequently harvested efficiently and during periods of limited visibility (e.g., at night) while avoiding dangerous contamination (e.g., salmonella poisoning resulting from bird feces) of previously harvested plants such that food safety is maintained and while avoiding costly shutdowns and decontaminations of a harvester or of a downstream (e.g., post-harvest) processing line (e.g., at a processing plant).
While this plant inspection system has been described as operable to push contaminated plants 103 downward during a pre-harvest operation, in some embodiments, a plant inspection system includes a removal system that severs contaminated plants from a planting bed during a pre-harvest operation. For example,
The plant inspection system 700 includes multiple components of the plant inspection systems 100, 500 that are constructed and operable as described above with respect to
The removal system 760 is located behind the machine vision system 102, which is located behind the tractor. The removal system 760 includes a conveyor 762 (e.g., a belt conveyor) and multiple cutting heads 764. The cutting heads 764 are individually operable to sever contaminated plants 103 from the planting bed 107 and transport the severed, contaminated plants 103 to the conveyor 762. The cutting heads 764 are mounted to one or more supporting structural components (e.g., rods or other components) and are individually actuated by pneumatic or hydraulic cylinders that are connected to respective, individual solenoid valves (omitted from
The cutting heads 764 are pivotable about a cutting axis 772 along rear ends of the conveyors 768. One or more cutting heads 764 can be selectively actuated (e.g., rotated downward about the cutting axis 772) simultaneously or at different times to sever one or more respective, contaminated plants 103 from the planting bed 107. The conveyors 768 of the cutting heads 764 transport the severed, contaminated plants 103 to the conveyor 762, which transports the contaminated plants 103 in a direction 703 (e.g., perpendicular to the forward direction 101) such that the contaminated plants 103 are deposited in a furrow 105. In some examples, foreign material (e.g., debris or other particles) may be disposed within or atop the plants 103 and be severed and transported away from the planting bed 107 along with the plants 103 by the removal system 760.
Owing to the selective, individual activation of the cutting heads 764, contaminated plants 103 are removed from the planting bed 107, while non-contaminated or otherwise healthy plants 103 disposed along the same cross-machine position (e.g., along a same lateral position across the width of the planting bed 107) as the contaminated plants 103 are maintained and therefore available for a subsequent harvest. Accordingly, use of the cutting heads 764 during a pre-harvest operation can increase a yield of the plants 103, as compared to lifting the cutting blade 109 (e.g., on the plant inspection system 100) to avoid all of the plants 103 (e.g., both non-contaminated and contaminated plants 103) located along the same cross-machine position during a harvesting operation. The cutting heads 764 and the conveyor 762 can be cleaned (e.g., using a high pressure wash) in place. In some examples, portions (e.g., the machine vision system 102) of the plant inspection system 700 can be moved (e.g., wheeled away) to facilitate cleaning of the cutting heads 764 and the conveyor 762.
In some examples, the conveyors 768 of the cutting heads 764 have a width of about 10 cm to about 20 cm (e.g., about 15 cm) and a length of about 40 cm to about 80 cm (e.g., about 60 cm). In some examples, the paddle wheels 770 of the cutting heads 764 have a diameter of about 6 cm to about 18 cm (e.g., about 12 cm). The paddle wheels 770 may be made of one or more flexible materials (e.g., urethane) such that the paddle wheels 770 can adapt to manipulate the severed, contaminated plants 103. In some examples, a height of the cutting axis 772 (e.g., defining a non-actuated height of the cutting heads 764) is located at a vertical distance of about 6 cm to about 18 cm (e.g., about 12 cm) above a center height of the wheels 108. In some examples, the cutting heads 764 are pivotable downward from an orientation approximately parallel to the planting bed 107 through a cutting angle of about 15 degrees to about 45 degrees (e.g., 30 degrees) to sever contaminated plants 103 from the planting bed 107. The cutting angle (e.g., a pre-set parameter) can be input to the computing device by the operator prior to the inspection. The cutting angle may be selected such that the contaminated plants 103 are cut at a distance of about 1.0 cm to about 2.0 cm (e.g., about 1.5 cm) above the planting bed 107. In some examples, the conveyor 762 may have a length (e.g., along the direction 703) of about 175 cm to about 225 cm (e.g., about 200 cm).
Using the image analyses described above with respect to the plant inspection system 100, 500, the plant inspection system 700 can quickly process large amounts of information to recognize colors and patterns to identify regions of contamination on surfaces of plants 103. When the PC 118 receives an indication (e.g., machine vision views 202, 204 showing one or more identified regions of contamination along with digital image position coordinates) that one or more regions of contamination have been identified as the plant inspection system 700 moves along the planting bed 107, the PC 118 sends a control signal to a light stand (omitted from
In response to identification of the one or more contaminated plants 103, the PC 118 also sends one or more control signals to one or more respective actuation cylinders associated with respective cutting heads 764 disposed above the respective one or more contaminated plants 103. According to the one or more control signals, the one or more cutting heads 764 are rotated downward about the cutting axis 772 to cut and lift the one or more contaminated plants 103. Meanwhile, non-contaminated plants 103 located along the same cross-machine position as the one or more contaminated plants 103 are maintained for subsequent harvesting.
Based on the forward movement of the tractor and the time at which the one or more cutting heads 764 are actuated, the PC 118 determines whether or not the one or more contaminated plants 103 were removed from the planting bed 107. If the one or more contaminated plants 103 were successfully removed (e.g., severed and transported to the conveyors 768), then the PC 118 sends a control signal to the light stand to actuate a green light (e.g., an “all clear” light) in the light stand. The PC 118 also sends a control signal to a computing device used by the operator of the tractor to display a message indicating that the one or more contaminated plants 103 were removed. The green light and/or the message displayed on the computing device alert the operator to the successful removal of the one or more contaminated plants 103 from the planting bed 107. If, on the other hand, the PC 118 determines that the one or more contaminated plants 103 were not successfully removed, then the PC 118 sends a control signal to the light stand to actuate a blinking red light. The PC 118 also sends a control signal via the WiFi system 122 to the computing device used by the operator of the tractor to display a message indicating that the one or more contaminated plants 103 remain and should be manually removed from the planting bed 107 or avoided in real-time during a subsequent harvesting operation (e.g., using the plant inspection system 100).
As discussed above with respect to the plant inspection systems 100, 500, the PC 118 is programmed to store a log of all machine vision views 202, 204 showing regions of contamination, field position coordinates of the regions of contamination, and types of contamination identified. In the plant inspection system 700, the PC 118 is additionally programmed to store a log of successful and failed attempts to remove contaminated plants 103 from the planting bed 107 and dates and times at which the attempts occurred. Using the information in the log, the PC 118 can generate a field report at a predetermined time (e.g., at an end of a pre-harvest operation or at a certain time of the day). The PC 118 may send the field report to the computing device used by the operator or to another computing device remote from the tractor and the plant inspection system 700 via the cellular system 146 so that actions can be taken by growers of the field. Such actions may include authorizing a subsequent harvesting operation, authorizing shipment of plant products, or holding plant products for manual inspection.
Owing to capabilities of the plant inspection system 700 to illuminate a planting bed, to identify contaminated plants, to remove contaminated plants from a planting bed in real time, and to alert an operator of a tractor to a failed attempt to remove a contaminated plant in real time, the plant inspection system 700 can allow a field to be subsequently harvested efficiently and during periods of limited visibility (e.g., at night) while avoiding dangerous contamination (e.g., salmonella poisoning resulting from bird feces) of previously harvested plants such that food safety is maintained and while avoiding costly shutdowns and decontaminations of a harvester or of a downstream (e.g., post-harvest) processing line (e.g., at a processing plant).
In some examples, the plant eradication device is a deflection plate, and selectively actuating the plant eradication device includes rotating the deflection plate down towards the plant to force the plant down into the planting bed based on the instruction. In some examples, the plant eradication device includes a cutting blade (e.g., a knife) and a rotating carrier (e.g., a paddle wheel or a conveyor). Accordingly, in some examples, selectively actuating the plant eradication device includes severing the plant with the cutting blade and transporting the plant away from a location at which the plant is growing with the rotating carrier. The plant may then be conveyed (e.g., by a conveyor extending across a width of the planting bed) away from the planting bed in a first direction perpendicular to a second direction in which the frame is being moved along the planting bed by the vehicle.
While the plant eradication device is being selectively actuated, the camera generates a second series of images of an additional plant disposed along the planting bed as the frame continues to be moved along the planting bed by the vehicle (808). In some examples, a non-contaminated or otherwise healthy plant adjacent to the plant is harvested while the plant is being removed from the surface of the planting bed. In some examples, a second plant eradication device of the multiple plant eradication devices is selectively actuated to remove a second contaminated plant from the surface of the planting bed while the first plant eradication device is being selectively actuated. Following inspection of the planting bed and removal of contaminated plants from the surface of the planting bed using either of the systems 500, 700 during a pre-harvest operation, the planting bed can be subsequently harvested during a harvesting operation.
While the plant inspection systems 100, 400, 500, 700 have been described and illustrated as being used to inspect baby green vegetable plants, in some examples, the plant inspection systems 100, 400, 500, 700 are used to inspect mature green vegetable plants (e.g., larger leafy vegetable plants). Example mature green vegetable plants that may be inspected by the plant inspection systems 100, 400, 500, 700 include iceberg lettuce plants, romaine plants, butter plants, and spinach plants. In some embodiments, a plant inspection system that is configured to inspect mature green vegetable plants is similar in structure and function to any of the plant inspection systems 100, 400, 500, 700 except that the plant inspection system is modified to include cameras disposed along sides of the plant inspection system to provide images of sides of the larger plants. In addition to data related to identified contamination, camera processors of such a plant inspection system may be programmed to provide additional data related to the plants, such as sizes of the plants, locations of the plants, and maturity of the plants, which can aid in harvesting the plants. A PC of the plant inspection system may be programmed to provide additional field data related to the plants, such as a total number of plants and a density of plants. Such a plant inspection system may also include sensors (e.g., laser sensors or acoustic sensors) disposed along sides of the plant inspection system to profile sides of the plants for precise determination of center locations of the plants. In some examples, the plant inspection systems 100, 400, 500, 700 and the modified plant inspection system can be used to examine crops (e.g., broccoli plants and cauliflower plants) grown in a field and harvested selectively (e.g., individually).
While a number of examples have been described for illustration purposes, the foregoing description is not intended to limit the scope of the invention, which is defined by the scope of the appended claims. There are and will be other examples, modifications, and combinations within the scope of the following claims.
This application is a continuation of U.S. patent application Ser. No. 15/266,324, entitled “Methods And Systems For Inspecting Plants For Contamination,” filed Sep. 15, 2016, which is a continuation in part of U.S. patent application Ser. No. 15/206,753, entitled “Inspecting Plants For Contamination,” filed Jul. 11, 2016, now U.S. Pat. No. 9,928,584, the disclosures of which are incorporated herein by reference in their entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 117705 | Warren et al. | Aug 1871 | A |
| 218795 | Thompson | Aug 1879 | A |
| 811978 | Weigen | Feb 1906 | A |
| 2544023 | Johnson | Mar 1951 | A |
| 2583571 | Howe | Jan 1952 | A |
| 2825273 | Faucheux | Mar 1958 | A |
| 4519193 | Yoshida | May 1985 | A |
| 4532757 | Tutle | Aug 1985 | A |
| 4843561 | Larson | Jun 1989 | A |
| 4873644 | Fujii et al. | Oct 1989 | A |
| 4975016 | Pellenc et al. | Dec 1990 | A |
| 5054232 | Rhode | Oct 1991 | A |
| 5544474 | Finkelstein | Aug 1996 | A |
| 6919959 | Masten | Jul 2005 | B2 |
| 7215420 | Gellerman et al. | May 2007 | B2 |
| 7725233 | Hendrickson et al. | May 2010 | B2 |
| 7765780 | Koselka | Aug 2010 | B2 |
| 8194916 | Ma | Jun 2012 | B2 |
| 8272200 | Ottaway | Sep 2012 | B1 |
| 8381501 | Koselka et al. | Feb 2013 | B2 |
| 8401231 | Maxik | Mar 2013 | B2 |
| 8437498 | Malsam | May 2013 | B2 |
| 8452501 | Lange et al. | May 2013 | B1 |
| 8642910 | Pellenc et al. | Feb 2014 | B2 |
| 8841570 | Vasilescu | Sep 2014 | B2 |
| 8855412 | Quan et al. | Oct 2014 | B2 |
| 9282688 | Casper et al. | Mar 2016 | B2 |
| 9462749 | Jens et al. | Oct 2016 | B1 |
| 9468152 | Jens et al. | Oct 2016 | B1 |
| 9928584 | Jens | Mar 2018 | B2 |
| 20010016053 | Dickson | Aug 2001 | A1 |
| 20020022929 | Ell | Feb 2002 | A1 |
| 20020024665 | Masten | Feb 2002 | A1 |
| 20050126144 | Koselka et al. | Jun 2005 | A1 |
| 20080110026 | Marcoe | May 2008 | A1 |
| 20090293442 | Chang et al. | Dec 2009 | A1 |
| 20110022231 | Walker | Jan 2011 | A1 |
| 20110046785 | Zeelen | Feb 2011 | A1 |
| 20110047951 | Moore | Mar 2011 | A1 |
| 20110211733 | Schwarz | Sep 2011 | A1 |
| 20120096823 | Moore | Apr 2012 | A1 |
| 20130204437 | Koselka | Aug 2013 | A1 |
| 20130238201 | Redden | Sep 2013 | A1 |
| 20130325346 | Mcpeek et al. | Dec 2013 | A1 |
| 20140168412 | Shulman et al. | Jun 2014 | A1 |
| 20140180549 | Azte | Jun 2014 | A1 |
| 20140259898 | Jens | Sep 2014 | A1 |
| 20140260148 | Jens | Sep 2014 | A1 |
| 20140314280 | Lee | Oct 2014 | A1 |
| 20150027040 | Redden | Jan 2015 | A1 |
| 20160078304 | Bremer et al. | Mar 2016 | A1 |
| 20160309650 | Jens et al. | Oct 2016 | A1 |
| 20170061211 | Hundley | Mar 2017 | A1 |
| 20180012344 | Jens | Jan 2018 | A1 |
| 20180012347 | Jens | Jan 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 267860 | Apr 1990 | EP |
| 1002460 | Apr 2003 | EP |
| 03506779 | Mar 2004 | JP |
| 2009027977 | Feb 2009 | JP |
| 1045848 | Oct 1983 | SU |
| 1289412 | Feb 1987 | SU |
| WO1996025842 | Aug 1996 | WO |
| WO1998053961 | Dec 1998 | WO |
| WO2005092077 | Mar 2006 | WO |
| WO2007088225 | Aug 2007 | WO |
| WO2008037035 | Apr 2008 | WO |
| WO2006063314 | Apr 2009 | WO |
| WO2010063075 | Jun 2010 | WO |
| WO2011041924 | Apr 2011 | WO |
| WO2015121809 | Aug 2015 | WO |
| WO 2016171789 | Feb 2016 | WO |
| WO2016171789 | Oct 2016 | WO |
| WO2016200494 | Dec 2016 | WO |
| WO2018013216 | Jan 2017 | WO |
| Entry |
|---|
| Authorized Officer E. Eskina, International Search and Written Opinion for International Application No. PCT/US2016/018859, dated May 12, 2016, 8 pages. |
| Authorized Officer Arnaud Benedetti, International Search and Written Opinion for International Application No. PCT/US2016/028631, dated Jul. 13, 2016, 13 pages. |
| Authorized Officer T. Vladimirova, International Search and Written Opinion for International Application No. PCT/US2017/033497, dated Aug. 31, 2017, 8 pages. |
| Authorized Officer Mineko Mohri, International Preliminary Report on Patentability for International Application No. PCT/US2016/018859, dated Nov. 2, 2017, 7 pages. |
| Authorized Officer Agnes Wittmann-Regis, International Preliminary Report on Patentability for International Application No. PCT/US2016/028631, dated Dec. 21, 2017, 9 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20180260948 A1 | Sep 2018 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15266324 | Sep 2016 | US |
| Child | 15938274 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15206753 | Jul 2016 | US |
| Child | 15266324 | US |
