The present disclosure generally relates to the field of refuse vehicles. More specifically, the present disclosure relates to control systems for refuse vehicles.
One embodiment of the present disclosure relates to a refuse vehicle. The refuse vehicle includes a body, a hopper, a robotic arm implement, multiple cameras, and processing circuitry. The body defines a pair of refuse compartments. The hopper is configured to receive refuse and direct the refuse into one of the pair of refuse compartments. The cameras are positioned at the hopper. The processing circuitry is configured to obtain image data of the hopper from the cameras. The processing circuitry is also configured to detect an object to be separated from other refuse in the hopper based on the image data. The processing circuitry is also configured to operate the robotic arm implement to move the object from a first part of the hopper to a second part of the hopper for storage in one of the pair of refuse compartments separate from other refuse. The processing circuitry may be configured to obtain at least part of the image data from a camera positioned within a customer's refuse container.
Another embodiment of the present disclosure is a refuse vehicle. The refuse vehicle includes a body defining a pair of refuse compartments. The refuse vehicle includes a hopper configured to receive refuse and direct the refuse into one of the pair of refuse compartments. The refuse vehicle also includes cameras position at the hopper. The refuse vehicle also includes processing circuitry configured to obtain image data of the hopper from the cameras. The processing circuitry is also configured to detect an object to be separated from other refuse in the hopper based on the image data. The processing circuitry is also configured to operate the hopper to direct the refuse in the hopper including the object into one of the pair of refuse compartments separate from other refuse.
Another embodiment of the present disclosure is a refuse vehicle. The refuse vehicle includes a body defining a refuse compartment. The refuse vehicle also includes a hopper configured to receive refuse and direct the refuse into the refuse compartment. The refuse vehicle also includes a robotic arm implement. The refuse vehicle also includes cameras positioned at the hopper. The refuse vehicle also includes processing circuitry configured to obtain image data of the hopper from the cameras. The processing circuitry is also configured to detect an object to be separated from other refuse in the hopper based on the image data. The processing circuitry is also configured to operate the robotic arm implement to remove the object from the hopper and place the object on a ground surface at a customer's pickup location to reject the object.
Another embodiment of the present disclosure is a refuse vehicle. The refuse vehicle includes a body defining a refuse compartment. The refuse vehicle also includes a hopper configured to receive refuse and direct the refuse into the refuse compartment. The refuse vehicle also includes multiple cameras positioned at the hopper. The refuse vehicle also includes processing circuitry configured to obtain image data of the hopper from the cameras. The processing circuitry is configured to detect an object to be separated from other refuse in the refuse compartment based on the image data, and predict, based on an amount of refuse in the refuse compartment, a location of the object within the refuse compartment. The processing circuitry is also configured to, during an ejection operation of the refuse from the refuse compartment, at least one of operate an alert device to notify an operator regarding the predicted location of the object within the refuse compartment or adjust the ejection operation of the refuse from the refuse compartment to eject the object at a different location.
This summary is illustrative only and is not intended to be in any way limiting. Other aspects, inventive features, and advantages of the devices or processes described herein will become apparent in the detailed description set forth herein, taken in conjunction with the accompanying figures, wherein like reference numerals refer to like elements.
The disclosure will become more fully understood from the following detailed description, taken in conjunction with the accompanying figures, wherein like reference numerals refer to like elements, in which:
Before turning to the figures, which illustrate the exemplary embodiments in detail, it should be understood that the present application is not limited to the details or methodology set forth in the description or illustrated in the figures. It should also be understood that the terminology is for the purpose of description only and should not be regarded as limiting.
Referring generally to the FIGURES, a refuse vehicle may include a single refuse compartment or multiple refuse compartments. The refuse vehicle can include a hopper that is configured to deliver refuse emptied into the hopper to one of the refuse compartments or into the refuse compartment, depending on the configuration of the refuse vehicle (e.g., if the refuse vehicle has a single refuse compartment or multiple refuse compartments). The refuse vehicle can also include cameras configured to obtain image data of the hopper including the refuse that is currently emptied into the hopper. The refuse vehicle may include processing circuitry configured to obtain the image data and detect one or more objects that should be separated from common or regular refuse. The processing circuitry can also receive image data from a camera positioned within a customer's refuse container and perform the image analysis to detect the one or more objects based on the image data from the camera of the customer's refuse container. The processing circuitry may operate a robotic arm implement to move the detected object to be stored in a separate one of the refuse compartments or to remove the object from the hopper and return the object to the customer's pickup location (e.g., reject the object).
Referring to
According to an alternative embodiment, the engine 18 additionally or alternatively includes one or more electric motors coupled to the frame 12 (e.g., a hybrid refuse vehicle, an electric refuse vehicle, etc.). The electric motors may consume electrical power from any of an on-board storage device (e.g., batteries, ultra-capacitors, etc.), from an on-board generator (e.g., an internal combustion engine, etc.), or from an external power source (e.g., overhead power lines, etc.) and provide power to the systems of the refuse vehicle 10. The engine 18 may transfer output torque to or drive the tractive elements 20 (e.g., wheels, wheel assemblies, etc.) of the refuse vehicle 10 through a transmission 22. The engine 18, the transmission 22, and one or more shafts, axles, gearboxes, etc., may define a driveline of the refuse vehicle 10.
According to an exemplary embodiment, the refuse vehicle 10 is configured to transport refuse from various waste receptacles within a municipality to a storage and/or processing facility (e.g., a landfill, an incineration facility, a recycling facility, etc.). As shown in
The tailgate 34 may be hingedly or pivotally coupled with the body 14 at a rear end of the body 14 (e.g., opposite the cab 16). The tailgate 34 may be driven to rotate between an open position and a closed position by tailgate actuators 24. The refuse compartment 30 may be hingedly or pivotally coupled with the frame 12 such that the refuse compartment 30 can be driven to raise or lower while the tailgate 34 is open in order to dump contents of the refuse compartment 30 at a landfill. The refuse compartment 30 may include a packer assembly (e.g., a compaction apparatus) positioned therein that is configured to compact loose refuse.
Referring still to
As shown in
Referring to
Referring still to
Referring to
The controller 102 includes processing circuitry 104 including a processor 106 and memory 108. Processing circuitry 104 can be communicably connected with a communications interface of controller 102 such that processing circuitry 104 and the various components thereof can send and receive data via the communications interface. Processor 106 can be implemented as a general purpose processor, an application specific integrated circuit (ASIC), one or more field programmable gate arrays (FPGAs), a group of processing components, or other suitable electronic processing components.
Memory 108 (e.g., memory, memory unit, storage device, etc.) can include one or more devices (e.g., RAM, ROM, Flash memory, hard disk storage, etc.) for storing data and/or computer code for completing or facilitating the various processes, layers and modules described in the present application. Memory 108 can be or include volatile memory or non-volatile memory. Memory 108 can include database components, object code components, script components, or any other type of information structure for supporting the various activities and information structures described in the present application. According to some embodiments, memory 108 is communicably connected to processor 106 via processing circuitry 104 and includes computer code for executing (e.g., by at least one of processing circuitry 104 or processor 106) one or more processes described herein.
The controller 102 is configured to receive inputs (e.g., measurements, detections, signals, sensor data, etc.) from the input devices 150, according to some embodiments. In particular, the controller 102 may receive a GPS location from the GPS system 124 (e.g., current latitude and longitude of the refuse vehicle 10). The controller 102 may receive sensor data (e.g., engine temperature, fuel levels, transmission control unit feedback, engine control unit feedback, speed of the refuse vehicle 10, etc.) from the sensors 126. The controller 102 may receive image data (e.g., real-time camera data) from the vision system 128 of an area of the refuse vehicle 10 (e.g., in front of the refuse vehicle 10, rearwards of the refuse vehicle 10, on a street-side or curb-side of the refuse vehicle 10, at the hopper of the refuse vehicle 10 to monitor refuse that is loaded, within the cab 16 of the refuse vehicle 10, etc.). The controller 102 may receive user inputs from the HMI 130 (e.g., button presses, requests to perform a lifting or loading operation, driving operations, steering operations, braking operations, etc.).
The controller 102 may be configured to provide control outputs (e.g., control decisions, control signals, etc.) to the driveline 110 (e.g., the engine 18, the transmission 22, the engine control unit, the transmission control unit, etc.) to operate the driveline 110 to transport the refuse vehicle 10. The controller 102 may also be configured to provide control outputs to the braking system 112 to activate and operate the braking system 112 to decelerate the refuse vehicle 10 (e.g., by activating a friction brake system, a regenerative braking system, etc.). The controller 102 may be configured to provide control outputs to the steering system 114 to operate the steering system 114 to rotate or turn at least two of the tractive elements 20 to steer the refuse vehicle 10. The controller 102 may also be configured to operate actuators or motors of the lift apparatus 116 (e.g., lift arm actuators 44) to perform a lifting operation (e.g., to grasp, lift, empty, and return a refuse container). The controller 102 may also be configured to operate the compaction system 118 to compact or pack refuse that is within the refuse compartment 30. The controller 102 may also be configured to operate the body actuators 120 to implement a dumping operation of refuse from the refuse compartment 30 (e.g., driving the refuse compartment 30 to rotate to dump refuse at a landfill). The controller 102 may also be configured to operate the alert system 122 (e.g., lights, speakers, display screens, etc.) to provide one or more aural or visual alerts to nearby individuals.
The controller 102 may also be configured to receive feedback from any of the driveline 110, the braking system 112, the steering system 114, the lift apparatus 116, the compaction system 118, the body actuators 120, or the alert system 122. The controller may provide any of the feedback to the remote computing system 134 via the telematics unit 132. The telematics unit 132 may include any wireless transceiver, cellular dongle, communications radios, antennas, etc., to establish wireless communication with the remote computing system 134. The telematics unit 132 may facilitate communications with telematics units 132 of nearby refuse vehicles 10 to thereby establish a mesh network of refuse vehicles 10.
The controller 102 is configured to use any of the inputs from any of the GPS 124, the sensors 126, the vision system 128, or the HMI 130 to generate controls for the driveline 110, the braking system 112, the steering system 114, the lift apparatus 116, the compaction system 118, the body actuators 120, or the alert system 122. In some embodiments, the controller 102 is configured to operate the driveline 110, the braking system 112, the steering system 114, the lift apparatus 116, the compaction system 118, the body actuators 120, and/or the alert system 122 to autonomously transport the refuse vehicle 10 along a route (e.g., self-driving), perform pickups or refuse collection operations autonomously, and transport to a landfill to empty contents of the refuse compartment 30. The controller 102 may receive one or more inputs from the remote computing system 134 such as route data, indications of pickup locations along the route, route updates, customer information, pickup types, etc. The controller 102 may use the inputs from the remote computing system 134 to autonomously transport the refuse vehicle 10 along the route and/or to perform the various operations along the route (e.g., picking up and emptying refuse containers, providing alerts to nearby individuals, limiting pickup operations until an individual has moved out of the way, etc.).
In some embodiments, the remote computing system 134 is configured to interact with (e.g., control, monitor, etc.) the refuse vehicle 10 through a virtual refuse truck as described in U.S. application Ser. No. 16/789,962, now U.S. Pat. No. 11,380,145, filed Feb. 13, 2020, the entire disclosure of which is incorporated by reference herein. The remote computing system 134 may perform any of the route planning techniques as described in greater detail in U.S. application Ser. No. 18/111,137, filed Feb. 17, 2023, the entire disclosure of which is incorporated by reference herein. The remote computing system 134 may implement any route planning techniques based on data received by the controller 102. In some embodiments, the controller 102 is configured to implement any of the cart alignment techniques as described in U.S. application Ser. No. 18/242,224, filed Sep. 5, 2023, the entire disclosure of which is incorporated by reference herein. The refuse vehicle 10 and the remote computing system 134 may also operate or implement geofences as described in greater detail in U.S. application Ser. No. 17/232,855, filed Apr. 16, 2021, the entire disclosure of which is incorporated by reference herein.
Referring to
Referring to
Referring particularly to
The refuse vehicle 10 may be a multi-chambered vehicle including at least two refuse compartments, each for a different type of refuse, or may be a single-chambered vehicle including a single refuse compartment for one type of refuse. For example, the refuse vehicle 10 may be a garbage truck or a recycling truck with a single corresponding refuse compartment for garbage or recycling. The refuse vehicle 10 can also be a combination of a garbage truck and a recycling truck so that the refuse vehicle 10 can collect both garbage and recycling. If the refuse vehicle 10 is configured to collect both garbage and recycling (e.g., different types of refuse), the refuse vehicle 10 may include a hopper 60 having two hopper openings for the different types of refuse, shown in
The stream sorting system 400 can also include a radio frequency identification (RFID) detector 404 (e.g., an RFID reader, an RFID transponder, etc.) that is configured to wirelessly transmit energy to proximate RFID tags. The RFID detector 404 may monitor responses that are received from nearby RFID tags. In particular, the refuse containers 200 may include RFID tags that are configured to provide a response signal to the RFID detector 404. The RFID detector 404 may be positioned on the lift apparatus 116 or on an exterior surface of the refuse vehicle 10 such that the RFID detector 404 is sufficiently close to the refuse containers 200 to communicate with RFID tags. The stream sorting system 400 may also include separate QR code scanners or barcode scanners positioned on the lift apparatus 116 (e.g., on the grabber assembly 52) or on a side of the refuse vehicle 10.
Referring particularly to
Referring to
As shown in
The image analysis 406 may also identify the text 214 on the refuse container 200. For example, the text 214 on the refuse container may indicate whether the refuse container 200 is a garbage can or a recycling can. The image analysis 406 may perform an optical character recognition (OCR) technique and use a database of terms to identify, based on the text 214, whether the refuse container 200 is a recycling container or a garbage can (e.g., to predict the type of refuse that is within the refuse container 200).
The image analysis 406 may be configured to implement any machine learning, neural network, or artificial intelligence in order to identify whether the refuse container 200 is a garbage can or a recycling container (e.g., to predict a type of refuse within the refuse container 200). For example, the controller 102 may implement the image analysis 406 by performing any of the functionality as described in greater detail in U.S. application Ser. No. 16/758,834, filed Apr. 23, 2020, the entire disclosure of which is incorporated by reference herein. The image analysis 406 may be implemented locally on the controller 102 or remotely by the remote computing system 134. The controller 102, or more generally, the stream sorting system 400, may also be configured to perform any of the functions or techniques to identify the type of refuse in the refuse container 200 or identify the type of refuse container 200 as described in greater detail in U.S. application Ser. No. 17/189,740, filed Mar. 2, 2021, the entire disclosure of which is incorporated by reference herein.
The decoder 408 can also be configured to obtain the image data from the cameras 402 and identify QR codes or bar codes in the image data of the refuse container 200. The decoder 408 may implement a QR code or barcode decoding technique based on the image data of the QR code 212 or the barcode 210 to determine results. The results may indicate a type of refuse container 200 that is present in the image data (e.g., a garbage can or a recycling bin).
The RFID manager 410 is configured to receive the RFID response from the RFID scanner 404 obtained from the RFID tag 216 of the refuse container 200. The RFID manager 410 may analyze the RFID response to determine, based on the RFID response, the type of refuse that should be present in the refuse container 200.
The image analysis 406, the decoder 408, and the RFID manager 410 may be configured to determine, using the above described functionality, a predicted type of refuse that is present in the refuse container 200 and provide their results to the pickup planner 412, the control manager 414, and the display manager 416. The pickup planner 412 may determine, based on the type of refuse in the refuse container 200 and abilities of the refuse vehicle 10 (e.g., the type of refuse that the refuse vehicle 10 is capable of collecting, whether the refuse vehicle 10 has multiple refuse compartments, a fill level of refuse in the refuse compartments, etc.), if the refuse vehicle 10 is capable of emptying the refuse of the refuse container 200 into the hopper 60. If the refuse vehicle 10 is not capable of emptying the refuse of the refuse container 200, the pickup planner 412 may use the GPS location as the container location, and the type of refuse in the refuse container to schedule a pickup by a subsequent refuse vehicle. The pickup planner 412 can provide the container location and the type of refuse in the refuse container 200 to the remote computing system 134 for scheduled pickup at a later time via the telematics 132.
The control manager 414 is configured to use the results of any of the image analysis 406, the decoder 408, or the RFID manager 410 to operate at least one of the lift apparatus 116 or the hopper actuator 418 to empty the contents of the refuse container 200 into the hopper 60 of the refuse vehicle 10. For example, if the refuse vehicle 10 is a single compartment refuse vehicle that is configured to collect the type of refuse that is within the refuse container 200, the control manager 414 may provide control signals to the lift apparatus 116 to grasp, lift, and empty the refuse of the refuse container 200 into the hopper 60. Similarly, if the refuse vehicle 10 is a multi-compartment refuse vehicle, that control manager 414 may operate the hopper actuator 418 to transition the partition 66 into a desired state and operate the lift apparatus 116 to empty contents of the refuse container 200 into the corresponding type of refuse compartment of the refuse vehicle 10 (e.g., the garbage compartment if the type of refuse in the refuse container 200 is garbage or the recycling compartment if the type of refuse in the refuse container 200 is recycling). In some embodiments, the control manager 414 is configured to operate the lift apparatus 116 without operating the hopper actuator 418 to empty the contents of the refuse container 200 into the appropriate hopper opening of the hopper 60 based on the type of refuse in the refuse container 200.
Referring to
It should be understood that while
Referring to
The process 500 includes obtaining sensor data of a curbside refuse container for pickup (step 502), according to some embodiments. The refuse container may be position on a curb, in front of a refuse vehicle, behind a refuse vehicle, etc. The sensor data may include image data or a reply signal from an RFID tag of the refuse container. The sensor data can be obtained from sensors that are disposed on an exterior of the refuse vehicle. For example, the refuse vehicle can include cameras positioned about a body or cabin of the refuse vehicle that have a field of view pointing outwards from the refuse vehicle towards a location where refuse containers are expected to be located for pickup. Step 502 can be performed by the controller 102 or any other processing unit of the stream sorting system 400 by receiving image data from the cameras 402, RFID responses from the RFID detector 404, etc.
The process 500 includes inferring a type of refuse in the refuse container based on the sensor data (step 504), according to some embodiments. Step 504 can include performing an image analysis or image recognition technique based on features, colors, sizes, shapes, etc., of the refuse container in the image data. Step 504 can also include performing decoding techniques to decode the information contained in a QR cod or a barcode. Step 504 can also include analyzing a response from an RFID tag of the refuse container. Step 504 may be performed by the controller 102 by implementing at least one of the image analysis 406, the decoder 408, or the RFID manager 410.
The process 500 includes identifying if the refuse vehicle is a multi-compartment refuse vehicle (step 506), according to some embodiments. Step 506 may be performed based on known characteristics of the refuse vehicle. Step 506 can be performed by the controller 102. If the refuse vehicle is a multi-compartment refuse vehicle (step 506, “YES”), process 500 may proceed to step 508. If the refuse vehicle is not a multi-compartment refuse vehicle (step 506, “NO”), process 500 proceeds to step 510.
The process 500 includes operating at least one of a lift apparatus or a divider actuator to empty the refuse of the refuse container into one of the compartments of the refuse vehicle (step 508), according to some embodiments. Step 508 may include operating the lift apparatus 116 (e.g., a robotic arm, a side loading arm of a refuse vehicle, etc.) and the actuator 418 in order to empty the contents of the refuse container into one of the first compartment 68 or the second compartment 70 through the hopper 60. Step 508 may include operating the lift apparatus 116 to empty contents of the refuse container into a hopper opening corresponding to the desired compartment without requiring operation of the divider actuator if the lift apparatus 116 is operably capable of lifting the refuse container to different locations of the hopper. Step 508 can be performed by the controller 102 by generating and providing controls to the lift apparatus 116 and the actuator 418. Step 508 may be performed if the refuse vehicle is a multi-compartment refuse vehicle and has sufficient storage space in a target compartment for the contents of the refuse container.
The process 500 includes identifying if the type of refuse inferred at step 504 is supported by the refuse vehicle (step 510) if the refuse vehicle is not a multi-compartment refuse vehicle (step 506, “NO”), according to some embodiments. Step 510 may include using stored data regarding whether the refuse vehicle is intended to collect garbage or recycling or whether the refuse vehicle is currently collecting garbage, recycling, or a different type of refuse. Step 510 can be performed by the controller 102 based on known characteristics or current route of the refuse vehicle. Step 510 can include comparing the type of refuse that the refuse vehicle is intended to collect to the type of refuse that is inferred to be present within the refuse container. If the type of refuse that the refuse vehicle is intended to collect matches the type of refuse that is inferred to be present in the refuse container (step 510, “YES”), process 500 proceeds to step 512. If the type of refuse that the refuse vehicle is intended to collect does not match the type of refuse that is inferred to be present in the refuse container (e.g., the refuse vehicle is a garbage truck and the refuse container is a recycling bin) (step 510, “NO”), process 500 proceeds to step 514.
The process 500 includes operating the lift apparatus to empty the refuse into the refuse vehicle (step 512) in response to determining that the type of refuse inferred to be present in the refuse container matches or is supported by the refuse vehicle (step 510, “YES”), according to some embodiments. Step 512 can be performed by the controller 102 by operating the lift apparatus 116 to lift and empty contents of the refuse container into a hopper of the refuse vehicle 10.
The process 500 includes scheduling subsequent pickup of the refuse container (step 514) if the type of refuse inferred to be present in the refuse container does not match or is not supported by the refuse vehicle (step 510, “NO”), according to some embodiments. Step 510 can be performed by the controller 102 (e.g., by the pickup planner 412) by providing type of the container (e.g., type of the refuse inferred to be present in the refuse container) and a location (e.g., a GPS location) of the container to a remote computing system (e.g., a fleet management system, a route planning system, etc.) such as the remote computing system 134. The remote computing system can use the indication of the type of the container or refuse and the location to schedule pickup by an appropriate refuse vehicle. For example, if the refuse vehicle is a recycling truck and infers that the refuse container is a garbage can, the remote computing system may schedule and dispatch collection of the contents of the garbage can based on the type of the container and the location provided by the controller 102.
Referring to
Referring particularly to
The hopper opening 62 may feed into the first refuse compartment 30a such that refuse emptied into the first hopper opening 62 is directed to the first refuse compartment 30a, while the hopper opening 64 feeds into the second refuse compartment 30b that is separate from the first refuse compartment such that refuse emptied into the second hopper opening 64 feeds into the second refuse compartment 30b. The robotic arm implement 700 is configured to be operated to grasp an object (e.g., a television, an appliance, etc.), lift the object out of the hopper opening 62 and place the object in the hopper opening 64. The hopper opening 62 and the hopper opening 64 are defined by divider 66 that extends between the hopper opening 62 and the hopper opening 64. The robotic arm implement 700 is configured to move objects between either side of the divider 66, based on the image data of the cameras 1002, from the hopper opening 62 to the hopper opening 64 in order to separate certain types of objects from normal refuse. For example, the second refuse compartment 30b may be provided for hazardous materials or materials that should be kept separate from regular refuse, while the first refuse compartment 30a is provided for regular refuse. The first refuse compartment 30a and the second refuse compartment 30b may be the same size or may be different sizes (e.g., as shown in
Referring to
Referring to
Referring to
Referring to
Referring to
Referring to
The second boom section 724 is pivotally coupled with an end of the inner member 716 at pivot point 726. The second boom section 724 may be driven to rotate about the pivot point 726 by extension or retraction of actuator 728. The second boom section 724 can be rotatably coupled with an end portion 730. The end portion 730 may be driven to rotate relative to the second boom section 724 about an axis extending lengthwise through the second boom section 724 by a rotary actuator 732. The rotary actuator 732 may be similar to the rotary actuator 708. The robotic arm implement 700 may also include a claw assembly 740 that is coupled with the second boom section 724. The claw assembly 740 may include a carriage 736 that is pivotally coupled with the end portion 730 and is configured to be driven to rotate (e.g., pivot, hinge, swivel, etc.) relative to the end portion 730 by actuators 734. The claw assembly 740 can include multiple fingers, shown as fingers 738 each including articulable sections pivotally coupled with each other and driven to articulate to grasp or release an object by operation of corresponding actuators.
Referring to
The camera 402 of the refuse containers 200 can be configured to obtain image data as people place refuse into the refuse containers 200. For example, the individual or customer may place tires 902 at a bottom of the refuse container 200 and then cover the tires 902 (or other restricted objects such as cans of car oil, appliances, batteries, etc.) with garbage bags 906. The camera 402 may be configured to obtain image data of the tires 902 when the tires 902 or other restricted object are first placed in the refuse containers 200. The image data of the interior of the refuse container 200 that is obtained by the camera 402 may be stored in memory of a controller of the control unit 422 and transmitted (e.g., to a controller of the refuse vehicle 10, to the remote computing system 134, etc.) wirelessly via the wireless transceiver.
Referring to
The memory 108 of the controller 102 may include an object detector 1006, a control manager 1008, a display manager 1010, and a packing tracker 1012. The object detector 1006 may use similar techniques as the image analysis 406 described in greater detail above but in order to detect different types or classes of objects that should be separated from ordinary refuse. For example, the object detector 1006 may be implemented to perform similar image analysis or object detection techniques in order to detect chemicals, canisters, tanks, appliances, tires, oil cans, etc., or any other object that should be separated from regular refuse. The object detector 1006 is configured to provide results to the control manager 1008 or the display manager 1010.
The control manager 1008 is configured to use the results of the object detector 1006 to determine controls for the hopper actuator(s) 418, the lift apparatus 116 (e.g., the lift assemblies 50, the grabber assemblies 52, the lift assembly 40, etc.), and the robotic arm implement 700. The control manager 1008 may operate any of the hopper actuators 418, the lift apparatus 116, or the robotic arm implement(s) 700 as available on the refuse vehicle 10 to either move objects from one hopper opening 62 to another hopper opening 64 (e.g., from one side of the partition 66 to another side of the partition 66). For example, the control manager 1008 may operate the robotic arm implement 700 coordinated with operation of the hopper actuators 418 to lift a detected object from one side of the partition 66 to another side of the partition 66 for storage in a different refuse compartment 30. The control manager 1008 may also operate the robotic arm implement 700 to remove detected objects from the hopper 60 and place the objects back onto a ground surface in cases where the refuse vehicle 10 only includes a single refuse compartment 30, or if the refuse compartment 30 designated for that type of object is full. The control manager 1008 can be configured to use the results of the object detector 1006 to target the object within the hopper 60 or within the intermediate container 800 and operate the robotic arm implement 700 to grasp the object and move the object to a desired location (e.g., to a different hopper, back to the ground, etc.). In some implementations of the refuse vehicle 10, the refuse vehicle 10 does not include the robotic arm implement 700, and the entire contents of the refuse container 200 are directed into the refuse compartment 30 by operation of the hopper actuators 418 in response to detecting an object in the contents of the refuse container 200 that should be kept separate from ordinary or regular refuse.
The display manager 1010 can be configured to generate display data for the display device 420 indicating a type of object detected and a location in the hopper 60 or the intermediate container 800 that the object is detected. The display manager 1010 can be configured to notify the operator of the refuse vehicle 10 regarding the detected object and provide a GUI that includes the image data of the cameras 402 or the cameras 1002 with the detected object highlighted. The controller 102 may also transmit, to the remote computing system 134, via the telematics 132, the customer location (e.g., the container location) and the object detected in the refuse container 200. The remote computing system 134 can be configured to use the received information in order to either charge the customer additional charges for the pickup of the detected object (e.g., additional charge for picking up and disposal of a freezer) or in order to provide training to the customer regarding which objects can be disposed of in the refuse container 200.
The packing tracker 1012 may be configured to use the results of the object detector 1006 and the image data in order to track a location within the refuse compartment 30 of one or more of the detected objects. For example, the packing tracker 1012 can be configured to estimate or monitor an amount of refuse that is loaded into the refuse compartment 30 based on image data provided by the cameras 1002 and the cameras 402. The packing tracker 1012 can also be configured to monitor points in time (e.g., based on fill amount of the refuse container 30) at which a detected object as provided by the object detector 1006 is loaded into the refuse compartment 30. The packing tracker 1012 may determine (e.g., estimate) a location within the refuse compartment 30 of the detected object (e.g., the hazardous or quarantine object) based on operations of a packer assembly 430 within the refuse compartment 30 that compacts the refuse. The packing tracker 1012 can also use feedback from a hopper fill sensor to identify a relative point at which the detected object is loaded in the refuse compartment 30 and predict a location within the refuse compartment 30 of the detected object. The packing tracker 1012 can be configured to predict the location of the detected object within the refuse compartment 30 by using the results of the object detector 1006 in combination with any of the functionality as described in greater detail with reference to: U.S. application Ser. No. 10/943,182, filed Sep. 16, 2004, issued as U.S. Pat. No. 7,108,473; U.S. application Ser. No. 10/389,195, filed Mar. 14, 2003, issued as U.S. Pat. No. 6,799,934; or U.S. application Ser. No. 18/342,343, filed Jun. 27, 2023, the entire disclosures of which are all incorporated by reference herein in their entireties. The object detector 1006 may have a list of objects (e.g., a database of detectable objects) that the object detector 1006 is configured to identify in the image data as well as a corresponding type (e.g., whether the detected object should be separated from other refuse).
The detection and sorting system 1000 can also be configured to achieve the detection of objects (e.g., hazardous objects, quarantine objects, objects of a particular type that should be kept separate from regular refuse) and operate to either move the detected objects into separate compartments, track the detected objects within the refuse compartment 30, or reject the objects by implementing any of the functionality as described in greater detail in U.S. application Ser. No. 16/758,834, filed Apr. 23, 2020, issued as U.S. Pat. No. 11,527,072, or U.S. application Ser. No. 16/851,196, filed Apr. 17, 2020, issued as U.S. Pat. No. 11,673,563, the entire disclosures of which are all incorporated by reference herein in their entireties.
The packing track 1012 can be configured to operate the packer 430 and lift cylinders 432 of the refuse vehicle to perform a dumping or ejection operation by raising the body 14 of the refuse vehicle 10 relative to the chassis 12 (e.g., operating the lift cylinders 432), and operating the packer 430 to drive discharge of the refuse in the refuse compartment 30. The packing tracker 1012 may operate the display device 420 to notify the operator regarding the location of the detected and tracked objects in the ejected load. The packing tracker 1012 can also implement autonomous or semi-autonomous transportation and ejection of portions of the load in the refuse compartment 30 at a first location, and portions of the load in the refuse compartment 30 where the detected object is predicted to be present at a second location.
Referring to
The process 1100 includes obtaining sensor data that indicates a presence of an object in a quantity of refuse (step 1102), according to some embodiments. Step 1102 can be performed by obtaining image data from the cameras 1002 or the cameras 402 of the refuse vehicle 10. Step 1102 can also include wirelessly receiving image data from a camera that is positioned at the customer's refuse container. The image data may be image data of a hopper, the interior of the customer's refuse container, or an interior of an intermediate hopper or container.
The process 1100 includes detecting the object based on the sensor data (step 1104), according to some embodiments. Step 1104 can be implemented by the controller 102 by using a machine learning, artificial intelligence, or neural network technique to detect the object from a library of different objects. Step 1104 may include additionally identifying a position of the object relative to a position or location of the camera that obtains the image data.
The process 1100 includes operating a system of the refuse vehicle to move the detected object or the quantity of refuse from other refuse of a refuse compartment of the refuse vehicle (step 1106), according to some embodiments. Step 1106 can be performed by the controller 102 by operating the hopper actuators 418 to direct the entire load (e.g., the quantity) of refuse from the customer's refuse container into a separate refuse compartment (e.g., the refuse compartment 30b). Step 1106 can also be performed by the controller 102 by operating the robotic arm implement 700 based on the image data of the detected object to remove the detected object from the quantity of refuse of the customer's refuse container and to place the object into a separate hopper location or into a separate refuse compartment (e.g., the refuse compartment 30b). Step 1106 can also be performed by the controller 102 by operating the robotic arm implement 700 to remove the detected object from the hopper 60 of the refuse vehicle 10 and return the object to the customer's pickup location (e.g., rejecting the pickup of the object).
Referring to
The process 1200 includes steps 1102-1104 that are substantially the same as steps 1102-1104 of process 1100. The process 1200 also includes predicting a location of the object in a refuse compartment of the refuse vehicle (step 1206), according to some embodiments. Step 1206 may be performed by the controller 102 by implementing the functionality of the packing tracker 1012. In some embodiments, step 1206 includes using various sensor data that indicates a quantity of refuse that has been packed into the refuse compartment in combination with a point at which the detected object is packed into the refuse compartment. For example, if the detected object is packed into the refuse compartment at the refuse vehicle's last pickup, the detected object may be predicted to be at distal end of the refuse compartment. Similarly, if the detected object is packed into the refuse compartment at the refuse vehicle's first pickup, the detected object may be predicted to be at a proximate end of the refuse compartment.
The process 1200 includes operating a display screen to notify an operator of the refuse vehicle regarding the predicted location of the object (step 1208), according to some embodiments. Step 1208 may be performed by the controller 102 by implementing the functionality of the display manager 1010 or the packing tracker 1012 in order to display a graphical representation of the load of the refuse compartment and the predicted location of the object within the load of the refuse compartment.
The process 1200 includes adjusting a discharge operation of the refuse vehicle based on the predicted location of the detected object in the refuse compartment (step 1210), according to some embodiments. Step 1210 can include autonomously or semi-autonomously ejecting portions of the loaded refuse in the refuse compartment to different locations. For example, portions of the refuse compartment where the object is predicted to be located may be ejected at a separate location. In some embodiments, step 1210 includes operating a display device to notify an operator when the detected object is being ejected. Step 1210 may also include transmitting a signal to a nearby robotic arm implement that is autonomously or manually controlled in order to grab and move the detected object after is it ejected from the refuse vehicle 10.
The present disclosure contemplates methods, systems, and program products on any machine-readable media for accomplishing various operations. The embodiments of the present disclosure may be implemented using existing computer processors, or by a special purpose computer processor for an appropriate system, incorporated for this or another purpose, or by a hardwired system. Embodiments within the scope of the present disclosure include program products comprising machine-readable media for carrying or having machine-executable instructions or data structures stored thereon. Such machine-readable media can be any available media that can be accessed by a general purpose or special purpose computer or other machine with a processor. By way of example, such machine-readable media can comprise RAM, ROM, EPROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to carry or store desired program code in the form of machine-executable instructions or data structures and which can be accessed by a general purpose or special purpose computer or other machine with a processor. When information is transferred or provided over a network or another communications connection (either hardwired, wireless, or a combination of hardwired or wireless) to a machine, the machine properly views the connection as a machine-readable medium. Thus, any such connection is properly termed a machine-readable medium. Combinations of the above are also included within the scope of machine-readable media. Machine-executable instructions include, for example, instructions and data which cause a general purpose computer, special purpose computer, or special purpose processing machines to perform a certain function or group of functions.
As utilized herein, the terms “approximately,” “about,” “substantially,” and similar terms are intended to have a broad meaning in harmony with the common and accepted usage by those of ordinary skill in the art to which the subject matter of this disclosure pertains. It should be understood by those of skill in the art who review this disclosure that these terms are intended to allow a description of certain features described and claimed without restricting the scope of these features to the precise numerical ranges provided. Accordingly, these terms should be interpreted as indicating that insubstantial or inconsequential modifications or alterations of the subject matter described and claimed are considered to be within the scope of the invention as recited in the appended claims.
It should be noted that the terms “exemplary” and “example” as used herein to describe various embodiments is intended to indicate that such embodiments are possible examples, representations, and/or illustrations of possible embodiments (and such term is not intended to connote that such embodiments are necessarily extraordinary or superlative examples).
The terms “coupled,” “connected,” and the like, as used herein, mean the joining of two members directly or indirectly to one another. Such joining may be stationary (e.g., permanent, etc.) or moveable (e.g., removable, releasable, etc.). Such joining may be achieved with the two members or the two members and any additional intermediate members being integrally formed as a single unitary body with one another or with the two members or the two members and any additional intermediate members being attached to one another.
References herein to the positions of elements (e.g., “top,” “bottom,” “above,” “below,” “between,” etc.) are merely used to describe the orientation of various elements in the figures. It should be noted that the orientation of various elements may differ according to other exemplary embodiments, and that such variations are intended to be encompassed by the present disclosure.
Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list. Conjunctive language such as the phrase “at least one of X, Y, and Z,” unless specifically stated otherwise, is otherwise understood with the context as used in general to convey that an item, term, etc. may be either X, Y, Z, X and Y, X and Z, Y and Z, or X, Y, and Z (i.e., any combination of X, Y, and Z). Thus, such conjunctive language is not generally intended to imply that certain embodiments require at least one of X, at least one of Y, and at least one of Z to each be present, unless otherwise indicated.
It is important to note that the construction and arrangement of the systems as shown in the exemplary embodiments is illustrative only. Although only a few embodiments of the present disclosure have been described in detail, those skilled in the art who review this disclosure will readily appreciate that many modifications are possible (e.g., variations in sizes, dimensions, structures, shapes and proportions of the various elements, values of parameters, mounting arrangements, use of materials, colors, orientations, etc.) without materially departing from the novel teachings and advantages of the subject matter recited. For example, elements shown as integrally formed may be constructed of multiple parts or elements. It should be noted that the elements and/or assemblies of the components described herein may be constructed from any of a wide variety of materials that provide sufficient strength or durability, in any of a wide variety of colors, textures, and combinations. Accordingly, all such modifications are intended to be included within the scope of the present inventions. Other substitutions, modifications, changes, and omissions may be made in the design, operating conditions, and arrangement of the preferred and other exemplary embodiments without departing from scope of the present disclosure or from the spirit of the appended claims.
This application claims the benefit of and priority to U.S. Provisional Application No. 63/545,990, filed Oct. 27, 2023, the entire disclosure of which is incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 63545990 | Oct 2023 | US |