Patent Application
20250171227
Various example embodiments relate generally to apparatus and methods for collecting waste, managing the waste being collected and analyzing the waste which has been collected.
Waste collection is commonly organized in waste collection tours run by waste collection vehicles. During a waste collection tour, a waste collection vehicle collects waste at several waste collection points. A waste collection tour is often dedicated to a type of waste, for example glasses, selected waste, residual waste etc . . . . Some waste collection vehicles are equipped to run two waste collection tours in parallel for example a tour dedicated to selected waste and a tour dedicated to residual waste.
There is a need for waste collection companies and their customers to obtain added value information about the waste collected during a waste collection tour. For example, such information can be used to take improvement actions impacting the sorting quality by the end user.
According to some aspects, there is provided the subject matter of the independent claims. Some further aspects are defined in the dependent claims.
In a first aspect, a waste management apparatus is disclosed. The disclosed apparatus is to be mounted on a waste collection vehicle equipped with at least one optical sensor configured to acquire images of waste entering the waste collection vehicle, the waste management apparatus comprising at least one memory and at least one processor configured to process at least part of the acquired images to: detect one or more objects, for each object detected in a given image, provide at least a class in a classification of objects, determine, amongst the detected objects, new objects entering the waste collection vehicle, and send to a remote server information about the new objects, comprising at least the class of the new object, a time and a location of the waste collection vehicle when the new object entered the waste collection vehicle.
In an embodiment, the disclosed waste management apparatus is configured to control the at least one optical sensor to acquire images continuously while the waste collection vehicle is in service.
In an embodiment, the disclosed waste management apparatus is configured to determine if the waste collection vehicle is in motion and disregard new objects identified from images acquired while the waste collection vehicle is in motion.
In an embodiment, of the disclosed waste management apparatus, the object classification relates to sorting rules so that a type of waste collection tour can be determined from the classes of the objects collected during a given waste collection tour.
In an embodiment, the disclosed waste collection management apparatus is to be mounted on a waste collection vehicle arranged to collect waste for at least two types of waste collection tours and it is configured to determine, from the acquired images, the type of waste collection tour associated with the new objects, and to include an indication of the type of waste collection tour in the information sent to the remote server.
In an embodiment, the disclosed waste management apparatus is configured to determine from the acquired images a type of entrance into the waste collection vehicle amongst at least the following: manual entrance, thrown from a bin lifted by the waste collection vehicle, thrown from a bin lifted manually or dumped from a container of a sorting terminal, and to include said type of entrance in the information sent to the remote server.
In an embodiment, the disclosed waste management apparatus is configured to provide, for each object detected in a given image, a position in said given image. And determining new objects entering the waste collection vehicle comprises: storing, in a memory, records corresponding to objects detected in previous images, a given record comprising at least the position of the given object in the given previous image and information descriptive of the given object; for each detected object in the current image, comparing the position and the descriptive information of the detected object with the records in the memory, when no match is found determine that the detected object is a new object; and storing in the memory a new record corresponding to the detected object in the current image.
In an embodiment of the disclosed waste management apparatus, the comparison of the position and the descriptive information of the detected object with the records in the memory takes into account an estimation of movement of the detected object.
In an embodiment of the disclosed waste management apparatus, the acquired images are processed through a machine learning module, the machine learning module detecting objects in a given image, and providing for each detected object, at least the class of the detected object and an object identifier that uniquely identifies the detected object, an object being determined as a new object when the object identifier is different from the identifiers of the objects detected in previous images.
In a second aspect, a waste collection vehicle is disclosed which comprises a waste management apparatus as disclosed in any combination of the above mentioned embodiments.
In a third aspect, a computer implemented method for managing waste at a waste collection vehicle. The disclosed method comprises at least acquiring images of waste entering the waste collection vehicle and processing the acquired images to: detect one or more objects; for each object detected in a given image, provide at least a position of the object in said given image and an object class in a classification of objects; Identify, amongst the detected objects, new objects entering the waste collection vehicle; and send to a remote server information about the new objects, comprising at least the class of the new object, a time and a location of the waste collection vehicle when the new object entered the waste collection vehicle.
In a fourth aspect, a computer program product is disclosed comprising instructions which when executed by an apparatus cause the apparatus to perform at least the steps of a method as disclosed herein for managing waste at a waste collection vehicle.
In a fifth aspect, non-transitory storage medium is disclosed, for storing a computer a computer program comprising instructions which when executed by an apparatus cause the apparatus to perform at least the steps of a method as disclosed herein for managing waste at a waste collection vehicle.
In a sixth aspect, a computer implemented method for analyzing waste is disclosed which comprises at least: receiving from a waste management apparatus, information about new objects entering a waste collection vehicle during a given waste collection tour, said information comprising at least, for each new object, an object class in a classification of objects, a time and a location of the waste collection vehicle when the new object entered the waste collection vehicle; mapping the class of the new objects with an expected type of waste collection tour based on sorting rules stored in a table; obtaining an actual type of the given waste collection tour; calculating a rate of waste contamination for a given location of the waste collection vehicle during the given waste collection tour as a function of the number of new objects identified at said given location which classes are mapped with an expected type of waste collection tour other than the actual type of the given waste collection tour; providing the rate of waste contamination.
In an embodiment, the method for analyzing waste as disclosed herein further comprises determining the actual type of a given waste collection tour from the distribution of the classes of the new objects entering a waste collection vehicle during the given waste collection tour.
In an embodiment of the method for analyzing waste as disclosed herein, the information received from the waste management apparatus comprises, for each new object, an indication of the actual type of waste collection tour associated with the new object.
In another aspect, a computer program product is disclosed comprising instructions which when executed by an apparatus cause the apparatus to perform at least the steps of a method as disclosed herein for analyzing waste at a remote server.
In yet another aspect, a non-transitory storage medium is disclosed, for storing a computer a computer program comprising instructions which when executed by an apparatus cause the apparatus to perform at least the steps of a method as disclosed herein for analyzing waste at a remote server.
Example embodiments will become more fully understood from the detailed description given herein below and the accompanying drawings, which are given by way of illustration only and thus are not limiting of this disclosure.
Various example embodiments will now be described more fully with reference to the accompanying drawings in which some example embodiments are shown.
Detailed example embodiments are disclosed herein. However, specific structural and functional details disclosed herein are merely representative for purposes of describing example embodiments. The example embodiments may, however, be embodied in many alternate forms and should not be construed as limited to only the embodiments set forth herein. Accordingly, while example embodiments are capable of various modifications and alternative forms, the embodiments are shown by way of example in the drawings and will be described herein in detail. It should be understood, however, that there is no intent to limit example embodiments to the particular forms disclosed.
A waste collection vehicle 1 is represented in
The waste collection vehicle further comprises one or more optical sensors 4 configured to capture images of the waste dumped into the hopper 2, and a waste collection management apparatus 5 comprising at least one memory and at least one processor configured to process the acquired images. The waste collection vehicle 1 further comprises a location device (not represented in
For example, the apparatus 5 is located inside the cabin of the vehicle and it is configured to provide the electrical interface between the optical sensor(s) 4 and the vehicle electrical system.
The apparatus 5 is communicatively coupled with the optical sensor(s) 4 to allow for the transmission of the images acquired by the optical sensor(s) 4 to the apparatus 5. In the example of
In an embodiment, the vehicle 1 also comprises one or more lighting source(s) 7 mounted on one or more walls 10 of the vehicle 1 to illuminate an area around the hopper 2 thereby facilitating the acquisition of the images by the optical sensor(s) 4. In the example of
For example, the one or more optical sensors 4 are cameras arranged to capture in their field of view the waste dumped into the hopper 2 and the interior of hopper 2. In some embodiment, the one or more cameras are arranged to capture additional elements such as the bin, or one or more vehicle parts which position provide meaningful information for the purpose of managing or analyzing the waste being collected. For example, the position of the bin, or the color of the bin, or the position of a vehicle part can be used to derive a type of waste collection tour as will be further described below.
In the example of
A waste collection vehicle as disclosed herein may comprise a plurality of dumpster compartments to handle a plurality of types of waste collection tour in parallel.
For example, as illustrated in
Advantageously, images are acquired continuously by the optical sensor(s) 4 while the waste collection vehicle 1 is in service (rather than being acquired upon reception of a triggering signal e.g. a triggering signal indicative of a start of the waste collection at a waste collection point). With the disclosed embodiment, there is no need to receive and monitor any additional triggering signal. There is no need for additional connectivity between the waste collection vehicle 1 and the apparatus 5.
The waste collection management apparatus as disclosed herein is configured to receive and process the acquired images 4 to detect objects, and, for each detected object, provide a class of objects in a classification of objects and determine, amongst the detected objects, new objects entering the waste collection vehicle. Then, the apparatus 5 sends to a remote server information about the new objects, comprising at least the class of the new object, a time, and a location of the waste collection vehicle when the new object enters the waste collection vehicle.
For example, an object can be a bag containing waste, a packaging, a box, a bottle, a bulky item, . . . .
In an embodiment, the classification relates to waste sorting rules and is such that a type of waste collection tour can be determined from the classes of the objects collected during a given waste collection tour as will be further described below. For example, the classification of objects can include the following classes: black waste bags, yellow waste bags, cardboard, expanded polystyrene, plastic bottle, plastic bags, metal packaging, glass, fabrics, wood, yard waste, kitchen waste, construction waste, hazardous waste, etc . . . .
An object is referred to as a new object entering the waste collection vehicle when it is detected for the first time. It is possible that an object is not visible first, because it is hidden by other objects. In this situation, the object may not be detected immediately when it is dumped into the hopper. It may be detected at a later point, in subsequent acquired images. It is also possible that an object is visible in several images, or visible alternatively in some images and not in others as objects move into the hopper. In this situation, the object is determined as a new object when it is first detected, and then it is ignored.
In the following of the description, an object detected in a given image is referred to as a detection and is denoted Ok,j where index k indicates the image and index j indicates the object.
There are several ways of determining a new object entering the waste collection vehicle.
In a first embodiment, the apparatus 5 uses a machine learning module (or AI module) to detect objects in the acquired images and classify the objects according to the classification, and a tracking module to determine new objects amongst the detected objects. In this embodiment, the AI module provides, for each object detected in a given image, a position of the detected object in said given image, in addition to providing the class of the object. The position of the objects in the acquired images is used by the tracking module to determine whether the detected object is a new object or whether it was already present in a previous image. The determination is made by comparison with detections in previous images. Although not essential, the tracking module can also perform an estimation of movement of the detected objects and use the movement estimation to make the determination. Estimation of movement can be performed by using known solution e.g. Kalman filters.
The determination of new objects in this first embodiment will now be described in more details with reference to
The records stored in memory 63 comprise at least the position of the detected object in the given image and information descriptive of the detected object. For example, the information descriptive of the object can be the class of the object.
Other descriptive information can be used. For example, rather than using the class of higher probability for a detected object, it is possible to use the probability distribution of the classes. Images can also be used. Using other descriptive information can provide a more precise tracking than just using the class of the object. For example, using the image can allow to differentiate similar objects based on their color. Using the distribution of probabilities allows to make better decisions when the model hesitates about the class of an object.
Other types of data can also be used in addition to the position and the descriptive information. For example, the record associated to a detection can include a score provided by the AI module which indicates the confidence associated to the detection made by the AI module. The score can advantageously be used by the tracking module when making determinations e.g. the score may need to exceed a certain threshold for a detection to be considered in the determination process. Or a comparison of scores can be used in addition to a comparison of classes (or probability distribution of classes) to make the determination.
As illustrated in
When a match is found between the current detection Ok,j and the previous detections stored in memory 63, the current detection Ok,j is recorded in the memory 63 at step 84 and the method resumes at step 83 with the next detection Ok,j+1. When all detections in image k are processed, the method resumes at step 81 with the next acquired image k+1.
On the contrary when no match is found, optionally, at step 85, an additional check is done based on the speed of the waste collection vehicle to eliminate false positives (it is assumed that no waste is being dumped into the hopper when the waste collection vehicle is moving). When the speed of the waste collection vehicle is below a threshold, at step 86, the detected object is determined a as a new object. Otherwise, the detected object is determined as being a false positive and is ignored. Then at step 84, the current detection Ok,j is recorded in the memory 63 and the method resumes at step 83 with the next detection Ok,j+1. When all detections in image k are processed, the method resumes at step 81 with the next acquired image k+1.
For example, the speed of the waste collection vehicle can be calculated from the positions provided by a location device embedded in the waste collection management apparatus 5. This implementation is advantageous because it limits the need for connectivity between the apparatus 5 and the waste collection vehicle 1.
The current detection Ok,j and the previous detections stored in the memory 63 can be compared by applying known solutions like Intersection Of Union (IOU) or alternatives like Generalized Intersection Of Union (GIOU) or Complete Intersection Of Union (CIOU).
In a second embodiment, the determination of new objects is performed by the AI module. In this second embodiment, the AI module receives the acquired images as input, detects the objects in the acquired images and provides at least the class of the detected objects and an object identifier that uniquely identifies the detected object. The detected object is determined as a new object when the object identifier estimated by the AI module is different from the object identifiers of the objects detected in the previous images. This second embodiment requires that the AI model is trained to estimate the object identifier in addition to estimating the object class. This is more costly to implement than the first embodiment as many more training images are required to train the model, which need to be labelled manually.
In this second embodiment, although it is not needed for determining new objects, the AI module advantageously provides the position of the detected objects so that it can be included in the information sent to the remote server. For example, the position can be presented to the waste collection service operator or/and its client in connection with proof images to improve the management of the waste collection services. For example, proof images can be checked by a physical person and can be used to re-retrain the AI module.
Other determinations can be made by the apparatus 5 based on the acquired images and added to the information sent to the remote server. Advantageously, these other determinations are made by the same AI module which provides several estimations in parallel. For example, the apparatus 5 can determine, from the acquired images, a type of entrance into the waste collection vehicle. For example, the type of entrance can be a manual entrance, an entrance via a bin lifted by the waste collection vehicle, or via a bin lifted manually, or via a container which is being discharged into the waste collection vehicle (e.g. a street container or a container at a drop off point). Such data are advantageously presented to the waste service operator and/or its clients as they can help improve the management of the waste collection services.
In another example, when the waste collection vehicle is arranged to collect waste for at least two types of waste collection tours, the apparatus 5 determines, from the acquired images, the type of waste collection tour associated with the new objects and includes an indication of the type of waste collection tour in the information sent to the remote server. As will be further described below, this indication can be used to calculate a rate of waste contamination at the remote server, when the waste is being collected by multi-stream vehicles. For example, the acquired images would show which dumpster is closed when the images are captured and the type of waste collection tour can be derived from this knowledge.
The information about the new objects, generated at the apparatus 5, is sent to a remote server through a wireless connection. This information comprises at least the class of the new objects as well as a time and a location of the waste collection vehicle when the new objects entered the waste collection vehicle. Advantageously, for each new object, the following data are included in the information sent to the remote server: the class of the new object, the position of the new object, the score of the new object, the acquired image in which the new object was detected, the date and the time of capture of the acquired image, and the location of the waste collection vehicle when the image was acquired. When the waste collection vehicle is a multi-stream vehicle, an indication of the type of waste collection tour associated with the new object is also included. Depending on the functionalities required on the client side, and depending on the bandwidth available, some of the data listed above may be omitted, or other data may be included. For example, a short video showing the waste dumped into the hopper could be sent as well (e.g. a few images before and a few images after the waste is dumped). Also, when bins are identified by an electronic tag, an identifier of the bin can be included. As previously mentioned, the type of entrance into the waste collection vehicle can also be added.
Advantageously, the information can be packaged so as to limit the bandwidth required. For example, when several new objects are detected in the same acquired image, the data which are common to all new objects are sent only once (for example, the acquired image, the location of the waste collection vehicle, the date and time when the acquired image is captured are sent once for all new objects detected in the same acquired image).
On the side of the remote server, the information is received and processed before being made available to a user through a user interface.
At step 91, the remote server receives, from a waste management apparatus of a waste collection vehicle, information about new objects entering the waste collection vehicle. This information comprises at least the class of the new objects and the time and location where the new objects have been collected. The time is used to associate a waste collection tour to the new objects. The location is used to associate a waste collection point within the waste collection tour.
At step 92, the class of the new objects are mapped with an expected type of waste collection tour based on sorting rules stored in a table at a remote server. For example, four different types of waste collection tour can be defined: residual waste, recyclable waste, biowaste and glasses.
At step 93, an actual type of the waste collection tour is obtained for the waste collection tour associated with the new objects. As will be further described below, there are several ways of obtaining the actual type of the waste collection tour associated with a new object.
At step 94, a rate of waste contamination is calculated, for a waste collection point in a waste collection tour, as a function of the number of new objects identified at said waste collection point during said waste collection tour, which classes are mapped with an expected type of waste collection tour other than the actual type determine for the waste collection tour. At step 95, the waste contamination rate is delivered e.g. to a user interface module.
Mapping the classes of the new objects with an expected type of waste collection tour at the remote server is advantageous because the mapping table used by the remote server can be updated at any time by the waste service operator or its clients. The mapping table can also be adapted client by client. For example, different cities may implement different sorting rules, so that a class of waste regarded as residual waste in one city, will be regarded as recyclable waste in another city. For example, yellow waste bags can be dedicated to recyclable waste in one city or to residual waste in another city.
In a first embodiment, when the waste collection vehicle is a single-stream vehicle, at step 93, the actual type of the waste collection tour is determined from the distribution of the classes of the new objects entering a waste collection vehicle during the given waste collection tour. This information varies as the waste collection tour proceeds. For example, the remote server waits for a certain number of new objects to be detected so as to obtain an actual type of waste collection tour which is reliable.
Advantageously, a weighting may be applied to the different types of waste collection tour when determining the actual type of waste collection tour from the distribution of classes of the new objects. For example, without weighting, a collection point with a total of 5 black waste bags, 7 plastic bags and 1 cardboard would be associated with a recyclable waste collection tour as the distribution is 8 recyclables items against 5 non-recyclable items. However, it is infrequent that black waste bags are placed in a recyclable bin or container. On the contrary, it is quite frequent that people make mistake about plastic bags and place them in non-recyclable bins or containers. With a weighting, it is possible to give more importance to classes for which errors are less frequent. For example, a weight of 2 can be given to black waste bags, a weight of 0.2 can be given to plastic bags, and a wait of 1 can be given to cardboards. With such a weighting, the distribution becomes 2*5=10 residual waste vs. 1+0.2*7=2.4 recyclable waste. And the actual type of waste collection tour is determined to be a non-recyclable waste collection tour.
Weighting can take into account several parameters, for example, how reliable the AI module is for detecting objects, how frequent it that the user makes sorting mistake in relation to a given class of objects, or even how big the objects are.
In a second embodiment, when the waste collection vehicle is a multi-stream vehicle, the information received from the waste management apparatus 5 comprises, for each new object, an indication of the actual type of waste collection tour associated with the new object. As disclosed above, this information is obtained by the AI module from the acquired image which shows which dumpster compartment(s) is/are closed when the waste is dumped into the hopper. In this second embodiment, at step 83, the indication of the type of waste collection tour associated with each new object is retrieved from the information sent by the apparatus 5.
For example, the user interface module can display a map where the collection points for a given collection tour period are displayed. For example, the collection point can be represented as a colored point, the color indicating the rate of contamination of the waste collected at said collection point during said collection tour. And when the user clicks on the point, access to further information can be provided such as the acquired image in which the new object was detected.
It should be appreciated by those skilled in the art that any block diagrams herein represent conceptual views of illustrative circuitry embodying the principles of the disclosure. Similarly, it will be appreciated that any flow charts, flow diagrams, state transition diagrams, and the like represent various processes which may be substantially implemented by circuitry.
Each described function, engine, block, step can be implemented in hardware, software, firmware, middleware, microcode, or any suitable combination thereof. If implemented in software, the functions, engines, blocks of the block diagrams and/or flowchart illustrations can be implemented by computer program instructions/software code, which may be stored or transmitted over a computer-readable medium, or loaded onto a general purpose computer, special purpose computer or other programmable processing apparatus and/or system to produce a machine, such that the computer program instructions or software code which execute on the computer or other programmable processing apparatus, create the means for implementing the functions described herein.
In the present description, block denoted as “means configured to perform . . . ” or “module configured to perform . . . ” (a certain function) shall be understood as functional blocks comprising circuitry that is adapted for performing or configured to perform a certain function. A means or module being configured to perform a certain function does, hence, not imply that such means necessarily is performing said function (at a given time instant). When provided by a processor, the functions may be provided by a single dedicated processor, by a single shared processor, or by a plurality of individual processors, some of which may be shared. Moreover, explicit use of the term “processor” or “controller” should not be construed to refer exclusively to hardware capable of executing software, and may implicitly include, without limitation, Graphic Processing Units (GPU), digital signal processor (DSP) hardware, network processor, application specific integrated circuit (ASIC), field programmable gate array (FPGA), read only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional or custom, may also be included. Their function may be carried out through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or even manually, the particular technique being selectable by the implementer as more specifically understood from the context.
As used herein, the term “and/or,” includes any and all combinations of one or more of the associated listed items.
When an element is referred to as being “connected,” or “coupled,” to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Other words used to describe the relationship between elements should be interpreted in a like fashion (e.g., “between,” versus “directly between,” “adjacent,” versus “directly adjacent,” etc.).
The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the,” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises,” “comprising,” “includes,” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any element(s) that may cause or result in such benefits, advantages, or solutions, or cause such benefits, advantages, or solutions to become more pronounced are not to be construed as a critical, required, or essential feature or element of any or all the claims.
