Patent Application
20200077667
In the prior art systems and methods for approximately determining the volume and many other properties of a carcass are generally known. Such prior art systems and methods are used on production lines at slaughterhouses for various purposes such as for grading carcasses on a scale, for sorting carcasses in readiness of further processing on the production line, and for controlling settings at automatic cutting machines on the production line. Such systems and methods improve automation and slaughter yield on the production lines and reduce the risk of injury to personnel working at the production line. Also, the farmer supplying the livestock to the slaughterhouse may be paid more accurately in accordance with the physical properties of the livestock.
One known procedure is, for example, to hang a carcass from a weighing system, to determine its weight and, on the basis of the weight and a scanning of the carcass or livestock, to approximate its shape and therefrom to make statements about the yield that can be expected. However, this method has the particular disadvantage that the volume and the slaughter yield to be expected can only be roughly estimated on the basis of the approximated volume. In this method, it is mostly ignored that each slaughter animal body has different expanses of tissue, meat, bone and fat tissue sections in particular, and therefore an exact statement about the most likely slaughter yield of meat of the slaughter animal body cannot be made. Also, such procedures are approximate and less useful for controlling cutting machines on the production line.
In one class of prior art systems such as described in EP 2 092 829 (Banss Schlact- and Fordertechnik), U.S. Pat. No. 5,194,036 (Normaclass) and CA 2 466 289 (Tong) it is known to record a surface profile using structured light, such as a laser line, to obtain a pseudo 3D representation of carcasses.
In another class of prior art systems it is known to use camera systems, such as those using time-of-flight or depth-from-defocus, to obtain 3D information without structured light. This class of prior art is exemplified in greater detail below.
US 2016/0029648-A1 (CSB-SYSTEM AG) describes an apparatus for volumetrically measuring the body of an animal for slaughter. The apparatus has a first depth camera and a second depth camera which are so-called time-of-flight cameras. The cameras have respective recording areas (field-of-views) and are arranged opposite to one another to record respective depth images from opposing sides of the body. The apparatus has an evaluation unit connected to the depth cameras and acquires spatial coordinate data provided by the depth cameras. The spatial coordinate data from the depth cameras are combined as combined spatial coordinate data in a common spatial coordinate system. A surface model of the body of an animal for slaughter is provided from the combined spatial coordinate data, and a volume of the object in the body of an animal for slaughter is estimated from the surface model. An object of that invention is to provide a device for volumetrically measuring a slaughter animal body that allows an improved determination of the volumes of slaughter animal bodies and a reliable assessment of the slaughter yield to be expected at relatively low costs.
However, use of time-of-flight cameras involves problems related to synchronisation or alignment between image (depth) data from the time-of-flight cameras and image data recorded at visible wavelengths such as colour image data. Also time-of-flight cameras involve problems related to use in an environment with daylight or light sources that are not controlled.
U.S. Pat. No. 7,399,220 B2 (Kriesel) describes a method and apparatus for measuring of the physical characteristics of livestock animals such as cattle and hogs or carcasses. A plurality of strategically positioned cameras are used to obtain data concerning volumetric, curvilinear (surface) and linear measurements of livestock animals such as cattle and hogs and the full carcasses thereof. Data from the cameras are analysed to provide information that substantially assists the commercial producer of livestock animals in producing a high-quality end-product for the consumer. Three cameras and data processing techniques are applied to achieve 3D data sets of the animal. In one embodiment the target animal is illuminated with structured light (predefined light patterns) and images are obtained from three strategic camera positions. Each of the images is then processed by a range image algorithm to obtain a three dimensional point cloud set. A processing algorithm converts the 3D data from the range camera into mesh data aligned on a rectangular grid, performs Delauney triangulation, which projects the 3D data set onto an X-Y plane and generates triangles in such a way that no data point is contained within any triangle's circumcircle, applies a mesh function to the data resulting from Delaunay triangulation, to obtain Delaunay triangles in three-dimensions, and then applies a surface function to apply a surface. Different types of camera technologies for acquiring images that can be combined to achieve the 3D data sets are described. One camera technology uses a depth-from-defocus system that employs a single visible spectrum camera, wherein at least three images are combined to establish a range map of the surface. Active stereoscopic vision systems may also be used to obtain 3D surface measurements. This technology uses two cameras separated by a distance sufficient to triangulate on a given point on the target surface. A minimum of two images are required to compute the target surface. However, as acknowledged, holes in the computed surface result when a physical surface feature of the carcass or livestock obstructs one of the camera views.
Imaging systems relying on emission of structured light involves problems related to distortions or image artefacts caused when the object to be imaged is moving. The problem escalates the faster the object is moving.
It is realized that there is a need for more accurate and reliable methods and systems for generating volumetric and spatial property representation of carcasses which has been cut up or cut open to reveal at least a portion of an inner surface of the carcass for visual inspection from a position next to a passageway. As an example, the carcass may be a so-called half carcass resulting from cutting up a carcass such as from ovine, bovine and porcine into two half carcasses as it is known in the art. The inner surface of a carcass, such as a half-carcass, may be partially visible, but not fully visible at least from a fixed station next to the passageway at which the carcass passes. The inner surface of the carcass may comprise portions of the thoracic cavity of carcass e.g. from ovine, bovine and porcine.
Multi-view imaging requires a line-of-sight between the camera's image sensor and a region at the surface of the carcass. However, multi-view imaging, at some regions of the surface of the physical carcass, may be of poor quality or absent e.g. when the regions face a cavity or otherwise are occluded by portions of the carcass. At such regions multi-view imaging for constructing an accurate 3D model are not sufficient.
There is provided a method of imaging carcasses, comprising:
Thereby, a more accurate 3D carcass model can be generated. By manipulating the 3D carcass model, computed from the first set of multi-view images and the second set of multi-view images, at least at the first region as mentioned above and described in more detail herein, the 3D carcass model computed from the first set of multi-view images and the second set of multi-view images is transformed into a 3D carcass model more accurately representing the carcass.
The first region, at which the carcass is spatially poorly represented, is subjected to spatial manipulation in accordance with one or both of a weight registered by a load cell and a representation of a suspension position at which the carcass is suspended.
Accordingly, the spatial manipulation (also denoted manipulation) may change one or both of volume of the 3D carcass model and an estimate of the weight of the carcass subject to imaging, if computed. The spatial manipulation may change one or both of the total cubic measure of the 3D carcass model and the distribution of volume of the 3D carcass model. Spatial manipulation changing the distribution of volume of the 3D carcass model changes estimates of a centre of gravity or a geometric centre of the 3D carcass model, if computed. Spatial manipulation changing the total cubic measure of the 3D carcass model changes estimates of a weight of the 3D carcass model, if computed.
The suspension position may be determined via image processing, e.g. based on locating a geometrical object placed to indicate the suspension position, or it may be a fixed position in the domain of the 3D carcass model. The suspension position is a position at which the carcass is suspended e.g. on a hook as it is known in the art. Thus the representation of a suspension position may be a representation referring to (or having a known transformation or inverse transformation to) a position in the domain of the 3D carcass model and representing a physical suspension position relative to the carcass subject for imaging. Thus, the represented suspension position may be relative to the domain of the 3D carcass model or relative to the 3D carcass model itself.
Generally, the suspension position and the estimate of the centre of gravity may be computed in a 3D space or in a 2D space or in a 1D space. In case of a 1D space, the centre of gravity (which is a position) and the suspension position may correspond to respective positions in a direction longitudinal to the passage way or transverse to the passage way or along another predefined horizontal axis. In case of a 2D space the respective positions may correspond to a coordinate in a horizontal plane. In case of 3D space, one or both of the respective positions are 3D positions, however vertical components may be discarded.
The centre of gravity and the suspension position may be represented in a domain of the 3D carcass model such as in a 3D Cartesian coordinate system or a 2D Cartesian coordinate system.
In a first example, the spatial manipulation is applied to change the distribution of volume of the 3D carcass model and in turn an estimate of a centre of gravity of the 3D carcass model. In that case, the spatial manipulation is applied to horizontally align the estimate of the centre of gravity of the 3D carcass model with the suspension position.
In a second example, the spatial manipulation is applied to change the total cubic measure of the 3D carcass model. In that case, the spatial manipulation is applied to change an estimate of the weight of the carcass to make the estimate of the weight of the carcass match or equal the weight registered by a load cell. The total cubic measure of the 3D carcass model may be multiplied by a predefined weight-volume conversion factor (an estimate of mass density) to compute an estimated weight for comparison with the weight registered by a load cell. Alternatively, the total cubic volume may be assigned a unity mass density and the total cubic volume or the weight registered by a load cell may be transformed, such as scaled, to enable the weight or volume of the carcass match or equal the weight registered by a load cell or another scale having a known relation to weight. The weight-volume conversion factor may be a scalar value, but is not limited thereto as explained in more detail herein.
In a third example, the spatial manipulation is applied to change one or both of the total cubic measure of the 3D carcass model and the distribution of volume of the 3D carcass model in accordance with the above.
The spatial manipulation changes estimates of an estimated weight of the carcass and estimates of a centre of gravity. Applying the spatial manipulation in accordance with one or both of a weight registered by a load cell and a representation of a suspension position at which the carcass is suspended, makes it possible to apply to the 3D carcass model an accurate amount or degree of change in absence of accurate spatial information due to occlusion or sparse representation at the first region. Thus, the 3D carcass model is adapted to correspond to more reliable observations such as a weight value measured by a load cell or a horizontal location of a centre of gravity estimated by identifying the location of a suspension point, in case the carcass is suspended.
The manipulating of the 3D carcass model to change the volume of the 3D carcass model or the distribution of volume of the 3D carcass model in accordance with one or both of a weight registered by a load cell and a representation of a suspension position at which the carcass is suspended, may be performed by an iterative procedure comprising iteratively computing a cost measure that represents how well weight estimated from the 3D carcass model and/or a centre of gravity estimated from the 3D carcass model corresponds to the weight registered by the load cell and/or the representation of the suspension position. In some aspects a single iteration may be sufficient. The 3D carcass model may be based on one or both of a 3D point cloud and a polygon surface. The manipulation of the 3D carcass model comprises one or more of: adding 3D points or polygons, removing 3D points or polygons and moving 3D points or polygons.
The locating of the first region of the 3D carcass model by locating an occluded or spatially sparsely represented region of the 3D carcass model may be performed in different ways e.g. by computing a measure of spatial density of the point cloud or polygons at predefined locations or throughout the 3D carcass model. Therefrom a region with relatively low density, e.g. below a threshold, may be identified as the first region. In some aspects multiple regions are identified and one or multiple regions thereof are subject to manipulation concurrently with the first region. In some aspects, predefined locations in the 3D carcass model are directly selected as regions for the manipulation or they are prioritized in a search for regions that satisfies a criterion of having a certain relatively low spatial density (of the point cloud or polygons), such as below a threshold.
The multi-view images of the first set of multi-view images and the second set of multi-view images are all captured at the first point in time. The cameras of the first set of cameras and the second set of cameras are triggered at the same point in time. Thereby distortion of the 3D carcass model caused by physical movement of the carcass or changes in lighting (illumination) of the physical carcass is reduced. The cameras may have a first exposure time in the range of 0.05 to 5 milliseconds, such as 0.5 to 5 milliseconds and the cameras may receive a trigger signal within a short period of 0 to 10 microseconds to capture the multi-view images at first point in time. The multi-view images, which are acquired at the first point in time, are the multi-view images of the first set of multi-view images and the second set of multi-view images. By the multi-view images being acquired at the first point in time is understood that the cameras register the amount of light impinging on the sensor elements of their image sensors at the first point in time i.e. that the cameras ‘capture’ the images at the first point in time.
In some embodiments the first set of cameras and the second set of cameras comprise each 20 cameras. The first set of cameras and the second set of cameras may each comprise a larger number of cameras such as, but not limited to, 24 cameras, 25 cameras, 32 cameras or a smaller number of cameras such as 16 cameras. In some embodiments the cameras have an optical system with a focal length in the range of 5 to 9 millimetres, such as 6 to 8 millimetres. The first set of cameras and the second set of cameras are in the below also denoted a multi-view camera system. In some embodiments the cameras are colour cameras e.g. RGB cameras. Colour cameras register the images with colour information. This is useful e.g. for classifying regions as meat, fatt, fat meat, lean meat or bone. The colour information and/or the classification of the region may be added to the 3D carcass model.
The 3D carcass model may be stored in a database or in other type of digital repository. The 3D carcass model may be analysed for the purpose of computing cut-lines and/or settings for a deboning machine to more accurately cut out valuable portions of meat. Also, the 3D carcass model may be analysed for the purpose of grading the carcass.
In some aspects the method comprises computing a grading of the carcass based on the 3D carcass model. The computing of the grading may comprise computing the volume of or a weighted volume of predefined regions in the 3D carcass model.
The passageway may be a passageway at which carcasses are guided to travel. The passageway may follow a conveyor that guide or conveys the carcasses. In some embodiments the conveyor is an overhead conveyor from which carcasses are suspended. In some embodiments the conveyor is a belt conveyor. Other transportation devices may be used to guide the carcasses to travel on or along the passageway. The carcasses may be carcasses of ovine, bovine, and porcine.
Thus, the 3D carcass model more accurately represents the physical carcass. Aspects thereof are described in more detail below.
The first set of cameras and the second set of cameras are also denoted a 3D camera system or a multi-view camera system. The first set of cameras and the second set of cameras in combination with a computer configured and coupled to process the multi-view images from the first set of cameras and the second set of cameras is also denoted a multi-view imaging system. A computer, or a system of computers, programmed to generate a 3D model from multi-view images is denoted a multi-view processor.
The first set of multi-view images is captured by cameras at respective multiple positions at the first side of a passageway, the second set of multi-view images is captured by cameras at respective multiple positions at the second side of the passageway.
In some embodiments the method comprises:
Thereby the 3D carcass model is verified to be accurate or is adapted to correspond to the observation on the weight of the carcass.
Relative to the occluded or sparsely represented first region, the observation on the weight of the carcass provides a reliable measure for adapting the 3D carcass model to correspond to the observation and thus obtain an accurate 3D carcass model.
A load call may be installed at an upstream location with respect to the multi-view camera system or at a downstream location; wherein upstream corresponds to a location from which the carcasses arrive to the multi-view camera system. The load cell may also be installed at the multi-view camera system to weigh the carcass at the first point in time, when the multi-view images are captured, or shortly before or after. The load cell may be installed with the conveyor as it is known in the art. Systems for registering information about a carcass and transmitting that information to a received which processes that information are known in the art of slaughterhouse operation automation. Thus weight information may be readily available.
From the 3D carcass model volumetric computations are performed to estimate the volume of the carcass from the 3D carcass model as it is and then a conversion parameter or set of parameters representing a volume-to-weight ratio is used to compute an estimate of the weight. The volume-to-weight ratio may be represented by a volume-weight conversion parameter. The volume-weight conversion parameter represents the mass density e.g. in units of Kg/m3. However, it is clear to a person skilled in the art that a variety of forms of a volume-weight conversion parameter can be used to relate volume to weight or vice versa. A difference between the estimated weight and the weight measurement value is computed and therefrom a compensation volume is added to or subtracted from the 3D carcass model by manipulating one or both of 3D points and polygons within the first region with the objective of making the estimate of the weight of the carcass correspond to the weight measurement value. An iterative method may be applied for that purpose.
In some embodiments the method comprises:
Thereby the 3D carcass model is verified to be accurate or is adapted to have a centre of gravity located at a horizontal position aligned with a horizontal position of the suspension device. An object suspended to hang freely from a suspension point will settle at a position with its centre of gravity right below the suspension point. Since a carcass may weigh in the range above 50 kg to hundreds of kilos, the carcass will settle at a position with its centre of gravity right below the suspension point.
A centre of gravity of the 3D carcass model, i.e. the second horizontal position, may be estimated by computing a geometrical centre of the volume of the 3D carcass model. This may be performed under the assumption of a uniform volume-to-weight ratio. Alternatively, a centre of gravity of the 3D carcass model may be estimated by computing a geometrical centre of the volume of the 3D carcass model, wherein, in computing the geometrical centre, a volume element (voxel) is weighted by a volume-to-weight ratio determined e.g. from the relative or absolute position of the volume element.
The first horizontal position of a suspension device may be a predetermined position determined in one or more of the multi-view images such as at one or more predetermined columns of pixels. A predetermined position may provide a sufficiently accurate positon in case the suspension device carrying the carcass has the same position relative to the multi-view camera system each time an image is captured. The first horizontal position of a suspension device may additionally or alternatively be determined by processing one or more of the multi-view images to locate the suspension device or a mark or sign attached to it.
Relative to the occluded or sparsely represented first region, the observation on the centre of gravity of the carcass (the horizontal location of the suspension point) provides a reliable measure for adapting the 3D carcass model to correspond to the observation and thus obtain an accurate 3D carcass model.
From the 3D carcass model volumetric computations are performed to estimate the volume of the carcass from the 3D carcass model as it is and then a conversion parameter or set of parameters representing a volume-to-weight ratio is used to compute an estimate of the centre of gravity. A difference between the estimated centre of gravity (second horizontal position) of the 3D carcass model and the first horizontal position of the suspension device is computed and therefrom a compensation volume is added to or subtracted or from the 3D carcass model and/or re-distributed in the 3D carcass model by manipulating one or both of 3D points and polygons within the first region with the objective of making the estimate of the centre of gravity correspond to the first horizontal position of the suspension device. An iterative method may be applied for that purpose.
In some embodiments the method comprises:
Thereby, regions providing only poor spatial representation of the carcass may be effectively identified. One or more regions may be identified in this way. One of those regions or prioritized one or more regions may be subject to the manipulation.
In some embodiments the manipulation of the 3D carcass model further comprises:
Such a geometrical indicator representing a spatial direction indicative of an expansion or reduction of the volume of the 3D carcass model, may be computed in different ways e.g. from an estimate of curvatures and a vector normal relative to face spanning the first region. Such a geometrical indicator may be used to advance the estimate of the weight of the carcass towards the weight measurement value or the estimate of centre of gravity towards the suspension point.
In some embodiments the method comprises:
Thereby, image observations which may be prone to spatial errors may be replaced by a well-defined region, such as a more densely represented region. The replacing region may have a predetermined shape or curvature corresponding to a representation of common carcasses at that region. This may improve the speed at which the 3D carcass model is modified to be a more accurate 3D carcass model.
In some embodiments the method comprises estimating a second curvature of a second region adjoining the first region and manipulating the 3D points and/or polygons within the first region subject to a smoothness criterion enforcing a smooth transition from the second curvature to a first curvature of the first region.
In some embodiments the method comprises:
Thereby, the first region may be modified to more closely conform to curvatures of physical carcasses. This improves the accuracy of the 3D carcass model. Stored curvatures may comprise parameters of predefined curvature classes such as Splines or polynomials. The stored curvatures may be derived from measurements of physical carcasses.
In some embodiments the method comprises:
In this way carcasses may more accurately cut out to improve yield of the carcass. This is especially desired to improve the yield of high quality meat sections. Also, the multi-view imaging system may control the cutting machines in a more efficient way to obtain faster cutting-up of the carcasses. The term ‘cutting machine’ comprises so-called ‘deboning machines’ known in the art.
In some embodiments the method comprises grading a carcass based on a quality measure computed from the 3D carcass model; and configuring a cut-up machine on a production line in accordance with a cut-up parameter computed from one or both of the grading and the 3D carcass model. The quality measure may relate to the volume of certain regions of the carcass.
The cut-up in accordance with the cutting parameters takes place while the cutting machines on the production line are configured in accordance with the cutting parameters.
In some embodiments the method comprises:
Thereby batch processing of the carcasses can be provided when collectively retrieved from the facility sequentially or in batches. The 3D carcass model may be the 3D carcass model or the 3D carcass model as modified.
The quality measure may comprise a scalar parameter or a multi-dimensional parameter. The quality measure may be based on one or more of total volume or the carcass, volume at predefined locations or locations with a predefined colour representation.
The classification may be based on a statistical model such as, but not limited to, a nearest neighbour model or a support vector machine.
In some aspects the method comprises: configuring a cutting machine on a production line in accordance with a cutting parameter assigned with the classification representation; retrieving the gathered batch of carcasses from the selected facility and routing them to the production line; and cutting the carcasses in the batch of carcasses in accordance with the cutting parameter. Thereby the multi-view imaging system may control the cutting machines in a more efficient way to obtain faster cutting-up of the carcasses.
There is also provided a computer programme product comprising instructions making a computer system perform the method set out above when executed by the computer system.
There is also provided a system comprising: a multi-view imaging camera system comprising a first set of cameras arranged at a first side of a passageway for capturing a first set of multi-view images from respective multiple positions, and a second set of cameras arranged at a second side of the passageway for capturing a second set of multi-view images from respective multiple positions; and a computer programmed to:
wherein the 3D carcass model is based on one or both of a 3D point cloud and a polygon surface;
There is also provided a slaughterhouse production line, comprising a system as set out above.
There is also provided a method of imaging carcasses, comprising:
In some embodiments the method comprises:
In some embodiments the locating a first region of the 3D carcass model comprises identifying first regions of the 3D carcass model as regions of the 3D carcass model that have a 3D point density that fails to exceed a 3D point density threshold; and wherein the 3D point density threshold is adjusted until a predefined number of first regions are identified; the method comprising:
There is also provided a method of imaging carcasses, comprising:
There is also provided a system comprising:
A more detailed description follows below with reference to the drawing, in which:
In some embodiments the first camera towers 103a-d are spaced apart by 0.4 to 1.2 meters centre-to-centre e.g. substantially equidistantly. The second camera towers 106a-d may be arranged symmetrically with the first camera tower 103a-d. The distance between the first camera towers 103a-d and the second camera towers 106a-d measured at a right angle to the conveyor may be in the range of 2 to 4 metres e.g. about 3 metres. The passageway 115 may have a width, wd, of about 2.3 metres e.g. in the range of 1 to 3 metres. In some embodiments the first camera towers 103a-d and the second camera towers 106a-d are arranged along in a first row and in a second row, respectively. The first row and the second row may be straigth rows. The first row and the second row may be curved rows e.g. curved inwards towards the conveyor or passageway. The first row and the second row may be substantially parallel with the conveyor or passageway or symmetrical about the conveyor or a longitudinal centre line of the passageway. The closest distance between a camera tower on the first side of the passageway and a camera tower on the second side of the passageway may be greater than the distance between two neighbouring camera towers on one side of the passageway.
The first camera towers 103a-d and the second camera towers 106a-d may have a smooth or flushing surface with a transparent screen protecting the multiple cameras, and their lenses, of the first set of cameras 104 and of the second set of cameras 105 from direct contact with the surroundings. This is expedient e.g. to facilitate easier and faster cleaning.
A control system (not shown) is described in more detail below.
At the conveyor 110 is arranged a load cell 109. The load cell 109 is arranged, as shown, some distance upstream of the 3D multi-view imaging station 102. However, the load cell 109 may be arranged at another position at the conveyor upstream or downstream of the 3D multi-view imaging station 102 or e.g. at the conveyor within a rectangle circumscribing the first camera towers 103a-d and the second camera towers 103a-d. The load cell 109 communicates, e.g. by transmitting to or making available for reading by the control system, a weight measurement value representing the weight of a carcass, e.g. carcass 113, suspended from the suspension device 111 at the conveyor. Thus, the control system can acquire a weight measurement value from the load cell 109 at which a carcass is weighed.
At the conveyor 110 is arranged a first sensor 107. The first sensor registers that a suspension device and/or carcass is detected at the first sensor 107. The first sensor 107 is arranged, as shown, at an entryway to the 3D multi-view imaging station 102 or at some distance upstream of the centre of a rectangle circumscribing the first camera towers 103a-d and the second camera towers 103a-d. The first sensor 107 communicates a first signal indicating that a suspension device or carcass suspended therefrom is detected at the location of the first sensor to the control system. The first signal may be used to arm cameras in readiness of being triggered to acquire multi-view images.
At the conveyor 110 is arranged a second sensor 108. The second sensor 108 registers that a suspension device and/or carcass is detected at the second sensor 108. The second sensor 108 is arranged at or about a position above a focal centre of the cameras or a geometrical mean thereof. The second sensor 108 may be arranged at the centre of a rectangle circumscribing the first camera towers 103a-d and the second camera towers 103a-d. The second sensor 108 communicates a second signal indicating that a suspension device or carcass suspended therefrom is detected at the location of the second sensor to the control system. The second signal may be used to trigger simultaneous acquisition of multi-view images. Thus, the multi-view images can be acquired at a first point in time, upon detection of a carcass being present at a predetermined position at the passageway;
The multiple cameras in the second set of cameras 105 are represented by dashed circles—designated by reference numeral 210. The multiple cameras of in the first set of cameras 104 are not shown in this side-view.
The conveyor may convey the carcasses (its suspension devices) at a substantially constant speed of about 0.05 to 1 metres per second or at intervals separated by stand-still periods.
The height (vertical extend) of the carcass may be in the range of 2 to 3.6 metres for calf carcass. Carcass may be any carcass from e.g. ovine, bobine, or porcine.
As described herein, in some embodiments a suspension position is given by a representation of a suspension position at which the carcass is suspended. The suspension position may coincide with a physical, geometrical feature of the suspension device 111 or a fiducial mark (cf.
The dashed lines illustrate field-of-view limits for cameras installed in a 3D multi-view imaging station. Each camera has a field-of-view. Outside the field-of-view the camera makes no registration of image information. Thus, if none of the cameras has a field-of-view registering image information from a region, that region is said to be occluded. For instance a field-of-view limit 302 illustrates that no camera registers image information from the second cavity 306. Correspondingly, a field-of-view limit 304 illustrates that no camera registers image information from the first cavity 305 and the first cavity is then occluded. Generally, the 3D multi-view imaging station 102 is configured to register image information from a point on the surface of the carcass by more than one camera. Such a system may be configured with field-of-views of its cameras arranged to acquire images with an overlap of 40-80% e.g. about 60% between two images that have the highest level of correlation. The overlap is necessary to ensure that 3D information can be derived from the images. Thus some regions on the surface of an object may be densely represented since relatively many images overlap, whereas other regions are sparsely represented since only one or relatively few images over at that region. The field-of-view limit 303 illustrates a transition from a densely represented region to a sparsely represented region on the surface of the carcass. The surface within a cavity may be sparsely represented or not represented at all by image information.
One strategy is to render the 3D carcass model with a coarser resolution at such sparsely represented regions e.g. as shown by a sparsely represented region 401 and a sparsely represented region 402. Another strategy is to ignore or filter out image information at such sparsely represented regions to render the 3D carcass model with a flat or smooth region where the information is dense or sparse e.g. as shown by the sparsely represented region 401 and the sparsely represented region 402.
A first centre of gravity can be estimated from the 3D carcass model. The horizontal position of the first centre of gravity 403 is illustrated by a first cross.
The first centre of gravity 403 is computed from the 3D carcass model e.g. as it appears in
The manipulation is illustrated by an estimated region 506 and an estimated region 507 and by an estimated region 508 and an estimated region 509. The shape of the first cavity 305 is then estimated by the estimated region 508 or the estimated region 509. Correspondingly, the shape of the second cavity 306 is then estimated by the estimated region 506 or the estimated region 507. The deepness or other shape parameters of the estimated regions may be estimated iteratively by aiming for one or both of a target location of the centre of gravity and an estimated target weight of the 3D carcass model. The target location of the centre of gravity may be the horizontal position of a suspension point. The target weight of the 3D carcass model may be obtained from the load cell. An estimated volume of the 3D carcass model may be converted to an estimated weight e.g. by assuming the each volume unit has a unitary volume weight.
A direction 510 indicates in increasing concavity that erodes the volume of the 3D carcass model. Thus, the estimated region 508 may be replaced, e.g. during an iterative process, by the estimated region 509 whereby the estimated weight of the 3D carcass model is lowered and whereby the centre of gravity of the 3D carcass model moves towards the second point of gravity 502. Correspondingly, a direction 511 indicates in increasing concavity that erodes the volume of the 3D carcass model.
Densely represented region 503, 504 are illustrated by closely located small circles, whereas a less densely represented region 505 is represented by more sparsely located small circles.
A first section 614 of the flowchart illustrates that a first event ‘Event #1’ is detected at step 612. The first event may be that a suspension device e.g.
suspension device 111, travelling on the conveyor 110, enters the load cell 109. Upon detection of the first event, a weight measurement value is acquired at step 613 from the load cell at which the carcass, if any, suspended by the suspension device is weighed. In connection therewith the weight measurement value may be recorded in an electronic record with an identification code identifying the carcass being weighted. The weight measurement value is then made accessible via the identification code. This is known in the art.
A third section 615 of the flowchart illustrates that a third event ‘Event #3’ is detected at step 616. The third event may be that a suspension device, travelling on the conveyor 110, reaches or passes the first sensor 107. Upon detection of the third event, the cameras at the 3D multi-view imaging station 102 are ‘armed’ at step 617. ‘Arming’ the cameras comprises preparing the cameras for registering (‘taking’) an image—at the same point in time—shortly after the third event being detected. ‘Arming’ the cameras comprises what is sometimes denoted ‘initializing’ the cameras such as one or both of: transmitting settings to each of the cameras and ‘waking up’ the cameras from a low power state. This reduces the risk that one or more of the cameras are not ready to register an image at the same point in time. The settings transmitted in connection with ‘arming’ the cameras may comprise one or more of ‘exposure time’, ‘white balance’ and ‘gain’.
A second section 616 of the flowchart illustrates that a second event ‘Event #2’ is detected at step 601. The second event may be that a suspension device, travelling on the conveyor 110, reaches or passes the second sensor 108. Upon detection of the second event, steps of image acquisition, 3D carcass model construction and manipulation of the 3D carcass model are performed. The steps may be performed in another order than shown and some steps may be performed concurrently.
In one embodiment, as shown, an identification code, ID, is read from an electronic record associated with the carcass present at the 3D multi-view imaging station 102 upon detection of the second event. An electronic data processing system may provide the electronic record associated with the carcass present at the 3D multi-view imaging station 102. At step 603 multi-view image acquisition is performed. Step 603 involves acquiring a first set of multi-view images, using the first set of cameras 104, from respective multiple positions at a first side of the passageway, and acquiring a second set of multi-view images, using a second set of cameras 105, from respective multiple positions at a second side of the passageway 115. The multi-view images are acquired simultaneous at a first point in time. Step 603 may be performed directly upon detection of the second event. Since the carcass is moving on the conveyor, such as at substantially constant velocity, the multi-view images are acquired at the exact same point in time to reduce motion artefacts and distortion of the 3D carcass model to be generated from the multi-view images.
Then, when the multi-view images have been acquired, the 3D carcass model is generated at steps 604 and 605. These steps are computationally heavy and may comprise a step 604 of generating a 3D point cloud and therefrom generating 3D triangular mesh in step 605. Software is publicly available either as proprietary software or as open source software for generating a 3D model. Step 604 and step 605 represents such generation of a 3D model—such as a 3D carcass model.
When a 3D carcass model is generated as a result of performing one or both of step 604 and step 605 it may be stored on an electronic storage medium. Then, in step 606 a volumetric computation is performed. The volumetric computation comprises one or both of:
The first horizontal position, SP, may be estimated from time-to-time a carcass arrives at the 3D multi-view imaging station 102 and/or it may be estimated or encoded at calibration events.
Then, at step 607, one or both of the following are performed:
Further, at step 607, it is determined, based on the comparisons in 1) and/or 2) above, whether to perform (Y) volumetric manipulation of the 3D carcass model or whether to forgo (N) performing volumetric manipulation of the 3D carcass model. The determining of whether to perform (Y) volumetric manipulation or to forgo (N) performing volumetric manipulation may be based on the amount of deviation between the compared estimates e.g. based on whether a threshold deviation is exceeded or not.
Based on the above determination the flowchart branches out at step 608 to store the 3D carcass model at step 609 in case (N) or to step 610 in case (Y).
At step 610 the 3D carcass model is processed to identify one or both of occluded regions or sparsely represented regions. Then, at step 611, the volume of the 3D carcass model is manipulated as described above and in the following to change the volume of the 3D carcass model or the distribution of volume of the 3D model in accordance with one or both of a weight (W2) registered by a load cell and a representation of a suspension position (SP) at which the carcass is suspended. In some embodiments such volume manipulation of the 3D carcass model is performed in one iteration wherein step 611 leads to step 609 whereat the 3D carcass model is stored. In other embodiments, as shown, such volume manipulation of the 3D carcass model requires multiple iterations wherein step 611 leads to resuming step 606 such that an iterative loop is formed. The iterative loop is eventually concluded by storing the 3D carcass model, as volumetrically manipulated, in step 609.
It should be noted that the generation of a 3D carcass model as performed in steps 604 and 605 and the multi-view image acquisition may involve filtering, cleaning and cropping of image information and 3D model information. Such techniques are however known in the art. Multi-view image acquisition may also involve correction or compensation acquired images.
The 3D carcass model may be stored at step 605, step 605 and/or step 609 in a digital format e.g. a so-called ‘.ply-format’. However, various other formats may be used as known in the art.
The centre of gravity, CoG, —in vector notation
The location of the polygon, n, is represented by the vector
Wherein
The person skilled in the art may consult www.geometrictools.com, handbooks and other resources for more details on this and other methods related to 3D models.
In
By manipulation, as described above, the occluded region (which is a multi-sided polygon) may be replaced by multiple triangles or other polygons, such as squares, with fewer sides than the occluded region. This may involve selecting a position 705 (coinciding with direction 713) within the occluded region 701 and adding triangle defined by sides of the occluded region along its edge and the position 705. This is illustrated in
Reverting to
The dashed curves 710, 709 and 708 represent different locations (depths) of the point 702. Dashed curve 710 represents that the volume of the 3D carcass model is increased whereas dashed curves 708 and 709 represent that the volume of the 3D carcass model is decreased (eroded).
As illustrated, the occluded region 701 is modified to comprise at least one additional point 702 or 802. However, multiple points may be added to the 3D carcass model e.g. to approximate a desired shape. The desired shape may be defined by a class of polynomials or Splines representing curvatures observed in carcasses. In some embodiments the manipulation of the 3D carcass model includes adhering computationally to such a class of polynomials or Splines.
As explained, the case may also be that the density of image information from the inside of the cavities is not lacking, but is relatively poor, which also leads to the 3D carcass model providing only poor or lacking spatial information at those regions. One strategy is to relocate the points in such areas instead of removing them. Another strategy is to both relocate the points in such areas and to add additional points.
The carcass 114 is located at a position at the 3D multi-view imaging station 102. At this position the horizontal position of an estimated centre of gravity—before the 3D carcass model is manipulated—is illustrated by a first vertical dashed line 902. The first vertical line crosses the first centre of gravity 403 illustrated above in a cross-section of the 3D carcass model. At the same position the horizontal position of the physical centre of gravity is illustrated by a second dashed line 903. The physical centre of gravity is assumed to be located straight below the suspension point indicated by the mark 901.
By the manipulation of the 3D carcass model the distribution of volume of the 3D carcass model may be changed in accordance with one or both of a weight registered by a load cell and a representation of a suspension position at which the carcass is suspended—here represented by the mark 901. Thereby the estimated centre of gravity of the 3D carcass model, as illustrated by the first dashed line 902, is brought to coincide with the physical centre of gravity as illustrated by the second dashed line 903.
The mark 901 may be attached to the suspension device 112 or it may be attached to the conveyor. In the latter case the position of the mark is aligned with the second sensor 108. In some embodiments the mark 901 is dispensed with since the suspension device itself has graphical or structural features that enable accurate determination—via image processing—of a horizontal suspension position at least in the travel direction along the conveyor.
As mentioned above, the horizontal position of a suspension device may be a predetermined position determined in one or more of the multi-view images such as at one or more predetermined columns of pixels. A predetermined position may provide a sufficiently accurate positon.
The slaughterhouse production line 1016 comprises one or more stations. A first station 1002 is located upstream of the 3D multi-view imaging station 102. The first station 1002 may comprise facilities for receiving livestock and for slaughtering the livestock, whereby carcasses are supplied via the conveyor 110 to the 3D multi-view imaging station 102. Downstream of the 3D multi-view imaging station 102 the conveyor 110 may extend to bring carcasses to a second station 1003. The second station comprises multiple parallel conveyors (sometimes denoted rails) onto which carcasses can be controllably routed via a switch 1006 and from which multiple parallel conveyors carcasses can be retrieved via a switch 1007. The switch 1006 and the switch 1007 are controlled by the control system 1001 via respective controllers 1011 and 1012. The control system 1001 routes the carcasses based on their identification code and a classification of the carcasses based on the 3D carcass model. The station 1003 may comprise a cold store.
Further, downstream of the second station 1003, a third station 1004 may comprise a cutting machine 1008 which is controlled via a controller 1013. The cutting machine 1008 is controlled via cutting parameters computed from the 3D carcass model. The cutting machine may be a deboning machine. The cutting parameter may be a cutting profile parameter. The third station 1004 is sometimes denoted a dressing line.
After cutting and/or deboning, the carcasses arrive at a cold store 1005. As an illustrative example, the travel time for a carcass from being slaughtered at the first station 1002 to arriving at the second station 1003, when configured as a cold store, is less than about one hour.
The flowchart 1101 illustrates control of a cutting machine, such as a deboning machine. At a first step 1103 cutting parameters are computed from the 3D carcass model, which may be spatially modified as described above. Since the cutting parameters, at least in some situations, are computed from the 3D carcass model which is spatially modified, the yield from the carcass is improved.
At a second step 1104 the cutting machine, such as the deboning machine, is configured with to perform the cutting or deboning in accordance with the cutting parameter. At a third step 1105 a carcass present at the cutting machine is identified and cut in accordance with the cutting parameter.
The flowchart 1102 illustrates batch processing carcasses. At a first step 1106 a carcass is graded by performing grading. Grading may comprise associating a carcass identified by an identification code with one or more of: age, sex, breed, weight, conformity, and compliance with ecologic breeding. ‘Conformity’ comprises one or more parameters representing one or both of shape of predefined portions of the carcass and volume of predefined portions of the carcass. In some aspects the predefined portions comprise the whole carcass.
Then, at step 1107 a carcass is classified into a class e.g. into one class among e.g. 20 to 50 classes of carcass. The classification may be stored in an electronic record with the identification code of the carcass. Further, based on the classification, a carcass may be assigned a rail at the second station 1003. When the carcass, which is classified, arrives at the second station 1003 the switch 1006 is controlled accordingly to bring the carcass onto the assigned rail. In some embodiments there is one or more rails per class.
Steps 1106 of grading the carcasses and step 1107 of performing classification and routing the carcass onto the selected rail may be performed on an ongoing basis as carcasses travels from the 3D multi-view imaging station 102 and onto the rails. Concurrently therewith—or at intervals or at later points in time—a step 1108 of retrieving carcasses from a selected rail is performed. The carcasses may be retrieved as a batch one after the other from the selected rail. Hence, a cutting machine or deboning machine may be configured at step 1109 to cut in accordance with cutting parameters which is the same for the entire batch of carcasses. Then at subsequent step 1110 the entire batch may be cut or deboned in accordance with the cutting parameters.
In some embodiments the cutting machine 1008 is a system of machines such as a robot comprising a cutting machine or deboning machine.
Herein, the term ‘spatial density’ or ‘3D point cloud density’ is used to represent a local resolution (3D points per area unit) of the 3D carcass model. The 3D point cloud density may vary across the 3D model as described herein. A threshold density may be an adaptive threshold density, which may be determined locally across the surface of the 3D model.
The below is based on strategies adapted to availability of one or both of a weight of the carcass as obtained via a load cell and a representation of a suspension position in the domain of the 3D model.
In some embodiments the method comprises loading a predefined standard model of the type of carcass being subject to imaging. The predefined standard model may be a geometrical model defined by 3D points, curvature shapes in 2D or 3D or by metadata defining properties for the geometrical representation of a carcass.
In general the predefined standard model defines one or more constraints in addition to one or both of the weight obtained from a weighing cell and the suspension position. One or both of the weight and the suspension position may be a main constraint that must be satisfied, whereas subordinate constraints defined by or derived from the predefined standard model may be relaxed. Thereby, two disjoint regions may be manipulated based on one or both of the weight obtained from a weighing cell and the suspension position as a main criterion and a curvature criterion derived from the predefined standard model; the curvature criterion controlling distribution of the amount of volume manipulation to be applied. The curvature criterion may be defined explicitly by metadata comprising one or more parameters or ranges of parameters.
Subordinate constraints may also be defined by a predefined standard model comprising a representation of 3D points or 3D surfaces. The constraints may be defined by distances between 3D points in the generated 3D model subject for volume manipulation and corresponding 3D points in the predefined standard model. The volume manipulation may be subject to optimization of a cost function representing a sum of distances between corresponding points.
A predefined standard 3D model may be used to partially or fully support modifying regions of the 3D model. In connection therewith correspondence between positions or regions of the predefined standard 3D model and the 3D carcass model which is obtained by imaging and manipulation, is obtained by enabling navigation in the respective 3D models. Thereby a predetermined region can be located in both models. Alternatively or additionally an identified first region in say the predefined standard model can be used to locate a corresponding first region in the 3D carcass model. In some embodiments navigation is based on identifying one or both of key properties, such as orientations and magnitudes of inertial axes (moment of inertia per axis), and key points on the 3D the surface of the 3D carcass model. Once these key properties and/or key points are determined, similar locations on the models may be well defined and used to navigate the respective models. A person skilled in the art will be familiar with such navigation. This may also define the scale and rotation of the 3D model with respect to the predefined standard 3D model.
In some embodiments there is no need for further manipulation of regions that have been replaced by that of a standard reference carcass. In some embodiments these regions are subsequently subject to scaling (along the geometric indicator direction) to satisfy the constraints on the mass and/or the location of the suspension point.
1) Embodiments Based on Determining a Weight of the Carcass by a Load Cell
In embodiments wherein a weight of the carcass is determined from a weighing cell (e.g. giving a reading of 147.6 Kg or 150.0 Kg) there is generally one degree of freedom for manipulating the 3D model by the method given an objective of making an estimate (based on the manipulated 3D model) of the weight of the carcass correspond to or match the weight measurement value. Thus, in case no other constraints are applied there is generally one degree of freedom for manipulating the 3D model. Thus, a robust method for improving the accuracy of the 3D model involves manipulating the 3D model in one coherent region where the 3D point cloud density is relatively low (i.e. relatively sparsely represented), such as below a threshold density. Such a method has shown to lead to generally more accurate 3D models in the sense that a 3D model is improved at an identified, local region in the 3D model. The method may forgo manipulating the volume of the 3D model at other regions.
The 3D model may be manipulated by adding or subtracting a 3D volume at the identified region, to make the estimated weight based on the 3D model correspond to a weight measured by a weighing cell, by striking a trade-off between the size (area) of the region and the required magnitude (depth) of manipulation.
In some embodiments, the method comprises:
Thereby, a least amount of curvature manipulation is required, which may increase the likelihood of generating a robust 3D carcass model.
In case that two or more regions are identified since the 3D point cloud density is below a threshold density at two or more disjoint regions, the two or more regions may be merged into one region subject to a merge criterion. The merge criterion may comprise a maximum distance between regions for enabling merging of the regions and/or one or more other criteria. In case the maximum distance between the regions is exceeded the largest area among the identified regions may be selected for manipulation whereas the method forgoes manipulation of the one or more other identified regions.
In some embodiments, in accordance with two or more respective regions being identified, the magnitude of volume manipulation at the respective regions may be weighted in accordance with the expanse, e.g. in a 2D plane, of the respective regions.
In some embodiments the threshold density may be set, e.g. iteratively, to identify a predefined number of regions that are mutually disjoint.
2) Embodiments Based on Determining a Suspension Position in One or Both of a Lateral Direction and Transverse Direction:
In embodiments wherein suspension position is determined in one of a lateral direction or a transverse direction, there is generally one degree of freedom for manipulating the 3D model by the method given the objective of making an estimate (based on the manipulated 3D model) of the centre of gravity of the carcass correspond to the lateral suspension position. Correspondingly, in case the suspension position is determined in both a lateral direction and a transverse direction, there is generally two degrees of freedom for manipulating the 3D model. In some embodiments, the volume-weight conversion parameter is not required as an input for manipulations. This applies in cases where the carcass density is assumed to be uniform.
In embodiments where the carcass density is not assumed to be uniform, an absolute or relative multi-dimensional volume-weight conversion parameter may be used in accordance with the location of the region selected for manipulation.
The constraining strength of the horizontal position of the suspension point depends on the locations of the regions selected for manipulation. In some embodiments, measures are taken to reduce the risk of manipulating the 3D model by adding or subtracting volume centred about a vertical axis through the suspension point, since then there is a risk of ruining the 3D model rather than improving it. The measures to reduce the risk of manipulating the 3D model unsuccessfully or in vain, may comprise verifying that the volume manipulation given by its cubic volume and horizontal distance from a vertical axis through the suspension point changes the estimated centre of gravity above a threshold change. The threshold change may be determined from experiments and/or simulations. Thereby, only manipulations of the 3D model at regions of the 3D model resulting in sufficiently large deviations of the estimated horizontal position of the centre of gravity of the 3D carcass model may be selected for manipulations. The method may forgo manipulating the 3D model in case such a measure is not satisfied. The method may forgo manipulating the 3D model at an identified region in case such a measure is not satisfied for the identified region; and try another identified region.
In case the suspension position is determined only in a lateral direction or only a transverse direction, the method may forgo manipulation that changes the estimated centre of gravity in a transverse direction or in a lateral direction, respectively, such as changes the estimated centre of gravity only in a transverse direction.
In some embodiments the method comprises:
The above may be iterated until a desired number of regions for manipulation have been identified.
One or both of the first location criteria and the second location criteria is/are based on one or more of an estimated position of the centre of gravity of the 3D carcass model and axes of inertia of the 3D carcass model.
The replacing at least portions of one or more first regions having a size larger than the size threshold and having a location satisfying the first location criteria by a corresponding portion based on a predefined standard 3D carcass model may be based on navigation in the 3D carcass model and the predefined standard 3D model as described herein.
An example of the one or more regions having a size larger than the size threshold and having a location satisfying the first location criteria may be the fore quarter or the thoracic cavity of the carcass.
3: Embodiments Based on Determining a Weight of the Carcass by a Load Cell, and Based on Determining a Suspension Position in a Lateral Direction and/or Transverse Direction:
Such embodiments may have two or three constraints for manipulating the 3D carcass model. This means that two or three regions may be accurately manipulated all other things being equal. The weight as measured by a load cell establishes one constraint and makes it possible to constrain the 3D carcass model irrespective of the location of a first region. Determining the suspension position in a lateral direction and/or transverse direction establishes additional one or two constraints, but usefulness of the additional constraints for modifying the 3D carcass model depends on the location relative to the horizontal location of the suspension point.
The method may be based on additional constraints from a predefined standard 3D model as described herein and/or based on inter-dependence relations between manipulation performed in one region and manipulation performed in another region. An inter-dependence relation may favour or enforce a minimum volume manipulation. In one example an inter-dependence relation may distribute a volume manipulation between multiple identified regions in accordance with a weighted distribution of the volume manipulation.
There is also described a method as set out in the below item(s):
1. A method of imaging a physical object, comprising:
2. A system comprising a first set of cameras (104) arranged at a first side of a passageway (115) and capturing a first set of multi-view images from multiple positions, and a second set of cameras (105) arranged at a second side of the passageway capturing a second set of multi-view images from multiple positions; and
a computer (1010) programmed to capture the first set of multi-view images and the second set of multi-view images at a first point in time upon detection of an object being present at a predetermined position at the passageway.
3. A system according to item 2 above, comprising a conveyor extending along the passageway (115).
4. A method of imaging carcasses, comprising:
In some embodiments herein the volume manipulations are restricted to add, remove or distribute relatively small amounts of volume compared to the total volume of the 3D carcass model. Relatively small amounts of volume may be less than 10% or 5% e.g. less than about 2% of the total volume of the 3D carcass model.
| Number | Date | Country | Kind |
|---|---|---|---|
| 17160568.6 | Mar 2017 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2018/056288 | 3/13/2018 | WO | 00 |
