This invention relates to a method and apparatus for inspecting the surface of articles and in particular, but not exclusively to a method and apparatus for the surface inspection and subsequent grading of fruit.
Processing lines for articles such as fruit often require different quality articles to be separated. This is an important commercial function, which may be used for example, to distinguish between fruit destined for export and fruit destined for the local market. Manually grading articles is costly, slow and prone to inaccuracies. Thus, automated methods and apparatus for grading articles have been developed.
The use of one or more video cameras is one known method of automatically sensing characteristics of articles. For example, the method and apparatus described in the specification of U.S. Pat. No. 4,825,068 (Suzuki) uses a video camera in conjunction with mirrors to obtain a picture of each article as it travels past the line of sight of the mirrors. The mirrors enable a larger portion of the article, which is typically generally spherically shaped, to be simultaneously viewed by the camera, but some portion of the article remains obscured from view.
US2004/0247193 (Qualtrough) uses one or more cameras together with a mirror arrangement in order to capture images of a spherical fruit as it is rotated through an inspection site. By fully rotating the fruit and capturing images on side of the conveyor, a composite picture of the entire surface of the fruit can be generated. Image processing can then be used to identify defects in the fruit from the composite picture as well as other factors such as color, and size which may be used for grading and sorting of the fruit downstream of the inspection site.
The reference to any prior art in the specification is not, and should not be taken as, an acknowledgement or any form of suggestion that the prior art forms part of the common general knowledge in any country.
It is an object of the present invention to provide a method and apparatus for article inspection which will overcome or ameliorate problems with such a present or at least provide the public with a useful choice.
According to one aspect, there is provided an article inspection apparatus including: conveying means for conveying an elongate article through an inspection site and rotating the elongate article about a lateral axis of the conveying means as it passes through the inspection site, the conveying means arranged to convey the elongate article with its longitudinal axis initially aligned with the longitudinal axis of the conveying means and wherein the total angle of rotation about the lateral axis is limited to substantially 180 degrees; an imaging system having at least one image capture means and image directing means arranged to provide to the image capture means a plurality of different views of the inspection site about the lateral axis of the conveying means; the image capture means arranged to capture a plurality of images of the elongate article at different rotational angles as it passes through the inspection site.
Such an arrangement provides good imaging coverage of the surface of elongate articles like cherries with intact stalk and which can only be partially rotated. This aids with defect identification and subsequent grading of such articles.
In an embodiment five views are provided about the lateral axis. This provides reasonably uniform image resolution over the entire surface of an article for the 180 degree rotation, and thereby improving defect detection.
In an embodiment the image capture means is one or more cameras, and the image directing means comprise a plurality of mirrors. This provides a cost effective arrangement for article inspection. The cameras may be arranged to take a series of still images of the views as the article rotates through the inspection site.
In an embodiment, a plurality of article inspection apparatus are provided in parallel in order to increase the inspection rate of the articles. The image directing means of each inspection apparatus may be arranged to overlap such that the combined width of the plurality of inspection apparatus is less than the total of their individual width in order to provide a more compact system.
According to another aspect there is provided a method of article inspection including: conveying an elongate article through an inspection site and rotating the elongate article about a lateral axis as it passes through the inspection site, and wherein the total angle of rotation about the lateral axis is limited to substantially 180 degrees; capturing a plurality of images of the elongate article at different rotational angles as it passes through the inspection site, the images including a plurality of different views of the inspection site about the lateral axis.
In an embodiment, the views are provided at substantially the same longitudinal location on the conveying means.
In an alternative embodiment, the views are distributed along two or more longitudinal locations on the conveying means. In one arrangement left-hand views are provided to an image capture means at a first longitudinal location and the right-hand views are provided to a second image capture means at a second longitudinal location. This allows images to be captured for substantially or nominally the first 90 degrees of rotation about the first location and images for substantially or nominally the second 90 degrees of rotation about a second location. This reduces the rate of rotation required, which improves control of the articles through the inspection site. Faster rotation can cause the articles to jump between conveying means pockets, or it can make it more difficult to rotate by 180 degrees due to slipping. A further advantage is that by slowing down rotation, more images of the fruit surface can be captured for the same frame-rate camera.
In another aspect there is provided an article inspection apparatus including: conveying means for conveying an article through an inspection site and rotating the article about a lateral axis of the conveying means as it passes through the inspection site; an imaging system having at least one image capture means and image directing means arranged to provide to the image capture means a plurality of different views of the inspection site about the lateral axis of the conveying means; the image system further arranged to provide the different views at two or more longitudinal positions along the conveying means; the image capture means arranged to capture a plurality of images of the article at different rotational angles as it passes through the inspection site.
Such an arrangement allows for the reduction of the rate of rotation required, which improves control of the articles through the inspection site. Faster rotation can cause the articles to jump between conveying means pockets, or it can make it more difficult to rotate by 180 degrees due to slipping. A further advantage is that by slowing down rotation, more images of the fruit surface can be captured for the same frame-rate camera.
In an embodiment, left-hand views are provided to an image capture means at a first longitudinal location and the right-hand views are provided to a second image capture means at a second longitudinal location.
In an embodiment, the article is elongate and the rotation is limited to substantially 180 degrees.
In an embodiment, five views are provided about the lateral axis. In another aspect there is provided an article inspection apparatus including: conveying means for conveying an article through an inspection site and rotating the article as it passes through the inspection site; an imaging system having at least one image capture means and image directing means arranged to provide to the image capture means five different views of the inspection site about the lateral axis of the conveying means.
The use of five different views improves surface inspection coverage of the article whilst maintaining reasonable cost both in terms of capital cost and maintenance cost.
In an embodiment the image system is further arranged to provide the different views at two or more longitudinal positions along the conveying means; the image capture means being arranged to capture a plurality of images of the article at different rotational angles as it passes through the inspection site.
In an embodiment, the views are provided at substantially the same longitudinal location on the conveying means.
In an embodiment the article is rotated about a lateral axis of the conveying means as it passes through the inspection site. In an alternative arrangement the article may be rotated about a longitudinal axis.
In an embodiment the image capture means is one or more cameras, and the image directing means comprise a plurality of mirrors.
In an embodiment the conveying means is arranged to convey an elongate article with its longitudinal axis aligned with the longitudinal axis of the conveying means and wherein the total angle of rotation about the lateral axis is limited to substantially 180 degrees.
In another aspect there is provided a method of operating an article inspection apparatus including: conveying an article through an inspection site and rotating the article as it passes through the inspection site; capturing images of five different views about a lateral axis of the conveying means.
The invention may also be said broadly to consist in the parts, elements and features referred to or indicated in the specification individually or collectively, in any or all combinations of two or more of said parts, elements, or features, and where specific integers are mentioned herein which have known equivalent in the art to which the invention relates, such known equivalents are deemed to be incorporated herein as if individually set forth.
Further aspects of the present invention, which should be considered in all its novel aspects, may become apparent from the following description, given by way of example only with reference to the accompanying drawings.
a: Shows a cross sectional view of an article inspection apparatus according to another embodiment.
a: Illustrates different views employed by an embodiment.
b: Is a detail of
a: Shows a side view of an article inspection apparatus according to another embodiment.
b: Shows a cross sectional view of the article inspection apparatus of
c: Shows a top or plan view of the article inspection apparatus of
The present invention relates to an apparatus and method for inspecting the surface of articles. The present invention may have application to the detection of features on the surface of fruit, vegetables or other articles for grading purposes. In particular, some embodiments of the present invention are envisaged to be particularly suited to the analysis of the surface of opaque articles that have a generally elongate cross section along at least one axis so that the article may have limited rotational freedom about this axis. Therefore, the present invention may have particular application to articles such as cherries with intact stalk and other articles having a similar shape. Other embodiments are envisaged to be suited to analysis of the surface of any shape article requiring reduced rotation rates for example to reduce slippage or misalignment. However, those skilled in the relevant arts may find applications of the present invention elsewhere.
Referring first to
In this embodiment
Right hand side views may be provided and corresponding article images captured with a corresponding imaging system on the other side; essentially a mirror arrangement to that shown. The imaging systems capturing left hand and right hand side views of the articles may be located at the same section of the conveying means 110 as illustrated in
a shows a right hand part of an imaging system according to another embodiment in which the same number of mirrors (two) are used for each view or path. An arrangement of mirrors which provide viewing angles of 0, +/−24 to 31 degrees (preferably 27.5), and +/−51 to 58 degrees (preferably 54.5) works well in practice to achieve a good level of article coverage whilst maintaining an acceptable level of complexity and cost.
Such conveying means are commercially available, for example from Compac Sort as part of their multi-lane sorter product and similar commercially available products.
a and 4b illustrate views and images provided by this embodiment in more detail. The additional side mirrors 345aL and 345aR provide two additional views at low angles compared with known arrangements which provide reasonably uniform image resolution over the entire surface of the article for the 180 degree rotation. This in turn enables improved defect detection by the image capture means. Typically both optical/color and infrared cameras would be employed and used for defect detection, however for simplicity of explanation only one camera is illustrated. Typically cherries are positioned on the conveying means with stem forward and rotated 180 degrees under the image capture means to have stem facing backwards. In the embodiment the image directing means (e.g. mirrors) of the imaging system are arranged to provide views from above the conveying means (0 degrees), at 27.5 degrees left and right of the top view, and at 54.5 degrees left and right of the top view. This provides high resolution views over the entire surface but whilst at the same time avoiding too low an angle for the mirrors which might attract dust, dirt and other debris thereby impacting on future imaging capabilities.
Three surface views of the cherry are indicated generally by 457a, 457b, 457c corresponding to the optical pathways 147a, 147b, 147c. Each of the views is provided to the image capture means or camera in adjacent strips on the cameras image collecting surface. Providing an additional view on each side improves the effective resolution of the views of the side of the cherries by providing substantially the same number of camera pixels for each side view as the top view. Adding such a wide angle view allows the image resolution to be kept as high as possible, and reasonably uniform over the entire cherry surface. This is illustrated in
The detail in
The articles are carried through the inspection site on a conveying means such as a conveyor 110, which rotates the article 120 during inspection. The conveying means may achieve this rotation through the provision of a number of substantially equally spaced rollers. A suitable conveyor for this purpose is described in international publication number WO 94/14547, which may also be used to direct graded articles to specific locations under the control of a suitable controller. Other conveying means such as individual trolleys may be used if required.
In an alternative embodiment, the mirrors 145a, 145b may be replaced with cameras if higher resolution is required; the additional resolution obtained at the cost of having to provide additional cameras.
Those skilled in the art will recognize that baffling, aperture stops and the like may be added to the system to prevent stray light entering the camera 1 from other sources. Also support structures will be provided to locate the mirrors 145a-d and camera 140 in position above the conveyor 110. The support structures may be of any suitable form that does not interfere with the optical paths from the article 120 to the camera 140 and also does not interfere with the lighting of the articles.
By having three optical paths with three corresponding views, an increased surface area of the article 140 may be viewed by a single camera. In addition, as each optical path intersects the article 140 at an angle that is substantially normal to the fruit surface, the effects due to the spherical shape of the article 140 are reduced. In combination with rotation of the article 120 on the conveyor 110, all of the surface of the article 120 may be viewed. This may be particularly advantageous for fruit, when features such as defects (for example cuts or splits) of less than one millimetre across may affect the fruit's grade. Such small defects may easily be missed if the system does not view the entire surface of the fruit.
Each field of view may overlap with its adjacent field of view to the extent that for most articles two views of the portion of the article 120 to be analysed are captured at each stage of rotation. This may be advantageous during image analysis. For example the or each camera may take multiple images of any surface features as the article 5 travels along the conveyor. Almost all surface features will be seen by the camera from the side and top views. A processing means 150 may then select the best image or combination of best images for analysis purposes, the selection process being performed according to some predetermined criteria, which may include the size and position the feature occupies in the image. Using only the best images for analysis purposes may increase the reliability of the feature identification process.
Light sources are provided to illuminate the article 120 so that the camera 140 may adequately detect the surface characteristics of the article. The light sources may be fluorescent tubes or linear LED extending longitudinally along the conveyor 110 and located so as to not obscure the article from the camera 140.
Referring to
By providing the different views of the rotating article at two (or more) longitudinal positions along the conveying means, the rotational speed of the article can be reduced resulting in improved handling of the article. This also allows a slower, cheaper camera to be used and results in less motion blur of surface details. The embodiment may be adapted to elongate articles such as cherries with intact stalk so that total rotation is limited to substantially 180 degrees. However, other types and shapes of articles may also be imaged through such an apparatus and different total rotational angles employed.
An advantage in extending the longitudinal length of the conveyer over which an elongate article is rotated, is to enhance the stability of this rotation. Because such articles are not rotated prior to the inspection site, suddenly rotating the rollers to rotate the article can result in slipping of the roller with respect to the article before sufficient friction results in rotating of the article itself. The point at which the article itself starts to rotate is randomly distributed across a large number of articles so that in prior known single inspection site systems the rollers have typically been controlled to rotate at a higher speed than required in order to ensure that the elongate article completes a full 180 degree rotation even if this is delayed. However a problem with this approach is that when a cherry or similar elongate article completes its 180 degree rotation before the end of the inspection site, the continued rotation of the rollers carrying the article can cause this to jump out of the pocket in which it is carried into an adjacent pocket resulting in imaging problems as well as conveying problems and wastage particularly if the pocket is already occupied by another cherry. Slowing down the rotational speed of the rollers reduces slippage at the start of the rotation and also limits jumping out of the pocket at the end of the rotation. There will still be some variation in when individual cherries start to rotate, however by appropriate control of the roller rotational speed and arrangement of the inspection site a full 180 degree rotation for every cherry passing through the inspection sites 730L and 730R is more certain. As a result of this, imaging and defect detection is improved and wastage of articles is reduced.
A further embodiment which employs only two side cameras but positions these at different longitudinal positions along the conveying means is shown in
A processing means 150, such as a digital signal processor, microprocessor or similar receives the information from the cameras 840L and 840R and tracks the movement of the article from the views of camera 840L into the views of camera 840R. Sequences of images from camera 840L and camera 840R form a composite image of the surface of the article. The processing means analyses a band from the image of each frame taken by the cameras. By rotating the article while it is within the field of view of each camera, a full picture of the entire surface of the article may be obtained.
Regarding the top views from each of the cameras only, while processing images from only one camera, the processing means 150 need only process a constant-sized region of each top view image it acquires. The width W of this region is equal to the product of the rotational speed and the camera imaging period. It is independent of the rotational diameter of the article 820. A larger article will take longer to complete a full rotation than a small article, but the width of each region processed is the same. For cherries the diameters would be similar.
The processing means 150 may use geometric and timing considerations to identify the top view images from each of the cameras 840L, 840R that should be analysed. The sequence of side view images from each camera will contain substantially the same region near the article's axis of rotation, repeated from many perspectives. Therefore, analysis of the side view images of the article may be formed by averaging over the image sequence. Alternatively, the analysis of the side view images may be formed by selecting one or more images from the image sequence, or extracting segments from each image to create a composite image.
When processing of the top views transfers from the camera 840L to camera 840R, the width of either the last region processed from camera 840L, or the first region processed from camera 840R, or both, will not be the normal width W. An adjusted width W′ may be calculated taking into account factors such as the camera height, separation and viewing angles (parallax), the overlap distance between the camera fields of view, the rate at which the processing means acquires images from the cameras, the linear and rotational speeds of the article, the size, shape and curvature of the article, and so on. The value of W′ is dependent on the specific geometry of the imaging system, conveying system, and articles. The degree of precision required in establishing W′ will also vary between applications, allowing a variety of approximations, or even non-geometrical techniques (for example image correlation), in its calculation.
In one embodiment, the height of the cameras above the conveyor is established through a calibration process. The fields of view of the cameras are also aligned so as to overlap by a predetermined distance. The overlap distance should be as small as possible to gain the maximum viewing area over the conveyor region, but must be large enough to see the relevant parts of the article, according to the inspection requirements. Calibration of the camera height and other parameters may be performed by a process such as that described in “An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision”, Roger Y. Tsai, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, Fla., 1986, pages 364-374.
Light sources 880 are provided to illuminate the article. The light sources preferably provide substantially uniform lighting over the article along the full length of the views through which the inspected article moves and rotates. The lighting is preferably uniform on the surface of the articles and is not necessarily uniform on the flat conveyor surface or plane. By providing uniform lighting, artificial bright patches on the items caused by non-uniform lighting are avoided which may be confused with natural bright patches. The light sources also provide enough light to get adequately bright images of the articles at the conveying speed and rotation speeds required, since short exposure times may be required to reduce motion blur effects.
The cameras and mirrors may be enclosed in a positive pressure enclosure that is supplied by filtered air to reduce the need to clean the mirrors and lenses. As the mirrors and cameras are located substantially above the line of the articles, the amount of dirt and other contaminants that can be expected to reach the mirrors or the enclosure may be reduced.
Referring next to
In some embodiments a centre of each end view camera 901, 902 is substantially aligned with at least one major axis of the conveyor. In the embodiment shown in
In preferred embodiments the end view cameras 901, 902 are positioned such that their viewing angle is as close as possible to being parallel to the longitudinal axis of the conveyor as possible. To achieve this the offsets mentioned above may be minimised and/or each camera 901, 902 may be placed at the longitudinal extremes of the inspection site.
Although
In embodiments of the invention having multiple conveyors (not shown) one laterally offset end view camera 902 may be positioned equidistant between two adjacent conveyors in order to capture images from both conveyors.
In the embodiment shown in
In other embodiments (not shown) the system may use one or more mirrors in place of the or each end view camera 901, 902. In some embodiments the end view images are reflected to a camera which is also used for capturing lateral images of the article. In most embodiments the end view camera(s) will provided outside a positive pressure enclosure which is provided for the lateral view cameras/mirrors. Accordingly, the end view cameras may also be provided with a self cleaning system. Independent lighting may also be provided.
By providing an end view of the articles, end cameras 901, 902 increase the proportion of the article which is inspected with adequate resolution, particularly for articles such as cherries and the like which are elongate and so cannot easily be rotated by more than around 180 degrees as they travel through the inspection apparatus.
It will be appreciated that although the embodiments have been described separately, features from one embodiment may be implemented into another, for example the light sources 880.
Where in the foregoing description reference has been made to specific components or integers of the invention having known equivalents then such equivalents are herein incorporated as if individually set forth.
Although this invention has been described by way of example and with reference to possible embodiments thereof, it is to be understood that modifications or improvements may be made thereto without departing from the scope of the invention as defined in the appended claims.
Number | Date | Country | Kind |
---|---|---|---|
616191 | Oct 2013 | NZ | national |
This application is a continuation of PCT Application Serial No. PCT/NZ2014/000212 filed on Oct. 3, 2014, which claims priority to New Zealand Application No. 616191 filed Oct. 4, 2013.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/NZ2014/000212 | Oct 2014 | US |
Child | 14566561 | US |