The present disclosure generally relates to automatically determining and assigning a score that indicates a level of integrity of a seal in a package irrespective of the shape of the package and the content inside the package.
The automated inspection industry is a very large and diverse industry (with annual revenues exceeding 30 billion dollars). A subset of this industry is focused on inspection of food, beverage and/or pharmaceutical products. The product inspection industry offers machines that check primarily for proper product weight as well as for existence of undesired foreign material in the packaged product. For simplicity, this inspection segment will be referred to as “food inspection” in the specification, though other packaged products are also within the scope of this disclosure.
Historically, very large concentration of global market share in the food inspection industry has remained with a handful of companies that enjoy established distribution chain in protected markets and hence tend to focus and rely on traditional inspection techniques that do not reply on advanced digital tools.
The state of the art in the food inspection industry can be best described as point solutions that are largely incrementally improved solutions based on capabilities developed long ago. Most production lines merely include the ability to check product weight and a device for inspecting if a foreign material exists. This type of disaggregated capability (i.e. lack of the ability to prevent defects rather than just detect them on a production line) makes it very hard for operators to get an insightful perspective on the quality of the product being produced. Therefore, defective products (for example, products that have undesired foreign materials and/or have less material than what the specification says) are produced more often than operators would like and these defects add extra costs to the business. Worst yet, defects that are not discovered during production may show up as recalls, and the company's reputation can be at stake.
In addition to providing less than optimal production insight, current state of the art inspection equipment fails to address several use cases that are critical to measuring product quality, particularly the measurement of sealing quality and seal strength for sealed products.
The following is a simplified summary of the disclosure in order to provide a basic understanding of some aspects of the disclosure. This summary is not an extensive overview of the disclosure. It is intended to neither identify key or critical elements of the disclosure, nor delineate any scope of the particular implementations of the disclosure or any scope of the claims. Its sole purpose is to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description that is presented later.
The present disclosure involves measuring the quality of the inspection process by assigning a score to indicate the level of integrity of a seal of a package. Note that though food is used as an illustrative example of what the package contains, the scope of this disclosure is not limited by what is inside the package. The scoring defines a complex process that distills a collection of data into a simple and easy-to-track metric, which is referred to as “seal score.”
The packaged product is run through an inspection machine, e.g., an X-ray or hyperspectral vision machine, to capture an image. Image processing techniques are used to identify the seal region from the image, which is then divided into a number of sub-regions. Seal scores for each sub-region are calculated using a mathematical framework that takes into account existence (or non-existence) of certain types of defects in the sub-regions. A report is generated to display the results of the seal score determination algorithm to the entity that requested the seal score.
The present disclosure will be understood more fully from the detailed description given below and from the accompanying drawings of various embodiments of the disclosure.
As hardware capability has increased, software has and continues to be a greater share of the enabling capability for machines to inspect merchandise during production. The hardware and software in unison measure the quality of the production process. Embodiments of the present disclosure are directed to, as part of quality control, automatically generating a seal score upon inspecting the seal integrity of a package irrespective of the shape and/or the content of the package. Though this seal score is described as a useful quality indicator during the production phase, the scope of the disclosure encompasses a distribution phase as well.
To leverage the quality control capabilities, it is important to think about inspection solutions as networks of machines that may be connected by a cloud platform that controls the machines, aggregates the data produced by them and then is capable of doing post processing on the inspection data to produce distilled insights, for example, producing a seal score. Using the food, beverage or pharmaceutical production line as an example, conceptually the quality control sequence can be divided into three stages: raw materials come to the plant, product is made, and product is packaged. The approach disclosed herein allows for real time quality assurance at each of these stages, and possibly across multiple production lines.
All the inspection devices are communicatively coupled with a Quality Management Platform 140—the dashed lines indicating communicative coupling. The platform 140 may reside in a server in a cloud, though in some embodiments the platform 140 may reside in a local network. The platform 140 receives inspection data from the various inspection devices (e.g., 110A-B, 120A-B, and 130A-B)
At operation 210, an image of a packaged product is retrieved. This image retrieval operation can be triggered automatically/semi-automatically/manually upon receiving a request for a seal score report (explained below) from a company. Alternatively, this operation can be performed as a quality control measure in one or more production lines. The image may be retrieved by running packaged products through an X-ray or hyperspectral vision machine or other image capturing devices. If more than one production lines are involved, a seal score report can be generated selectively from any of the lines. A sampling interval may be set to generate seal score reports from selected packaged products. Sampling can be random or deterministic along a selected production line and can jump from one production line to another.
At operation 220, a sealed region is identified from the retrieved image of the packaged product using image processing techniques. The sealed region can be along the periphery of the packaged product, as shown in
At operation 230, the identified sealed region is divided into one or more sub-regions. For example, in
At operation 240, a computer processor executes a computer vision model, to determine respective seal scores for the one or more sub-regions. This computer vision model could be any model that takes an image of a product as input and returns output data that can be used to make decisions about the product. An example mathematical framework for a possible computer vision model to address this problem is described in further details below, but there are many other possible approaches.
At operation 250, a report can be generated based on the respective seal scores. The report can take many forms, but the general idea is that the report is an indicator of a level of integrity of the seal in the packaged product. That way, the report facilitates in overall quality control. The report can help discarding defective products from appearing into the distribution chain. The report can also serve as a guide to correct the packaging process if seal scores do not meet certain threshold criteria set to certify the seal to have a desired level of integrity.
Specifically, in
In this example, mathematically, the area for each sub-region is defined as Aregion, where, the set region contains {top, bottom, side, corner, content}. Assuming there are five sub-regions, then the total region
The score percent of each region could be defined as Sregion, where,
Thus, the equation could be formulated as
The Side Seal includes Left Side Seal (325) and Right Side Seal (320). The Corner Seal includes Top Corner Seals (330 and 335) and Bottom Corner Seals (340 and 345). Thus, we define
In each sub-region, there can be a variety of defects. As an illustrative example, four types of defects are considered, namely, Product In Seal (PIS) (alternatively Product in Seam), Micro-leak, Seal Width, and Foreign Material. The respective weight percentages for each type of defect is defined as Dtype, where the set type contains {product, microleak, width, foreign}. Therefore, the equation for the defect types is formulated as
The Micro-leak defect type includes Fatal Micro-leak and Non-fatal Micro-leak. So we can formulate the Micro-leak defect as,
An indicator function I is defined as
where the set X denotes the seal region without a particular type of defect.
In this example, the quantitative value of the Seal Score is defined as
Q is in the range between 0 and 100, where a value of 100 indicates there is not any defect in any of the seal sub-regions, and a value of 0 indicates every type of defect exists in all the sub-regions. Based on the defect region and type, the Seal Score is defined as
In other iterations, the seal score does not have to be based on indicator functions, and the score for a certain region could be a continuous function based on the determined quality of the seal in that region, for instance.
The Table in
As mentioned above, the mathematical framework shown using a rectangular pouch can be extended to a more general framework for packed products with an arbitrary shape. For example,
Just like the rectangular pouch, for the packaged product shown in
Therefore the total area is:
The area fraction for each region is:
For different defect types Dj (where j=1, . . . , M), the indicator function is:
where the set X denotes the seal region without a j type of defect.
The quantitative value of the Seal Score is defined as
A specific example illustrating an application of the general framework is an oval shaped pouch shown in
If there is only Product in Seam (PIS) type defect in the seal region, then the seal score can be even simpler as:
The machine can be a personal computer (PC), a tablet PC, a set-top box (STB), a Personal Digital Assistant (PDA), a cellular telephone, a web appliance, a server, a network router, a switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.
The example computer system 700 includes a processing device 702, a main memory 704 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM) or Rambus DRAM (RDRAM), etc.), a static memory 708 (e.g., flash memory, static random access memory (SRAM), etc.), and a data storage system 718, which communicate with each other via a bus 730.
Processing device 702 represents one or more general-purpose processing devices such as a microprocessor, a central processing unit, or the like. More particularly, the processing device can be a complex instruction set computing (CISC) microprocessor, reduced instruction set computing (RISC) microprocessor, very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets, or processors implementing a combination of instruction sets. Processing device 702 can also be one or more special-purpose processing devices such as an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), network processor, or the like. The processing device 702 is configured to execute instructions 728 for performing the operations and steps discussed herein. The computer system 700 can further include a network interface device 708 to communicate over the network 720.
The data storage system 718 can include a machine-readable storage medium 724 (also known as a computer-readable medium) on which is stored one or more sets of instructions 728 or software embodying any one or more of the methodologies or functions described herein. The instructions 728 can also reside, completely or at least partially, within the main memory 704 and/or within the processing device 702 during execution thereof by the computer system 700, the main memory 704 and the processing device 702 also constituting machine-readable storage media. The machine-readable storage medium 724, data storage system 718, and/or main memory 704 can correspond to a memory sub-system.
In one embodiment, the instructions 728 include instructions to implement functionality corresponding to the seal score determination component 713. While the machine-readable storage medium 724 is shown in an example embodiment to be a single medium, the term “machine-readable storage medium” should be taken to include a single medium or multiple media that store the one or more sets of instructions. The term “machine-readable storage medium” shall also be taken to include any medium that is capable of storing or encoding a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure. The term “machine-readable storage medium” shall accordingly be taken to include, but not be limited to, solid-state memories, optical media, and magnetic media.
Some portions of the preceding detailed descriptions have been presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the ways used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. The present disclosure can refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage systems.
The present disclosure also relates to an apparatus for performing the operations herein. This apparatus can be specially constructed for the intended purposes, or it can include a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program can be stored in a computer readable storage medium, such as, but not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.
The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct a more specialized apparatus to perform the method. The structure for a variety of these systems will appear as set forth in the description below. In addition, the present disclosure is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages can be used to implement the teachings of the disclosure as described herein.
The present disclosure can be provided as a computer program product, or software, that can include a machine-readable medium having stored thereon instructions, which can be used to program a computer system (or other electronic devices) to perform a process according to the present disclosure. A machine-readable medium includes any mechanism for storing information in a form readable by a machine (e.g., a computer). In some embodiments, a machine-readable (e.g., computer-readable) medium includes a machine (e.g., a computer) readable storage medium such as a read only memory (“ROM”), random access memory (“RAM”), magnetic disk storage media, optical storage media, flash memory devices, etc.
In the specification, embodiments of the disclosure have been described with reference to specific example embodiments thereof. It will be evident that various modifications can be made thereto without departing from the broader spirit and scope of embodiments of the disclosure as set forth in the following claims. The specification and drawings are, accordingly, to be regarded in an illustrative sense rather than a restrictive sense.