Patent Application
20200331645
Efficient handling of many different types of items that exhibit a variety of shapes and/or sizes, can pose an increasingly complex technological challenge. For example, produce items alone offered by a conventional grocery store, may exhibit sizes ranging from that of a raisin to that of a watermelon. Moreover, the quality of such produce items can degrade over time, affecting their monetary value.
Product handling systems according to embodiments facilitate transfer of individual product items from incoming bulk form into dedicated trays for subsequent inspection, sorting, selection, and packaging for consumption. Inspection may comprise interrogation of product items within a tray by electromagnetic (e.g., optical, hyperspectral) or other (e.g., physical, acoustic, gas sensing, etc.) techniques. Prior to packaging, product items disposed within the tray may be stored in a moveable carousel that is responsible for controlling environmental factors such as temperature, humidity, illumination, ambient gases, product-to-product interactions, and/or others. Movement of product items from a carousel's transfer station to an outside staging position may be accomplished using robots and/or conveyor belts. Embodiments may allow rapid, low-cost consumer selection of specific individual product items based upon their accompanying metadata (e.g., source, identifier), in combination with the results of inspection (e.g., visual appearance). Some embodiments may receive product items that are already pre-packaged in tray format in order to expedite inspection, sorting, selection, and packaging.
An embodiment of an apparatus comprises a frame translatable to a staging position in front of a carousel, and a fork supported by the frame and configured to extend in a direction toward the carousel to engage a tray holding an item. The apparatus further comprises a moveable member configured to project vertically into a first opening of the tray, contact the item at a first point, and to dispense the item from the tray in a direction away from the carousel, into packaging.
The patent or application file contains at least one drawing executed in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.
In particular, product handling system 100 comprises an initial sorting, inspecting and tray loading areas 102 that are configured to receive bulk items. This input region 102 is in turn in communication with tray conveyor network 104.
In this particular embodiment, the tray conveyor network is located above a second customer conveyor network 106. However, this is not required and the relative vertical locations of the first and second conveyor networks could be reversed, or in accordance with still further alternative embodiments, the two (tray, customer) conveyor networks could be positioned at a same vertical level. According to yet other embodiments, the separate customer conveyor network that delivers packages for output, can be located on an opposing side as the incoming tray conveyor network.
Trays are fed along the tray conveyor to the input area from a tray buffer 108. The tray buffer in turn receives the trays from a tray washer station 110 that functions to sanitize the trays once they have completed their previous product handling activities.
As further shown in
As previously mentioned, the product handling system of
For particular embodiments utilizing conveyor belts, those conveyor belts may cause fixed separation between transported trays and/or packaged product items. The belts may be imaged to determine cleaning needs. A conveyor belt may be configured to lift layer-by-layer, and may feature grooves and or duster-like lifters. Certain embodiments may provide a human-assisted station sharing a conveyor with robot lifters.
Via the customer conveyor network, the customer packaging deployment tool 114 provides empty product packaging (e.g., a bag or box) to the customer packaging loading station 118 that is proximate to the carousel. There, individual product items are loaded from the tray into the product packaging, and transported via the customer conveyor network to a shipping dock 120 and thence to the customer in packaged form.
The unloading equipment 130 will remove from the tray, the particular item that has been requested by the customer. The tray will return back into the carousel until called again or empty.
Once the order is fulfilled, the customer conveyor network will transport 131 the selected product item 124 together with the customer packaging 131 to the shipping or distribution area.
It is noted that is some embodiments, the conveyor belts transporting the customer package, may also move a passive or active device adjacent to some or all of the packages. That device may assist the placement of the items into the bag or box.
An example of a passive device could have spring loaded ‘landing pads’ or an active height controlled ‘scoop’. With the latter, the items land into the scoop (which can change height, and then drop into a package). Another example of an active device could be a robot. The devices (which may also have cameras) can be powered through the conveyor belt (wired or induction) or be battery powered.
While the above description has indicated the loading of incoming product items onto trays, this is not required by all embodiments. According to alternative embodiments, items incoming to the product handling system may arrive already disposed onto a tray.
An example could arise for standard items (e.g., boxes of cereal). There, each cereal box product item could be associated with data such as an expiration date (and potentially an individual serial number).
Trays could arrive pre-loaded with non-standard items (e.g., produce, individually cut deli/cheese/meat, others), and already be associated with individual product item data such as images or other sensor data. The incoming tray may also include a mechanism for measuring and storing environmental conditions since packing into the tray took place.
Such post-tray packing environmental conditions can include temperature and vibrations/impact (e.g., via a G meter). Thus a system according to an embodiment would receive the pre-packed tray for handling, as well as data associated with that particular tray.
According to certain embodiments, product handling systems may also be used to break down packages comprising one product each, into other packages having specific mixtures of product items. In such a ‘break pack’ implementation, a distribution center breaks incoming packages of one item type (e.g., typically from the factory) into mixed packages intended for the neighborhood grocery store to replenish what has been consumed (e.g., 3 boxes of cereal A, 6 tuna cans, 3 salt shakers, etc.)
It is further noted that temporal factors (e.g., delivery urgency) may impact the manner in which individual product items are dispensed from the carousel into packaging. Thus where necessitated by a delivery deadline, under a ‘split bag’ mode of operation a product handling system could utilize more than one customer package to fulfill a customer order. There, the more urgently needed item(s) would dispensed into packaging first, followed by less urgent items being dispensed and shipped in a different package.
Product handling systems according to embodiments could also pre-calculate and store estimates regarding time of order fulfillment. Such expected fulfillment data could be referenced by the system in deciding whether or not to resort to the split-bag mode in order to meet an urgent order.
An exemplary sequence of actions that may be performed by a product handling system according to an embodiment, is now described in connection with the flow diagram 1000 of
The transfer of product material into the trays can be automatic or performed manually. In the case of automatic transfer into dedicated trays, products are placed or dumped by a human or mechanical tool, or picked by a robot onto a conveyor.
At 1004, the conveyor will sort the individual product items in an ordered fashion for inspection. During an inspection process 1005, the individual product items are imaged/inspected, and the images/data are associated with each item (e.g., as metadata).
After inspection, the system will load 1006 each product onto trays and transport the trays to the storage area (e.g., carousel). In some cases, the items are sorted in the Trays based on certain criteria (e.g. different size, quality or ripeness in different Columns)
Once transferred to a known location on the tray, at 1008 that information will be maintained at least until the item has been placed in packaging for delivery. All or some of the data and metadata may be stored for longer to permit learning once feedback from a customer or other entity is obtained on the items.
At 1010, the Trays are loaded into carousels. The trays may be loaded manually or automatically. The trays may be loaded through the same door from the product items are later dispensed, or (as in the illustrated examples) through a separate door or level. In one possible alternative, the trays remain within the carousels and the items are loaded individually. Alternatively, the trays are loaded onto other trays next to a carousel and placed in the carousel manually.
At 1012, when an item is selected by a customer and needs to be dispensed into packaging, the carousel may bring the tray to the dispensing door. There, the tray may be moved to the staging position from which individual items will be dispensed. The tray may be moved by a conveying element within the carousel, or by an external lifting mechanism (e.g., belt or robot). Alternatively, the individual product items could be dispensed from the tray while the tray remains within the Carousel.
From the staging position, at 1014 items are dispensed from the trays into packages (e.g., delivery, inner, or transient) which are moving along an aisle conveyor. Alternatively, the packages may be moved by an independent motion vehicle/robot.
The aisle conveyor is positioned to support one or more parallel rows of carousels along its direction of travel. The aisle conveyor carries the various packages and potentially the inbound trays (e.g., with product items) and outbound trays (empty, expired, or otherwise pulled out of the system).
The packages may be moving continuously or indexing and stopping near the staging positions. The packages may stop at every staging position along their aisle or as needed. The packages may step at fixed intervals or as needed.
Packages may be placed into an aisle conveyor in coordination with placed orders and the expected availability of items for conveying from staging positions along the aisle. Alternatively, transient packages are places at fixed intervals.
Specific packages (e.g., including specific inserts if needed) may be placed in the aisle conveyor for specific orders. If the exact number of packages needed per order is not accurately known in advance (e.g. a certain order may not fit in one package) than extra ‘buffer’ packages can be placed in the aisle conveyor every several packages.
A package may travel on more than one aisle conveyor to collect the needed items for a particular order. Alternatively, the delivery package may travel on one or more aisles and either/or transient packages or internal packages will travel down one or more other aisles.
In this case the items may be merged into one or more delivery packages manually or automatically. Additional items deemed too large or fragile for automation could also be manually added to the shipment at the end of the aisle conveyor.
As described herein, robots may be used in one or more stages of the product handling sequence. Examples of product handling activities that can be performed by a robot can include but are not limited to:
There can be one or more robots per location in the product handling system. Robots could be optimized in terms of weight and grip to the specific product items and/or system components (e.g., trays). One robot can have several grippers, switch grippers, or each of the robots can have a different gripper.
The robots may reference various pieces of information to grab a product item. Examples of such information may pertain to tray layout, images taken on the main or side-conveyor before, and/or dedicated images taken proximate to the robot (or on the robot's arm).
Robots can be of various types, including but not limited to cartesian, Selective Compliance Assembly Robot Arm (SCARA), cylindrical, delta, polar or a 4- or 6-axis articulated robot. Robot grippers can use pressure by rigid or flexible fingers, vacuum/suction, magnetic, electrostatic lifting, leaky vacuum (e.g., Bernoulli lifters), or a combination thereof.
A robot may use features within the trays in order to allow the lifting of product items, especially delicate/damageable items. In certain embodiments these can be rake/dustbin like, grabbing items which have been pin-lifted (e.g., from below with a potential balancing/locking ‘thumb from above).
Packaging can pause adjacent to carousels or can have a parking area for one or more (e.g., three) customer packages. Multiple packages outside of a carousel may all be reached by the robot, or they may need to move to a specific parking position for the robot to reach.
Owing to its intimate relationship with the product being handled, the tray component forms one component of a product handling system. According to certain embodiments a tray may be wider than it is tall.
In some embodiments, a tray may comprise a formed sheet of plastic or metal, that holds individual items or containers of items, in a controlled order while the tray moves through the system.
Product items may be disposed on the tray loose (e.g., an individual apple). Alternatively, product items may be contained within a box (e.g., a box of tomatoes).
A tray may receive one type of item as a standard, but may also receive more than one variety. This may occur for a low consumption items. A tray may receive a subcategory of an item (e.g., oranges between 3 and 4 Ounce Vs oranges 4 and above.)
Some product items may be partially packaged. Examples include parsley tied with a rubber band, bunches of bananas, etc.
Product items may be individually labeled, for example with bar codes and/or RFID tags. The tray itself could have a bar code, RFID tag, or some other marking to permit tracking if needed.
Product items within the trays may be arranged in rows and columns. Rows may be oriented parallel to the edge of the tray from which they are unloaded. Columns may be separated by barriers to allow product items to arrange within the columns utilizing grooves or other shaped features.
Specific embodiments may sort the items such that each bin is within a different Column. Here, bin may represent the quality, size, appearance, or a different product type.
Trays may be open, or have an opening wall at one or two of the edges parallel to the rows. Alternatively, trays may be closed on all sides.
Trays may have openings, grooves, holes, or other features to allow lifting and/or movement of the product items from below.
Certain tray features may hold a particular type of product or a certain variety (e.g., stone-fruit of a given size range, oval shape, loose-leaf, etc.)
Trays exhibiting different features may be employed to effectively handle various product types. Trays may maintain product items at a known location that will not change during transportation.
Trays may be designed with sufficient spacing and other features to permit tasks such as imaging, scanning, sensing, and/or lifting. Trays may exhibit features to allow removing the tray for packaging, or removing the tray from the product handing system.
Embodiments of product handling systems may inspect the trays for cleanliness and integrity. Trays can be single or multi-use.
Some tray embodiments may feature a multi-use part covered by a single-use layer. Such a single-use layer may comprise paper, plastic, cardboard, or other materials.
Embodiments of product handling system may include mechanisms for cleaning the trays after some or all the product items have been removed. Cleaning can include washing, brushing, electrostatic discharge, UV, steaming, or other disinfecting techniques.
For purposes of imaging, product items that are to be transferred to trays, may be loaded into transfer conveyor belts. Such conveyor belts may have imaging and/or other sensing stations.
Examples of imaging techniques that may be employed can include but are not limited to:
Examples of sensing technology that involve other than electro-magnetic imaging can include but are not limited to:
All items, or a sampling thereof may be imaged and/or sensed. The product items can be imaged or sensed individually, or collectively or in sub-groups.
Weight can be estimated from produce size and/or from produce size relative to other produce in the tray when the total weight of the items in the tray is known.
Weight can be determined if the item is lifted by a robot. That weight can be used to improve the estimation.
The imaging and/or sensing may occur while the product items are located on a transfer conveyor belt, are grabbed by a robotic arm, or as the product items are rolling or dropping before being disposed on a tray.
A transfer conveyor may be optically transparent to permit imaging of product items from multiple sides and/or accessible angles. However, items can also be grabbed and lifted for inspection, imaging, or sensing, or simply raised such as via cushioned pins projecting though holes in the conveyor belt and/or tray.
Additionally, secondary imaging/inspection can occur during storage, or prior to dispensing a product into a package. Such imaging/inspection can be used as a final go-no go verification step after the attributes of the product item have been determined from previous imaging.
Other sensors and/or imagers may be used to confirm the successful transfer of trays, items, and packages.
As described below in detail below, the product handing system may comprise the trays utilized in conjunction with carousel elements. One or more sensors or cameras may be installed within the carousel.
Such carousel sensors/cameras could image some or all of the items as the carousel rotates (e.g., a part of regular motion or to specifically allow imaging). Several images could be taken during the motion to permit viewing from different perspectives, and even optionally 3-D reconstruction of the product.
Some embodiments could install cameras at the top of the carousel to afford a view of the top tray. The cameras can be positioned on top in the front and in the rear in order to ‘triangulate’ an image.
Items can be imaged once upon entering, periodically (such as every day, using the off-hours for example) or based upon other rules. Items can be imaged from the top while supported in the trays, and then flipped over individually or as a whole tray to be imaged from the other side, or otherwise manipulated to rotate so they can be imaged from the other side.
The imaging can take place on the main conveyor belt, and/or on the side conveyors, and/or at the storage cabinets, and/or at the parking spots before a robot manipulates the item.
A block may represent a plurality of trays. A given block may contain a set of items (e.g., all fruit and vegetables, all dairy, etc.) and is expected to hold many (e.g., tens, hundreds, or even thousands) of unique items.
A block may contain all or some of the items for sale. The block may hold several trays of frequently accessed product items in order to ensure rapid supply. Conversely, a block may have partial trays or even none of rarely-accessed items that will instead be loaded upon demand.
The conveyor belt may lead to a set of modules that may be situated orthogonally on one or both sides of the conveyor belt. The modules may be dedicated for a specific variety of items (like fruit versus vegetables, packaged product items versus loose product items, etc.). Specific trays can be moved to side conveyors of individual modules as they pass by on the conveyor.
According to embodiments, a loading mechanism may be employed to move product items to and/or from the trays. In particular, once product items have been inspected (e.g., by imaging and/or sensing), they are sorted onto trays.
One method to accomplish this sorting is to move the items along conveyor belts. The trays (e.g., with slots) can be positioned above a lifting conveyor. The lifting conveyor will lift through the tray and thereby allow the product items to be conveyed over the tray.
Once in place, the conveyors will move down and the items will rest on the tray. This is depicted graphically in the plan view of
Various stations could be used to match the variety of the items. Alternatively, an automatically adjustable conveyor could accommodate all or many of the possible varieties of product items.
In certain embodiments, trays having product items already disposed therein, will be loaded into carousel elements. According to an alternative approach, product items may be loaded into the trays that are already in the carousel.
The carousels may be oriented in a horizontal direction, containing several tens of trays. The carousels may exhibit the ability to bring individual trays to the transfer station 502, from which they can be moved onto the staging position 504 (
The carousels may operate in a continuously indexing manner, bringing following trays to an offload station. Alternatively, the carousel may call trays in turn as needed.
Each carousel may operate with its own specific controlled environmental conditions. Examples of such conditions can include but are not limited to:
According to embodiments, a carousel may have an opening (e.g., slit) for loading and unloading. The opening could be parentally open, an actively actuated gate, opened/closed by the motion of the tray, or closed with a material permitting tray motion (e.g., a plastic sheet).
Carousels may be designed to allow placement of trays with different spacings. This can permit handling of product items exhibiting different sizes (e.g., heights).
Alternatively, trays may be designed with fixed spacing. A control program may dictate which slot is appropriate for each tray height. A single carousel (one motor) could turn one or more columns of Trays.
Carousels may be designed with belt in cameras and sensors to continue to monitor the products as they age. One specific design could have the cameras or other sensors located at the top of the carousel path, inspecting the products in the tray passes past the top.
Additionally, there could be reject mechanisms built into the carousels to allow items to be automatically removed from the system. One design could employ lifting conveyors at the bottom of the carousel path. These conveyors are able to lift up and remove an item from the tray as needed.
Trays can be removed from the carousel manually or automatically. In automatic embodiments, the tray can be transported out by a walking beam, a walking beam conveyor, or a conveyor or rollers built into the carousel itself.
For walking beam embodiments, the carousel could stop with the appropriate tray in front of the dispense door. One or more beams would move into the carousel, lift the tray up and pull it out of the door and down onto a conveyor belt or rollers.
For the case where the conveyor is built into the carousel, each row of the carousel could have its own set of belts or rollers actuated to remove the tray. Each row could have its own motor, or a single motor may engage with the conveyors at the dispensing door.
Once located at a staging position, the tray will be in proximity to the aisle conveyor. This will allow items to be transferred from the tray at the staging position to the packages.
Referring now to
The product items may also be pushed, lifted, or dragged from the tray to that position for packaging. Product items could also be individually picked by a robot and an End of Arm Attachment (EOAT) specially designed for secure engagement.
Once the product item or items have been removed from the tray, the tray will be conveyed back into the Carousel to store the remaining items.
Details regarding a transfer mechanism for moving the product items apart from the tray according to embodiments, are now discussed. Specifically, once an item is out of a tray, it can be moved to the packaging by multiple methods.
One method is to have a robot pick up the item and place it into the packaging. Here, there may be specific EOATs for various classes and sizes of products. Some EOATs may have suction cups, others may have grippers, still others may shovel items up.
An alternative method is to have the product items fall off of the end of a conveyor into a receptacle such as a rigid box or flexible bag, which may or may not have padding to break the fall.
The item could fall onto a spring-loaded slide or flap that drops down into the bag or box to soften the landing. The bag or box could be actuated to lift the box up to the edge of the conveyor. The box could also be constructed in a manner that the one more sides of the bag or box temporarily collapse to allow items to be brought into them.
A third possible approach is to employ a vertical conveyor whose belt has large flaps to hold the product item.
The conveyor would lower into the bag or box, or the bag or box could rise up around the conveyor. The conveyor could be actuated in a manner to select the location to drop the item in the box or bag.
A fourth approach is to have the item move above the package on a conveyor. Once in position, the conveyor pivots down like a trap door.
The conveyors could lower down into the bag, controlling the decent of the item until it is beyond the reach of the conveyor. The belts could be made of a thick foam that helps capture the items on the way down. There could be two or more belts on each side with various spacing that can process various sized items. Each station could be designed for a certain shape, weight or volume. Or, one design could be actuated in a way that the location of each belt is movable to handle item variety.
A fifth method is to push or pull the product items from the back of the column or row of the tray. This could be accomplished from above via a mechanical gantry system, from below via an actuator through a slot, or utilizing a robotic arm.
A sixth possible method is to convey the product items through a hole with a stack of mesh or plastic rolled onto it. As the items are pushed through, the bag will roll forward until full and then be terminated and closed.
Certain embodiments could feature one transfer mechanism per tray. Other embodiments could feature one transfer mechanism per column or row of the tray.
There could be one or more transfer mechanisms per carousel. The transfer mechanism may be actuated in such a manner as to move between positions. Alternatively, the individual product items could be lifted onto conveyors at the staging position and then conveyed to one or more fixed transfer mechanisms.
Embodiments of product handling systems may permit customer selection of specific items. In some cases, e.g., produce or other non-uniform products (meat, deli cheese, etc.,) the consumer may be offered the ability to choose from a number of specific items.
According to such embodiments, images or other (e.g., sensing) data provided to customer, may allow him or her to successfully execute the product selection. The specific item may be reserved for a certain amount of time allowing the selection. Once selected by the customer, the specific item will be vended from the carousel.
Certain embodiments may allow dispensing only from the front row (e.g., closest to the package). There, customer selection may be limited to the front row, and/or selection of sequential items in the same column will be possible.
In alternative embodiments, a series of conveyors could move some items out of a column or row, so that an interior item can be vended to the customer. Then, the conveyors could move the remaining items back into the tray for return to storage.
Embodiments could offer the customer multiple qualities for evaluating when selecting their item. The customer could evaluate color pictures of the item from multiple angles. The customer could also be offered hyperspectral images.
A system according to embodiments could show a customer a rating that signifies the firmness of the item. The rating could be on a scale showing maximum firmness for similar items. For example, the firmest pear (right off a tree) could be rated with a value of 10, while a pear on the day of expiration could be rated a value of 0.
The scale could be adjusted based on the season of the item. For example, off-season produce might have a narrower range of possible ratings.
Similarly, a rating scale could be shown for various characteristics of product items. Examples can include but are not limited to:
Product handling systems according to embodiments may include merging and/or shipping areas. At the end of the aisle conveyors, other conveyor systems (or product moving mechanisms) may transfer packages either to other aisles or to a shipping area.
Multiple packages can be merged into one or more delivery packages. For embodiments featuring transient packages, individual items may be lifted or otherwise transferred (such as poured) into delivery packages.
Product handling systems may feature a buffer station where packages are waiting. This waiting may be for other Packages in the same shipment, or for other reasons.
The nature of delivery packages according to embodiments, are now described. Delivery packages can be bags, simple boxes, or boxes having spacers inside.
One packaging option is for single layer stackable boxes. Another option is to have boxes with several heights, with the box ultimately used determined by the largest item it is designated to hold.
A box height could be a designed to have 1 or more layer, with a packaging material for contact with fragile items, and remaining layers for regular items. Packaging for a fragile layer could have a formed plastic or cardboard insert sized to fit various types of items (e.g., round, oblong, flat, etc.)
Packaging may be single use from recyclable material, and may be recycled. Packaging could also be designed to be returnable from the customer.
Transient packages can be individual bags or boxes with items collected into a larger box. Transient packages can be a large tray on which items in an order are placed for transport to the packing area.
Several delivery packages can be collected for one customer order. This can, for example, be by employing vertical stacking or nylon wrapping or other technique.
Embodiments may employ techniques for controlling the queuing and/or timing of product movement through the product handling system. Particular embodiments may use information from orders received, as well as from orders that are expected. The latter may be accomplished through machine learning techniques that are keyed off of past behavior.
One example of product movement that may be coordinated, is to send packages into aisle conveyors. This package sending may be for delivery, internal movement only, or for transient packages.
Another example of product movement that may be coordinated according to embodiments, is to bring trays to an unload position from the carousels and to the staging position. This movement may need to be done in time to preempt the loading of specific items into the packages.
Still another example of product movement that may be coordinated is action to bring trays, transient packages, or customer pack boxes to the robot loading area. The activity of the conveyors, carousels, and loading mechanisms may be coordinated in order to maximize system throughput and minimize time from receipt of the order to packaging.
Embodiments of product handing systems may function to optimize location of items on trays, and the location of trays in carousels. This allows each carousel and packaging location to be fully utilized.
Embodiments may track performance of the various components. An overall system health may be displayed to the users.
Embodiments of product handling systems may also operate to plan the optimal packing, for example calling upon specific pack boxes as needed to fulfill each order. In one embodiment, the system may place larger, heavier items at the beginning of a cue so that those items are at the bottom of the packaging. Then, as the packaging gets filled, progressively lighter and more fragile items can be positioned on the top.
Embodiments could also function to determine where in the box a particular item should be placed. This allows knowing where product items are located through the entire packaging sequence.
The imaging and sensing data collected from the product items (as well as other available information such as images or other measurements/information taken before arrival at the facility) may be used to attribute properties to the item. Such other information can include data provided on barcodes or similar mechanisms such as RFID tags. These can be located on the individual item, on the incoming tray, or be traced to the incoming shipment. In some embodiments the other information may be measured ‘off-line’, such as a sample of fruits that are sent to lab tests or tasted.
The customer can be offered various packages if desired. In one example, produce could be selected based upon an expected day of ripeness. Thus an instruction received from the customer may be to “select 7 bananas, one that ripens every day for the next week”. Alternatively, a customer could request a box of fruit per day, each box ripening as needed.
In another example, a customer could select kits specifically intended for a certain recipe. Thus bananas slated for use in banana bread, could be ordered as bananas slightly paste their ripe stage.
Further alternatively, a customer could order a lasagna kit. During the selection process, the customer could be offered a list and choose some or all of the ingredients from that list.
Embodiments may allow external links to be placed on recipe sites, so that a user can add the recipe to their list of saved recipes. For example a customer may prefer the pasta sauce available from a particular website, and click the save recipe option. Then, that product item will be added to the customer's profile.
Later, the customer might look through their recipes and plan out meals for the week by selecting each meal and the day it is expected to be eaten. Embodiments could then sort through the order and select items of the best quality on the day scheduled for consumption.
According to embodiments, the product handling system could also be linked to other Internet-of-Things (IoT) devices that are able to determine or predict particular items already present in a customer's refrigerator or cupboard. Those particular items may then be able to be automatically ordered on a recurring time period. Alternatively, when a recipe is ordered, items already possessed by the customer can be automatically removed from the list.
A system according to embodiments could be linked to voice controlled devices. This allows a user to add an item to their grocery list so that it can be saved for a future order. A customer could accomplish this by asking to have items saved to their cart, or by notifying the system of a deficit of certain item. Information relating to particular dishes or recipes could be saved in a similar manner.
System users may have the ability to save past quality preferences. Thus if a customer ordered B level zucchini (for example), that customer could mark a box saving that preference for all future zucchini purchases. This shortens check out time and improves consumer confidence.
Imaging and other (e.g., sensing) parameters may be stored for sufficient time to allow comparison after customer feedback has been received. (This may be about a week for fresh produce purchases.) The stored information could be the raw data or a subset thereof that has been processed (e.g., for compression or encryption).
Parameters or ratings can be referenced to offer different pricing schemes or dynamic (adjustable) pricing based upon quality and/or supply-demand variations.
A customer can provide feedback on particular parameters of the delivered product. Examples of such parameters can include but are not limited to the quality, ripeness date, blemishes, or other parameters of purchased produce.
Systems according to embodiments may utilize machine learning to process feedback information together with other collected parameters (e.g., images of product items). This allows the system to improve an accuracy of predicting general properties (e.g., ripeness day, color, farm, and other attributes) as well as the preferences of a specific customer. Such data processing can aid customers in obtaining preferences, allow sellers to evaluate supplier performance, and permit suppliers to collect valuable quality control information and improvement feedback.
The user interface (UI) may include virtual or augmented reality images or scenarios. In one example, a user could enter a virtual supermarket where they can inspect and select the specific to be placed into their cart. Such a virtual store environment can be optimized to minimize time spent by down selecting items in an order they are shown according to preferences.
Product items not normally bought by a particular customer can be presented for inspection (including physical handing).
Recipes offered by the system accompanying product items, could be available for customer review to inform about possible uses for the product. If a recipe is selected, the user can be solicited to add the other recipe items to their cart. Alternatively or in addition, a user selecting a recipe could enter a virtual environment displaying finished dishes or recipes for ordering other ingredients as desired.
According to embodiments, a product handling system can provide estimated packing time based upon factors including but not limited to:
A product handling system according to embodiments could suggest changes in the order to reduce pack readiness time. The system could also actively suggest changes to the order based upon price considerations, for example to meet a budget requirement.
An example of a product handling system is now described. An embodiment of an automated system allows rapidly distributing a large number (on the order of many 10,000's) of unique incoming products into packages for customer delivery.
According to one embodiment, the system provides an automated grocery store offering rapid (e.g., minutes) fulfillment and packaging. In particular, grocery items typically involve 1,000s or 10,000s of unique items.
For example, individual product items in the field of fresh produce, may comprise a large volume of associated data. That is, an individual fruit or vegetable (e.g., an apple) may be associated with one or more of the following pieces of information:
This information could be stored in electronic format in a non-transitory computer readable storage medium. In one example, the information could be stored as fields in a data object stored in a database. The information could travel associated the particular product item, together with product handling specific data (such as a tray identifier, tray location identifier).
Upon user selection of the particular product item (e.g., apple), the product handling system could dispense the item to the consumer together with packaging (e.g., box, bag, together with any insert) within a short time frame.
While this particular example describes an embodiment that is designed for use in the packaging and delivery of fresh produce, this is not required. Other potential applications may call for the rapid fulfillment from a large number of product options, for example in the area of recycling of clothing or other items.
Particular embodiments may be especially suited to allow rapid distribution of a large number (on the order of many 10,000's) of unique incoming products into packages for customer delivery. By contrast, a conventional on-line retailer may carry millions of different individual items in warehouses for delivery.
Embodiments may thus offer a space efficient solution that may be local within proximity of customer demand. However, some embodiments could be located in larger warehouses outside of densely populated areas. Systems may permit a high level of quality control, allowing customers to avoid the inefficient travel to and from a store in order to select individual items for freshness and relevance.
In summary, embodiments may offer optical (hyperspectral, potentially 3D, potentially multi-angle) or other (like acoustic, pressure gauge, gas sensing systems, spectroscopy) inspection of some or all incoming material (e.g., loose produce) in bulk form.
Embodiments permit customer selection of specific product items (e.g., a particular tomato) based upon one or more of the following:
Embodiments may offer storage of product items within carousels that control multiple environmental factors (e.g., temperature, humidity, product to product proximity and interactions, mold growth, gas detection). Such a carousel-based storage system could be designed to allow rapid packing of individual products to provide rapid order fulfillment.
Embodiments may allow the vending of fresh produce into either a delivery package, an inner package, or a transient package. Here, an inner package represents a tray that will later go into a box with others. A transient package serves just to carry the product item(s) to a place where a shipment package is consolidated.
In the case of direct vending into a delivery package, that packaging may include a generic form of spacing or cushioning or an insert that will be placed before loading based on the expected items to be places into that delivery package (e.g., round indents for the packaging of stone fruit).
According to certain optional embodiments, inner packages, and/or individual items, and/or transient packages may be merged into a delivery package if appropriate.
In some cases, other material may be added to prevent damage to items in shipping or packaging. Examples can include but are not limited to cardboard, shredded paper, inflatable plastic bags, or other fillers. Alternatively, the individual dispensed product items may be covered with another flat or shaped surface or a stretchable or inflatable membrane for protection during shipping.
Embodiments may permit individual item selection by a consumer based on imaging and other sensor data. This selection may be enhanced by learning consumer preference(s) based upon image, collected data, and other sensors. This may be achieved independently, or in connection with automated features.
Various product handling systems according to embodiments may exhibit one or more of the following features. Certain embodiments may hold for a customer, a specific selected unique product items while they are in their cart. Some embodiments may save quality preferences of a customer for future orders.
Particular embodiments may feature recipe buttons that allow user to save a recipe, and select one or more particular product items of that recipe as part of an order. Product bundling features may facilitate a user ordering whole recipes, kits or a series of items. Various embodiments may allow optional browsing and selection by meal options instead of individual items.
Voice controlled additions to a cart may be allowed for future orders. Various embodiments may implement dynamic pricing based upon factors that can include but are not limited to expiration dates, consumer feedback, and conveyor load. Some embodiments may dispense product items for packaging according to expected expiration date.
The processor is in communication 1107 with both a database 1108 in a non-transitory computer readable storage medium 1110, and the various other system components 1112. Specifically, those other components serve to sort 1114 incoming items, with the processor assigning each a product item identifier that is stored in the database.
The individual product items are then inspected 1116. The processor assigns an identifier 1118 to each of the inspections, the results of which are stored in the database together with the product identifier 1120 and the inspection result identifier 1122.
As described extensively above, the individual unique product items are then loaded 1124 into tray(s). Again, the processor stores in the database, the corresponding tray identifier 1126 and tray location identifier 1128 associated with each individual product item identifier.
Next, the trays are moved to the carousel for storage 1130. A carousel identifier 1132, and a location 1134 of the tray within the carousel, may be stored by the processor in the database associated with the other IDs, thereby allowing tracking of tray and product item.
The specific data relevant to various product items is then communicated 1136 from the processor to the GUI, where it is displayed 1138 to the user. Based upon this displayed product item data, the user provides an input to select 1140 a particular product item.
The processor receives this selection, and in response communicates an instruction 1142 to the carousel. Based on that instruction, the carousel dispenses 1144 the unique product item into packaging. Finally, the product item together with the packaging are delivered 1146 to the customer.
Clause 1A. A method comprising:
sorting a first unique product item from a plurality of product items;
assigning a first identifier to the first unique product item;
conducting a first inspection of the first unique product item;
storing in a non-transitory computer readable storage medium, a first inspection result associated with the first identifier;
disposing the first unique product item on a tray at a location that is assigned a second identifier;
storing in the non-transitory computer readable storage medium, the second identifier associated with the first identifier;
moving the tray to a carousel; and
storing the tray in the carousel under a controlled environmental condition.
Clause 2A. A method as in clause 1A further comprising:
communicating the first identifier and the first inspection result to a customer;
receiving from the customer an instruction including the first identifier;
in response to the instruction, referencing the second identifier based on the first identifier;
in response to the referencing, causing the carousel to dispense the product item from the first unique location into a packaging; and
distributing the product item in the packaging to a consumer.
Clause 3A. A method as in clause 2A wherein:
conducting the first inspection comprises conducting an optical inspection; and
the first inspection result comprises an image of the unique product.
Clause 4A. A method as in clause 2A further comprising:
conducting a second inspection of the first unique product item;
storing in the non-transitory computer readable storage medium, a second inspection result associated with the first identifier; and
communicating the second inspection result to the customer along with the first identifier and the first inspection result.
Clause 5A. A method as in clause 4A wherein the second inspection is other than optical in nature.
Clause 6A. A method as in clause 1A wherein the controlled environmental condition is selected from temperature, humidity, illumination exposure, and proximity to another unique product item.
Clause 7A. A method comprising:
communicating to an interface, an inspection result for a unique product item stored under a controlled environmental condition within a carousel;
receiving from the interface, a selection of the unique product item;
in response to the selection, referencing a data object stored in a database to correlate a first identifier of the unique product item with a second identifier of a specific location within a tray in which the unique product item is disposed; and based upon the second identifier, communicating an instruction to the carousel to dispense the unique product item into packaging.
Clause 8A. A method as in clause 7A wherein prior to the communicating the method further comprises:
sorting the first unique product item from a plurality of product items;
assigning the first identifier to the first unique product item;
conducting a first inspection of the first unique product item;
storing in a non-transitory computer readable storage medium, the data object associating the first identifier and the inspection result;
disposing the first unique product item on the tray at the specific location assigned to the second identifier;
storing in the non-transitory computer readable storage medium, the data object associating the second identifier with the first identifier;
moving the tray to the carousel; and
storing the tray in the carousel under the controlled environmental condition.
Clause 1B. An apparatus comprising:
a first carousel configured to receive from a first transport network, a tray including a unique location associated with a first identifier and bearing a unique product item associated with a second identifier, the first carousel configured to maintain the unique product item under a first controlled environmental condition;
a transfer mechanism configured to dispense the unique product item into a first packaging at a staging location; and
a second transport network configured to move the unique product item and the first packaging from the staging location for distribution to a customer.
Clause 2B. An apparatus as in Clause 1B wherein the first packaging comprises a delivery packaging.
Clause 3B. An apparatus as in Clause 1B wherein the second transport network is configured to move the unique product item for distribution via a second carousel.
Clause 4B. An apparatus as in Clause 3B wherein the first packaging comprises a transient packaging.
Clause 5B. An apparatus as in Clause 3B wherein the second transport network comprises an aisle conveyor belt.
Clause 6B. An apparatus as in Clause 1B wherein the transfer mechanism comprises a robot.
Clause 7B. An apparatus as in Clause 1B wherein:
the tray defines an opening; and
the transfer mechanism comprises a member moveable into the opening to contact the unique product item.
Clause 8B. An apparatus as in Clause 1B wherein:
the first transport network is in a first plane intersecting the carousel;
the second transport network is a second plane also intersecting the carousel; and the carousel is configured to move the tray between the first plane and the second plane.
Clause 9B. An apparatus as in Clause 1B wherein:
the first transport network is in communication with a first end of the carousel; the second transport network is in communication with a second end of the carousel opposite to the first end; and
the carousel is configured to move the tray between the first end and the second end.
Clause 10B. An apparatus as in Clause 1B further comprising an inspection station configured to interrogate the unique product item prior to being disposed to the unique location.
Clause 11B. An apparatus as in Clause 10B wherein the inspection station includes an optical camera.
Clause 12B. An apparatus as in Clause 1B wherein the first packaging comprises a bag or a box.
Clause 13B. An apparatus as in Clause 1B wherein the first packaging further comprises an insert.
Clause 14B. An apparatus as in Clause 13B wherein the insert comprises a filler.
Clause 15B. An apparatus as in Clause 1B further comprising a loading station configured to dispose the unique product item in the unique location of the tray.
Clause 16B. An apparatus as in Clause 11B wherein:
the tray defines an opening; and the loading station comprises a member moveable into the opening to contact the unique product item.
Clause 17B. An apparatus as in Clause 16B wherein the member comprises a robot.
Clause 18B. An apparatus as in Clause 11B wherein:
the tray defines an opening; and
the first transport network comprises a member moveable into the opening to contact the unique product item.
Clause 19B. An apparatus as in Clause 18B wherein the member comprises a pin.
Clause 20B. An apparatus as in Clause 19B wherein the first transport network comprises a conveyor.
Storage of particular items within a specific carousel, is typically determined at least in part by the conditions of that carousel. Certain items may be stored in groups calling for the same or similar conditions.
For example a set of fruit product items that call for a suite of conditions such as:
Product types calling for different storage conditions, may all be stored together in a different carousel. For example, in contrast with fruit product items, certain dairy product items may call for:
Returning to
It is further noted that the arrangement of carousels, and even banks of same, may be carefully determined in order to enhance the efficiency of flow of materials through the system. For example, carousels storing items typically purchased together, may be grouped to be physically near one another, thereby reducing inefficiencies of travel time/distance.
The specific location of product items in the carousels can be adjusted over time. This may be done in response to the identification of product purchasing behaviors that emerge for particular customers or subsets thereof. Such trends may be determined with the aid of intelligent learning approaches that are trained on past customer behavior.
Other considerations may influence carousel location within the system. For example, as described herein the placement of items within a bag may be carefully determined, with fragile products being dispensed last, on top of more durable items. Thus, carousels storing such fragile products may be located downstream in the process flow, with items ordered in a way that those slated for the bottom of the bag drop first, and those slated for the top of the bag drop last.
In addition to relative fragility, a dispensing order may also depend upon relative product item size. Certain approaches may seek to dispense a large item (e.g., a tall box of cereal) early in the sequence, and then dispense smaller products around it.
Returning to
Conditions within each carousel are carefully maintained, and the environment of each selectively modified in order to extend product life. Environmental conditions that may be relevant for storing products in a desired state, can include but are not limited to:
Detection of such conditions can take place within the carousel itself, checking multiple trays at a same time. Alternatively, such conditions can be detected on an item-by-item basis during product intake and sorting, or prior to dispensing.
Hence, products that are desired to be maintained under similar conditions, may be stored within the same carousel. Incorporated by reference herein for all purposes, is the following document: Agricultural Handbook No. 66, “The Commercial Storage of Fruits, Vegetables, and Florist and Nursery Stocks”, printed by the U.S. Department of Agriculture as revised February 2016. This publication provides a table listing compatible fresh fruits and vegetables during 7-day storage.
Considerations other than environment may dictate the storage of products within or outside of, the same carousel. For example, consumer safety reasons may require the segregation of product items that are the source of food allergies. Hence, the certain carousels may be dedicated to the storing of nuts only.
Details regarding the dispensing items from the tray with a conveyor from below, are now provided. In particular, the product retrieval and dispensing apparatus according to particular embodiments, may offer three services:
Details regarding the architecture of the machinery along the retrieve axis, are now described in connection with
After some amount of travel, the forward drive gear 1238 engages the rack. The gearing is designed so that both drive gears can interact with the rack at the same time and pass off the rack to each other. At full extension only, the forward drive gear is engaged.
Between
Here, the belt stays tensioned as it moves between fixed pulleys. Pulleys 1251 move forward and back. They stay between the outside pulleys. This helps to ensure that tension is maintained in the belt.
Belt movement and sliding action may be driven by two splines 1250 and gears so that motors are fixed under the unit, making it easily removable and cleanable. As the conveyor, 1240 slides left and right to fit into the correct slot in the tray, the spline will stay engaged with the gears that drive the sliding action and the belt movement.
A benefit of this approach is that the drop height is always the same height. Additionally, some items can be positioned in specific locations by controlling one or more parameters such as:
In an example, a product in the form of a bottle of wine could be initially disposed in the tray, oriented parallel to the product conveyors. As the bottle tips over the edge of the product conveyor, the bottom of the bottle will reach the bag and stop at an angle, resting against the product conveyor. The product conveyor can then push the top of the bottle forward until it is sitting upright.
As described herein, one benefit of product handling systems according to embodiments, is that they may allow for the full diversity of products of a conventional grocery store, to be simply organized, stored, and then accessed. One feature that is particularly helpful in this regard is the tray element.
Particular embodiments may feature two different tray types, each type configured in various sizes to receive different products. A first tray type is designed to store products that are round or cylindrical in shape, and hence exhibit a roughly circular cross-section
By contrast,
Given these two basic types, various sizes of row dimensions may allow for the support of a wide variety of potential products, as is described in the following tables.
As previously described, embodiments may employ sensing of characteristics of products that are stored in the carousel, in order to ensure delivery at predetermined levels of freshness. One way that this can be done is by product imaging.
Specifically, optical characteristics of specific products at specific stages of freshness can be taken and recorded, in order to provide one method of quality measurement.
Various optical characteristics of the image may be measured for each picture and stored, and a database created from the measurements. Exemplary optical characteristics which can be sampled by imaging according to embodiments, can include one or more of the following color properties:
Using machine learning techniques in combination with artificial intelligence (AI) approaches, these images taken of produce along its freshness lifecycle, can be automatically segregated into quality groupings, depending upon how their optical properties (e.g., colors) change over time.
In one example, thirty-eight (38) images were taken of the specific bunch of bananas over its freshness lifetime. Saturation color property data for pixels of each of these images was sorted into 16 bins. Then, over ten iterations of K-means clustering, the resulting five groupings of product quality in
In summary: calculating one or more color properties of pixels in various optical images of a perishable product (such as produce), and then using that color property data in order to develop a database, can provide a model of the typical freshness life cycle of that produce. Such approaches would allow future color measurements to accurately predict a current status of newly-received instances of that perishable product during its freshness lifecycle (e.g., from harvested, to pre-ripe, to ripe, to over-ripe, to no longer useable).
It is noted that the freshness correlation just described above, is not limited to being based upon the sensed optical property of saturation. Other pixel optical data (such as from hyperspectral or multi-spectral imaging) can be referenced in the database.
Moreover, freshness prediction need not be based exclusively upon sensed optical characteristics. To enhance accuracy of the product freshness model and database, other factors could also be correlated within the database with the product and considered, including but not limited to one or more of:
In addition to the inspection data mentioned above, data incoming with the product item from the source, could also be included in the database and correlated with expected freshness and other AI procedures. Examples of such incoming data types can include but are not limited to:
One or more of the data types described above, may be further correlated with product item data that is also stored in the database. Examples of such product item data can include:
One or more of the data types described above, may be further correlated with customer data that is also stored in the database. Examples of such customer data can include but are not limited to:
One or more of the data types just described may be further correlated with system data that is also stored in the database. Examples of such system data can include but are not limited to:
Thus according to some embodiments, the database could be a relational-type database having rows corresponding to individual incoming product items that are being handled, and various columns corresponding to different properties of that item (e.g., source, freshness state, carousel conditions, many others as mentioned herein). Such a database structure allows the data relevant to each handled product item to remain associated with that product over the entire lifetime of the handling process (e.g., from intake, through storage, retrieval, and final dispensing to the customer).
As mentioned above, embodiments may employ artificial intelligence (AI) and machine learning techniques—for example to assess product freshness and predict future freshness based upon inspection and other data types. Such AI approaches involve a specialized procedure executed by a processor, which recognizes correlations between a corpus of known starting conditions and resulting historical outcomes. The specialized procedure is trained and refined based upon the historical outcomes, and is then used to predict outcomes for new starting conditions not previously encountered (e.g., a new product being received for imaging).
At 3204, a model for predicting the results from the corresponding inputs is created. That model is mathematical in nature, and can be in the form of weighted equations, neural nets, and others.
The model is configured to receive inputs in numerical form. Accordingly, at 3206 preprocessing is performed upon the various inputs of the training set, in order to convert them into numbers recognizable by the model. Such preprocessing can involve a variety of techniques such as normalization, tokenization, and others.
At 3208, the training data is applied to the model to refine its structure. In certain embodiments this training phase may involve the adjustment of weights within the model, to allow it to better reflect the historical results that are produced from the known inputs. These model outputs are also in numerical form, and typically represent likelihoods of a plurality of outcomes. In this manner, the model is trained by the training data.
At 3210, a new input is applied to the model that has been trained by the training data. In response, the trained model outputs a probability that the new input corresponds to a particular outcome. This application of new data to the trained model is known as the inference phase.
At 3212, the numerical results output by the model are mapped into useful information. For example, the mapping to an outcome may correlate a numerical result into to a particular product freshness state.
At 3214, the accuracy of the new predicted output can be evaluated. This evaluation of accuracy can then be fed back 3216 into the training corpus to help further refine the accurate performance of the model in predicting outcomes from new inputs. This new input is applied to the model as trained by the training data.
The optical data resulting from the optical inspection is stored in database 3306. As described previously, other data may also be stored in the database, for example incoming product source and transportation data 3308.
Based upon this stored data, an assessment of the physical state 3309 of the product may be produced by an inspection artificial intelligence model. That state may reveal blemishes or other features characteristic of product freshness (e.g., water content).
The product is then routed by automation control 3310 to a particular carousel 3312 based upon considerations such as its freshness, expected shelf life, and popularity. A separate AI degradation model 3314 may predict the progression of the stored product through its freshness lifecycle (e.g., as harvested, pre-ripe, ripe, overripe, not usable). This degradation model can ensure that only products of an accurately predicted freshness, are ultimately dispensed to the customer according to his or her preferences.
In particular,
As described herein, artificial intelligence and machine learning approaches may prove useful in predicting a number of different type of outcomes in the overall product handling process. One example is using automated quality prediction procedures and models in order to separate Stock Keeping Units (SKUs) into quality groups that a consumer can select from between based upon their preferences. Actual images of the individual items may be offered to the customer during this product selection process.
Artificial intelligence and machine learning approaches may also be utilized in connection with dynamic pricing of specific items. That is, automated quality prediction procedures and models can be used to dynamically price each individual item based upon properties such as freshness, expected expiration, and demand.
Automated quality prediction models may also be referenced in order to rank the individual product items belonging to a SKU, by their ripeness and expiration dates.
In summary, the following table lists several AI models, together with their expected inputs and outputs.
Successful application of various such AI approaches may help to ensure that a product exhibiting the quality desired by a customer, can be rapidly and efficiently delivered to that customer. For example, the consumer may be offered his or her selection of SKUs in a way that allows the customer to always consume the item at peak ripeness. This can lead to increased customer satisfaction.
Product handling systems according to embodiments, may provide the customer with the ability to purchase a set of individual product items belonging to a particular SKU, with each individual item turning ripe on the day requested by the customer.
Embodiments may also provide the customer with the ability to purchase individual product items belonging to a complementary set of SKUs. That is, items belonging to certain SKUs may commonly be purchased together (e.g., cake ingredients), and the system facilitates handling and delivery of a set of those SKU items together.
Embodiments also allow for dynamic pricing of handled items. By generating and storing in the database an up-to-date and accurate assessment of product quality and expiration date for each item, the items can be dynamically priced for sale. For example, aging produce having an earlier expiration date can be dynamically priced at a lower value, allowing for more rapid sale and product turnover, and reducing waste.
Product handling approaches according to embodiments can exhibit increased efficiency and flexibility in other ways. For example, the large number of data types stored in the database and available to various AI models, can permit automatic filtering of a consumer's product choices or meal kit choices. This filtering can be based upon stored factors such as consumer diets, consumer allergies, and current availability of bulk items in stock at a consumer's home.
As described above, product handling methods and apparatuses according to embodiments, may be particularly useful for delivering perishable items. In some embodiments the shelf life of a product may be measured in days, for example in the case of fresh produce. Other products may have lifespans that are significantly longer than a week, for example pre-packaged products or aged items such as cheese and wine. Examples of product lifespans may be accommodated according to embodiments are 1 day, 2 days, 3 days, 4 days, 5 days, 6 days, 1 week, 2 weeks, 1 month, 6 months, 1 year, and 5 years.
While the above description has focused upon the handling and storage of perishable items, embodiments are not limited to such examples. Alternative embodiments can store non-perishable goods, e.g., those expected to have lifetime of five years or greater.
Clause 1C. A method comprising:
receiving a perishable product item;
sensing an optical image of the perishable product item, the optical image comprising a plurality of pixels;
calculating color properties for at least some of the plurality of pixels;
storing the color properties in a database; and
predicting a freshness state of the perishable product item based upon the color properties, by referencing a freshness model trained according to historical freshness data.
Clause 2C. A method as in Clause 1C further comprising:
identifying a blemish on the perishable product item based upon the color properties, by referencing an inspection model trained according to historical blemish data; and storing the blemish in the database.
Clause 3C. A method as in Clause 2C wherein the freshness model further considers the blemish in predicting the freshness state.
Clause 4C. A method as in Clause 1C further comprising:
loading the perishable product item into a carousel maintained under an environmental condition based upon the freshness state; and storing the environmental condition in the database.
Clause 5C. A method as in Clause 4C further comprising:
controlling the environmental condition based upon the freshness state to prolong a freshness of the perishable product item.
Clause 6C. A method as in Clause 4C further comprising dispensing the perishable product item from the carousel by,
causing a product conveyor to protrude through an opening in the tray to contact the perishable product item, and
advancing the conveyor to cause the perishable product item to fall into a bag having a bottom positioned at a first height proximate to the conveyor.
Clause 7C. A method as in Clause 6C further comprising:
lowering the bottom of the bag away from the conveyor prior to receiving another product item dispensed on top of the perishable product item.
Clause 8C. A method as in Clause 6C wherein the dispensing is in response to a consumer input accepting a price of the perishable product item.
Clause 9C. A method as in Clause 8C further comprising:
dynamically calculating the price according to a price model based upon the freshness state.
Clause 10C. A method as in Clause 8C further comprising:
prior to receiving the consumer input, separating the perishable product item into a freshness grouping based upon the freshness state and a consumer preference stored in the database.
Clause 11C. A method as in Clause 8C further comprising:
prior to receiving the consumer input, displaying the optical image to the consumer.
Clause 12C. An apparatus comprising:
a product handling carousel storing a perishable product item under an environmental condition; a database storing a freshness state of the perishable product, a carousel identifier, and the environmental condition;
a fulfillment engine configured to reference the freshness state and the carousel identifier in response to a customer input, and cause the carousel to dispense the perishable product item into a bag.
Clause 13C. An apparatus as in Clause 12C further comprising a frame having forks extendable to retrieve a tray from the product handling carousel.
Clause 14C. An apparatus as in Clause 12C further comprising a product conveyor configured to,
protrude through an opening in a tray to contact the perishable product item, and advance to cause the perishable product item to fall into a bag having a bottom positioned at a first height proximate to the conveyor.
Clause 15C. An apparatus as in Clause 14C wherein the frame is moveable to lower the tray over the product conveyor.
Clause 16C. An apparatus as in Clause 14C further comprising a moveable platform configured to lower a bottom of the bag upon receiving the perishable product item.
Clause 17C. An apparatus as in Clause 12C wherein:
the perishable product item exhibiting an approximately circular cross section is supported on a tray of a first type; or
the perishable product item exhibiting an approximately rectangular cross section is supported on a tray of a second type.
Clause 18C. An apparatus as in Clause 12C further comprising a camera configured to: sense an image of the perishable product item, the image comprising a plurality of pixels; calculate color properties for at least some of the plurality of pixels; and store in the database, the color properties associated with the perishable product item.
Clause 19C. An apparatus as in Clause 18C wherein the database is configured to store the color properties together with at least one of system data, customer data, and incoming data associated with the perishable product item.
Clause 20C. An apparatus as in Clause 18C wherein the fulfillment engine is configured to:
process the color properties to generate a freshness state for the perishable product item; and store in the database, the freshness state associated with the perishable product item.
Further details regarding the inspection of products to determine defects, are now described. Embodiments may use machine learning in combination with inspection to detect defects in produce based on their area and defect type.
There are at least two ways to inspect for defects. A first approach involves developing a model for each variety of produce. This amounts to around 700 different models.
An alternative approach develops a universal model for each defect. Examples of such universal models could comprise a bruise model and a cut skin model. This defect-based approach results in only about 40 models being needed, reducing development time and enhancing robustness.
Specifically, utilizing visual/near-infrared imaging in combination with machine learning, allows object detection for product item quality. Camera systems are used to take images of produce in a controlled lighting environment.
The images cover wavelengths in the visible RGB range as well as in the near infrared (NIR) wavelength range. This can be accomplished with multiple cameras (e.g. one RGB camera and one NIR camera), or a single multi-channel camera.
Specific filters can be added to the cameras to focus the inspection on a particular band of wavelengths, especially for the NIR camera. Each of the image modalities may reveal different types of defects on produce. For example,
The acquired images are labeled, and then used to train machine learning models to detect defect(s) that may be present. The models can be trained in at least two different ways.
According to one approach, a model is trained to detect all possible kinds of defects on a single type of item (e.g., apples), by using a training set comprising known defects on a series of images taken of that one item type.
According to another approach, a model is trained to detect a specific defect (e.g., a bruise) across many types of produce (e.g. as found on apples, pears, tomatoes, peaches, etc.) using a training set comprising images including the specific defect on a variety of produce types.
The object-detection type model is one possible example. Here, the RetinaNet deep learning algorithm was adapted for this purpose. The size of the resulting bounding box around the detected defect, indicates the size of the defect.
The resulting information on differentiated defects in the images, and the size of each defect, is combined with a produce-specific algorithm to determine a quality grading for the item.
Moreover, this information can also be combined with other information (including harvest date, prior storage conditions, future storage conditions, etc.) in a procedure which predicts how the produce will ripen or degrade over time.
While the above has described inspection of items utilizing a camera, this is not required. As an alternative or supplement to a camera, a spectrometer may also be used.
Specifically, spectrometer measurements (e.g., reflection, absorbance, transmission, etc.) of an item in the VIS/NIR wavelength range (˜350-2500 nm), permits analysis of the external/internal properties of the item, which can include but are not limited to:
In one implementation, portable spectrometers have been developed for agricultural applications in inspecting produce before picking, or for later handheld inspection of individual pieces of produce.
An embodiment of a process according to a particular example, may take spectrometer VIS/NIR measurements of incoming produce items. That spectrometer data may be correlated with quality, defects, and current ripeness level of produce.
The spectrometer ripeness prediction is combined with data from other camera inspection systems and historical information on produce (e.g., date picked, variety, growing location, previous storage conditions, storage conditions while in grocery store). This data may be used to assess optimal time to deliver produce to customer and predict timeline for how produce will ripen and/or decay while in the grocery store. This approach reduces waste, improves operations/logistics, and improves customer satisfaction.
Products may be inspected prior to entering storage in the carousel(s). Product lifetime may be estimated based upon the inspection results.
In some embodiments, however, products can be automatically inspected after storage, either exclusively or in combination with inspection prior to storage. Such post-storage inspection can employ one or more of the following techniques.
Cameras may be used to inspect a full tray or products directly after being pulled from the carousel. Cameras may also inspect individual products after they are removed from a tray and before they are dispensed into a bag.
Moreover, sensors other than visual sensors (e.g., cameras) may be employed to inspect the products. For example, berries degrade quickly by getting moldy—a negative experience for the customer.
It is possible to sense mold by “sniffing” the gas around the product. In once embodiment, a nozzle could just be positioned to suck air from inside a clamshell enclosing a berry. The sampled air could then be analyzed. In another approach each case could go through a closed box where the air is analyzed for mold spores.
Details regarding apparatuses that may be utilized for product inspection, are now provided. A variety of methods may flip and/or spin product items during camera inspections to allow interrogation from multiple sides.
Product item shapes may be generally classified into spheres, cylinders, and cuboids. After inspection, the items are sorted into trays based on various criteria. A number of approaches may be used to place items onto trays as part of intake, inspection, and/or sorting of individual unique product items incoming to the system.
One approach utilizes a pivoting belt conveyor.
Items can translate through the conveyor.
Spheres and cylinders will roll when the conveyor is rotated up at an angle. This is shown in
Also, the unit can fold over itself in order to flip a box item. This is shown in
It is noted that two O-ring belt conveyors can convey almost all items. And, cameras from 6 directions take images of the items.
Certain approaches may utilize free fall imaging of items. That is, as items fall off of a conveyor, they are imaged in the air.
Multiple cameras capture all sides of the items. The item is caught with a cushioned conveyor or other soft landing pad, before moving the produce further along the conveyor. This method allows for quick image capture of all sides of an item, without having to deal with other hardware being in the camera field of view. This permits a single machine learning model to be used with the images taken from all cameras.
Another possible approach utilizes a star wheel conveyor, as shown in
Cuboid items can be flipped when going through the system. This is shown in
Such a star wheel conveyor allows many items to be completely imaged with a limited number of cameras. The cameras may have the same mechanical components in the FOV, which simplifies the machine learning.
Still another approach may utilize a popup roller conveyor.
According to this approach, two o-ring belts convey items in the store. The rollers pop up next to the conveyor when the item is in place. Cylinders and spheres are rolled along the conveyor, thereby presenting all sides of the item to a few cameras. Different size rollers may be used based on the item size.
This system allows for a majority of items to be fully imaged with a limited number of limited cameras. The cameras may have the same mechanical components in the FOV, which simplifies the machine learning strategies.
Another mechanism that may be useful for handling items within the system, is a roller and spinner conveyor.
Here, the conveyor comprises trapezoidal rollers, and items are carried along by the conveyor. The roller/spinner conveyor stops and the rollers spin, thereby rolling cylinders and spheres. Again, this action allows sides of the item to be presented to only a few cameras. The cameras will have the same mechanical components in the FOV, which simplifies the machine learning.
A pop through conveyor offers a possible approach for item handling.
Specifically, this concept provides an auto tray loading concept that is similar to the dispense station. Two belt conveyors distribute the items along the length of each row. The tray moves so the items are located in the correct row. This approach allows items of different grades to be separated in their distinct row of the tray, without requiring a buffer system or unique classification separation system.
Still another item handling approach may utilize an XYZ gantry mechanism.
Here, the trays are disposed on a moveable XYZ gantry, and move the tray to a distinct position so the item may be placed in the exact position desired. This concept allows items of different grades to be separated in their distinct row of the tray without needing a buffer system or unique classification separation system.
Still other item handling approaches may utilize a vertical stack buffer system.
Specifically, this approach offers a post-inspection, pre-tray loading grade and size filtering system, allowing items of the same grade and/or size to be are grouped together. This mechanism allows for a simpler tray loading mechanism as all items are filtered into the correct category, prior to being taken to the tray loader.
In the specific approach illustrated in these figures, there is a vertical buffer that builds each row. So, the infeed becomes more complex, moving up and down and left to right to feed a fixed tray position.
As shown in the top view of
This multiple conveyor setup shown in
As has been previously discussed, embodiments may call for items to be stored under controlled conditions, in a carousel structure. Various aspects of possible designs for the carousel element, are now described.
Initially, it is noted that trays can be shifted from one carousel to another. This may be done automatically in order to control ripeness.
For example, an avocado should be initially stored in a cold carousel in order to preserve its life. Then, a few days prior to being dispensed, the avocado should be shifted to a different carousel that is maintained under warmer temperature conditions, and paired next to an ethylene producing item. This will initiate ripening.
Once the ripening process has commenced, the avocado could be shifted back to a colder carousel to keep it from over-ripening. The avocado can then be sold to a consumer at the desired ripeness. Such a carousel-shifting approach allows more products to be stored in inventory, without spoilage and loss of desirable characteristics (e.g., flavor and nutrients in the case of a produce item).
Specifically, by avoiding the space occupied by a bottom support beam, a carrier design can fit 7, 6, 5, or 4 shelves. Carriers of various capacities may be available in each carousel, as needed.
Extending out of the page in
Various configurations may further enhance densities of vertically stacked trays within a carousel. For example, while the specific embodiment of
Different types of carriers may be present in each carousel. Each carrier can be constructed to handle different level counts in order to maximize density.
For example,
Once items are stored within the carousel upon a tray supported by a carrier, individual items may be dispensed therefrom. Various details regarding particular embodiments of dispensing mechanism are now described.
An item dispensing station according to embodiments may feature one or more elements to enhance performance. For example, it is important to avoid items falling off the dispense station. Accordingly, certain embodiments may feature a dispense conveyor backstop on the pop through conveyor, in order to keep items from falling backwards.
Dispense stations according to embodiments may also feature sensors for detecting dispensed items.
This presence sensor detects when an item arrives at the end of the conveyor. The length of the item is already known, so once the item is detected the conveyor can be driven the known amount to dispense the item off of the conveyor end. The sensor also detects that the item has left the conveyor, thereby allowing reliable confirmation of dispensing a single item.
Weight can also be detected as a sensed quantity during dispensing.
Load cells may permit weight measurement to detect items falling off the dispense station (e.g., off of the back). Load cells can also be used to detect the change in weight as an item leaves the conveyor.
Load cells may be positioned in a number of different possible locations, including but not limited to:
As an item is dispensed off the end of the conveyor, the load cell reading marks a step change, indicating the reduction in mass on the conveyor. This again permits reliable dispensing of just one item at a time from the tray.
In order to accurately accomplish item dispensing in a reliable manner, it may be useful to precisely control tray position. Maintaining positional control of the tray within a tolerance, allows the dispense conveyors to repeatedly fit through the slots in the tray, while reducing the layers of closed loop motion control.
The exercise of positional control over the tray may commence when the tray arrives at the carousel (
The carrier has pins that index with the tray. The tray arrives at the upper tray lifter and is lifted by the end effector. The end effector pins index with the tray, locating it precisely. The upper tray lifter inserts the tray into the carrier, where it also indexes on pins. Using such passive (e.g., pin) features to control position of the tray during storage, allows for closer tolerances on the dispensing end effector while still allowing retrieval without closing the loop between the end effector position and the tray position.
Exercising positional control of the tray on the end effector, also permits precise disposition of the dispensing conveyors so they fit through the slots in the tray without closing the loop between tray position and dispensing conveyor position.
Details regarding designs for the end effector are now described. It was discovered that reaching relatively deep into a narrow space called for better control over the angle of the end effector, than:
Accordingly,
The tray slides on the carrier shelf and also slides on a shelf on the tray lifter. This eliminates the issue of leveling the carriers to the tray lifter and maintaining level tolerance for every carrier in every loaded state.
This allows for increasing the pitch of the carrier shelves receiving the tray. Such increased shelf pitch promotes an increase in tray packing density, by allowing choice of the best shelf to put the tray on to allow it to be closest to the next tray above or below.
It is noted that the slides holding the end effector need to fit between the shelves. Not needing to preserve open space for slides, allows the shelf pitch to be tighter. The tray can be positioned closer to the one above or below it depending on the height of items in the tray.
Details regarding various designs for the customer bag and the transporting same for shipping, are now discussed.
A traveler containing a bag to receive dispensed items, may feature a platform that can be raised and lowered. There are several possible ways to raise and lower the platform.
Alternative designs for travelers are of course possible. Certain embodiments could feature the platform attached to a scissor lift. Some embodiments could feature the platform lifted in corners by belts or pulleys. Also, a traveler design could comprise an assembly of springs that compress when the bag is full, and extend when the bag is empty.
Some embodiments may add load cell(s) to the platform, allowing measurement of changes in weight at the traveler.
In this embodiment, the load platform comprises two plates. The bottom plate is anchored to lead nuts that travel up and down the screws.
Four cantilever beam load cells support the top plate. These load cells measure the weight of items in the bag, helping to determine if items were successfully dispensed.
While the specific configuration of load cell may vary, the basic concept features the measurement of weight changes to evaluate dispensing. Given known item weights, with proper calibration the difference between weight changes (e.g., 1 apple vs. 2 apples) can reveal when possible errors occur.
Bag(s) for housing items dispensed from the carousel, are supported by the traveler. One or more methods may be utilized to hold the bag in place at the top of the frame.
According to particular embodiments, the bags may have fabric grommets installed in their upper corners. Bags are loaded into the traveler by hooking the grommets over bent pegs located in the upper corners of the frame.
A hinged lid (with an integrated latch) opens and closes to lock the bag in place. The inner rim of the lid may be lined with bulb edge guards to protect dispensed items from being damaged.
However, alternative designs for the traveler are possible, with the bag being fixed according to other approaches. For example,
Still further alternative traveler designs are possible. Some embodiments may feature a quick turn arm and locking post. Other embodiments may include velcro tabs sewn on to the upper bag corners. Magnets could also be used as clamps.
A traveler according to an embodiment may feature a frame topped with a hinged lid.
According to certain embodiments, the traveler may be powered by a quick-swappable battery pack design that can be easily replaced for charging. The battery pack may comprise a housing and two sealed lead acid batteries wired in series inside the housing.
A processor can measure the battery's state of charge by reading its voltage. When the battery pack is low on charge, it can be removed from the traveler base without tools. It may be unplugged without any loose wires; the power plug and jack are embedded in the traveler base and battery pack housing, respectively.
The battery pack may be charged in a parallel rack system located close to the bag unloading station. A fully charged battery can be loaded into the traveler so it can continue its work with only a short interruption.
As noted previously, particular embodiments may afford movement of the traveler based on conveyor action.
In particular embodiments, the conveyor system for the traveler may comprise a collection of modular conveyor sections and 90° transfer stations. This conveyor system moves the travelers around the fulfillment center to accomplish their tasks (e.g., receiving dispensed items, possibly loading trays into carousels).
The conveyors may be arranged in grids and loops. Each section is capable of running in both forward and reverse directions.
According to embodiments, each conveyor section is an independently controlled set of rollers linked together with drive O-rings. One roller pin in each conveyor section has an integrated motor with servo control that allows precise positioning of a traveler on that section, in the dimension of conveyance (e.g., across the dispensing station located at the front of the carousel).
Position control along this dimension, combined with position control of the dispense arms of the tray lifter in a perpendicular dimension, allows items to be dispensed to precise locations in the bag.
When a traveler is supported by rollers of adjacent conveyor sections, control software may instruct coordinated movement of both conveyors until the traveler is fully in control of the downstream section. At that point, the section can move the traveler around without affecting the positions of other travelers.
This independent control can facilitate precise item dispensing at every dispense station. A photoelectric sensor may be integrated in each conveyor section to trigger a reference position of the traveler as it moves onto the section.
Different variations on the design and operation of the traveler component, could serve to enhance system operation. For example, in certain embodiments the traveler may include its own drive system (e.g., wheels, tracks, rails, others) and/or sensors (e.g., cameras, line-following optics) that permit autonomous or semi-autonomous movement throughout the fulfillment center.
Certain embodiments may feature the addition of duplicate bag locking mechanisms. These could allow each traveler to carry multiple bags at once.
It is noted that a single traveler could fulfil a single order, multiple orders, or parts of separate orders, during various trips around the fulfillment center. This can lead to enhanced efficiency.
Certain embodiments may feature the addition of a tray-carrying attachment to the traveler. This would allow the traveler to bring trays to each carousel for loading. Such integration of tray loading and item dispensing with a common moveable structure (the traveler), would lead to substantially simplified operation.
Some embodiments could feature the addition of a high speed actuator capable of moving the load platform downward at high speed. Such near-matching the downward projectile velocity of a dropped item could desirably reduce impact forces on the item and upon the load platform.
Some specific details regarding design and operation of trays according to particular embodiments, are now provided. Trays may be fabricated utilizing an injection molding approach that ensures high strength.
According to certain embodiments, the entire tray may be created as a single, unitary injection-molded piece.
Here, the largest outer diameter (OD) of a stored item, matches pitch. The smallest OD is determined by a center of a round cross-section not tipping past hump on the center beam of the tray.
The strength of the center beam is dependent upon height. This reduces packing density in the carousel. The base may be widened with ribs in order to resist side loads
Reliance upon a single, unitary tray design, can increase manufacturing cost. Moreover, the resulting fixed size of tray features (e.g., number of rows, width of rows, number and shape of separators) can limit flexibility and degrade efficiency.
Accordingly, alternative embodiments may feature a design that allows different tray types to be assembled from a plurality of injection molded parts.
The enlarged tray views of
In some embodiments, trays having various different features may be assembled on demand. Equipment may be designed to accomplish this tray assembly process.
Specifically, item handling systems according to embodiments may utilize a fixed number (e.g. 10) of tray types. Each tray type may hold a certain size of item.
Individual trays may comprise a plurality of parts in the form of long, narrow, injection molded pieces that clip together to form a tray.
Following item inspection, the tray loading station needs to receive an appropriate tray type to load the inspected item into.
Rather than store stacks of trays of each type, a machine may access the individual pieces stored sorted according to type. Then, on demand, the machine may then assemble a tray of a certain size.
It is noted that the unassembled pieces have a higher storage density than the assembled tray. Thus, on-demand tray assembly can save valuable space within the fulfillment facility.
According to embodiments, the machine could also assemble on-the-fly, trays in configurations other than the limited number of standard types. This could be done by building rows with different spacings, as called for by the tray loading station.
Empty trays removed from the carousel could be returned to the machine. There, the empty trays are automatically broken back down into individual pieces for efficient storage.
Such approaches featuring on-demand tray assembly, may offer one or more benefits. As mentioned above, these approaches may save space on the line.
Also, on-demand tray assembly may allow for a larger variety of trays types, and may decrease the distance a tray has to travel to get to the tray loading station if the machine can be located closer (potential time reduction).
On-demand tray assembly may reduce the total number of tray piece parts required. This is because trays can be reconfigured as necessary, rather than wasting tray inventory space on tray types that aren't being utilized.
Returning now to discussion of general system operation, under some circumstances it may be desired to dispense an item that is not located at the end of a tray row (e.g., is stored in the middle of the tray). Accordingly, certain embodiments may utilize a gantry robot for dispensing purposes, as is shown in the simplified view of
For example, when each item in a SKU has a different value, it may be desired to dispense from the center of a tray. Thus where the product is a steak, there may be 40 images of different cuts of steak that are available to show the customer for selection. By allowing a gantry to access center portions of the tray, all of the steak cuts can be available for dispensing.
Another possible circumstance that may be favorable to the use of a gantry robot, is where the items are not amenable to conveyor dispensing. This could be due, for example, to the items' size, shape, and weight. In such cases, a gantry robot may be a viable alternative.
Also, when items are small and thin (e.g., a gravy packet or beef jerky pouch), it would be advantageous to stack them vertically in one tray and then pick the top item from each tray. This increases density.
Finally, dispensing with a gantry robot could be useful when an item is rarely purchased. Under such circumstances, it is inefficient to carry an entire row of that SKU. The use of a gantry robot to pick from the center of a tray, allows placement of multiple SKUs in one row, and picking them out as needed.
It is further noted that a gantry robot could be employed in combination with a conveyor. Specifically, a gantry fixed at the end of conveyor belt(s) could catch the item and lower it into the bag. This could reduce damage to items.
Item handling systems according to embodiments, may not be limited to a single physical location. Items may be dispensed and delivered from multiple smaller sites that are located relatively close to one another (e.g., in different neighborhoods within a single large metropolitan area.
Under such conditions, an inventory management solution may feature one central store (the hub) and many satellite stores (the spokes). This approach could be implemented with tray packaging performed at the hub, and trays shipped to each spoke. Carrying racks could be designed to allow distribution vehicles to rapidly and easily perform loading, transport, and unloading.
In summary,
A system according to various embodiments may include one or more features to enhance automated dispensing and delivery of handled items. For example, in order to replace pallet jacks, robotic pallet jacks can be utilized remove items from trucks that are delivering bulk goods for intake, inspection, sorting, and loading into carousels. Also, robotics systems that can raise and lower pallets and insert into shelving, may be relied upon to lift pallets onto shelves. Further, once a pallet is ready for inspection, automation equipment may move boxes from pallets, and move the box contents onto a conveyor.
Moreover, particular embodiments may automate order packing as follows. When a full traveler arrives, equipment would lift a bag out of the traveler and move it to a shipping area. Another piece of equipment could place a new bag into the traveler.
A robotic cart would be deployed to transport bags from the factory to the parking lot. The robotic cart would place bags into the mode of transportation for delivery.
A variety of delivery options are of course possible. Vehicles could be manually driven by humans, or self-driven. Alternatives can include delivery by bikes, carts, drones, or trucks plus drones.
It is noted that various embodiments may enhance a customer's experience by allowing flexibility and choice in selecting items. A customer may search for items on a website utilizing a search bar.
Under this approach, the user enters a product name, and in response is presented with a list of options. The customer selects an option, which is added to the customer's cart. While the search bar approach is fast for individual items, it can be laborious for a cart comprising multiple items. Also, the search bar approach requires a customer to know desired items in advance.
In contrast, with the search bar approach, an aisle-based approach involves a customer seeking a particular product item, to traverse a complex hierarchy. Thus, a customer seeking to purchase a red onion, may need to:
Such an aisle-based approach takes longer than searching if a customer knows ahead of time what is desired. But, the aisle approach affords the customer the ability to browse different options, and get inspiration for other products. However, aisle-based selection can be tedious due to the many possible available options to scroll through in each aisle.
Accordingly, particular embodiments may permit customer searching for items based upon categories. This approach can specifically lead customers to the particular items that they want to buy.
Under the category-based approach, a customer browses item categories, permitting visualization in a more useful manner. The categories would begin as general, and then narrow to become more specific.
For example,
One possibility is to display for customer selection, standard pizza options (e.g., pizza crust, sauce, cheese). Meat Lovers will add sausage and salami (for example). Such category-based searching can depict ingredients going into making a pizza. So, this approach shows different types of pizza dough (frozen, refrigerated, different brands), the sauces (bottled, cans, brands), different vegetables, etc.
Then, the user could (de)select one or more suggested items, removing or changing the brand. Finally, the user could add to the cart all of the items selected by category, and move on to a different category.
Over time, an engine of the system could learn preferences of specific users/user groups, and suggest appropriate option the next time the pizza category is selected.
Embodiments may suggest ideas to a customer based upon machine learning. Embodiments could suggest specific items for purchase available in stock in large volumes.
Through the user of category-based searching, the consumer's experience is more rapid and intuitive. For example, in shopping for grocery items, customers traditionally think in terms of item groups of meals and categories, e.g.:
“We need three dinners tonight, I need breakfast and lunch food for the week and I wanted to make that chocolate cake on Wednesday for Sue to take to school”.
Here, the user can go to each category and select out their dinners in a few action steps (e.g., clicks). Similarly, a customer can select a recipe for the cake and obtain all the ingredients at once, or they can click each ingredient from one basic list if they have a recipe.
In this manner, customers are free to select items in a more intuitive manner, mirroring how they think and in terms of groups of items. Thus instead of having to choose and select 30 separate items, they are selecting instead only 7:
Category-based selection can enhance the speed of purchase, moving customers through the site faster. Currently it can take around 60 min to fill a virtual shopping cart, a time comparable to (or even longer than) physically going to a grocery store. Improving the check out time is can aid in securing a steady on-line customer base.
Category-based item shopping can also enhance the flexibility of the seller. Grocery stores may seek to leverage relations between items by placing them in physical proximity (e.g., salsa next to chips, marinades by the meat counter). However, this is done in a crude, opportunistic manner, constrained by available physical space (it is unfeasible to position pizza sauce, veggies, pepperoni, and pizza dough, all in a same aisle).
By contrast, a website offers a virtually unlimited variety of ways to group related items to facilitate common purchase. Category-based selection leverages this inherent advantage of web-based shopping, to enhance the user's experience.
As described in detail above, embodiments of systems and methods for handling items may be particularly suited for implementation in conjunction with a host computer including a processor and a computer-readable storage medium. Such a processor and computer-readable storage medium may be embedded in the apparatus, and/or may be controlled or monitored through external input/output devices.
Embodiments according to the present invention can be implemented in a single application program such as a browser, or can be implemented as multiple programs in a distributed computing environment, such as a workstation, personal computer, mobile device or a remote terminal in a client server relationship.
According to one example, computer system 7110 includes a Pentium™ class based computer, running Windows™ XP™ or Windows 7™ operating system by Microsoft Corporation. However, the apparatus is easily adapted to other operating systems and architectures by those of ordinary skill in the art without departing from the scope of the present invention.
As noted, mouse 7170 can have one or more buttons such as buttons 7180. Cabinet 7140 houses familiar computer components such as disk drives, a processor, storage device, etc. Storage devices include, but are not limited to, disk drives, magnetic tape, solid-state memory, bubble memory, etc. Cabinet 7140 can include additional hardware such as input/output (I/O) interface cards for connecting computer system 7110 to external devices external storage, other computers or additional peripherals, further described below.
In certain embodiments, the subsystems are interconnected via a system bus 7175. Additional subsystems such as a printer 7174, keyboard 7178, fixed disk 7179, monitor 7176, which is coupled to display adapter 7182, and others are shown. Peripherals and input/output (I/O) devices, which couple to I/O controller 7171, can be connected to the computer system by any number of approaches known in the art, such as serial port 7177. For example, serial port 7177 can be used to connect the computer system to a modem 7181, which in turn connects to a wide area network such as the Internet, a mouse input device, or a scanner. The interconnection via system bus allows central processor 7173 to communicate with each subsystem and to control the execution of instructions from system memory 7172 or the fixed disk 7179, as well as the exchange of information between subsystems. Other arrangements of subsystems and interconnections are readily achievable by those of ordinary skill in the art. System memory, and the fixed disk are examples of tangible media for storage of computer programs, other types of tangible media include floppy disks, removable hard disks, optical storage media such as CD-ROMS and bar codes, and semiconductor memories such as flash memory, read-only-memories (ROM), and battery backed memory.
As previously mentioned, item handling systems according to embodiments are not limited to conveyor-type systems. Alternative embodiments could include a drive system for a traveler that is responsible for receiving items dispensed from a carousel, and even also responsible for moving trays having items disposed thereon, for loading into a carousel.
Moreover, certain embodiments of travelers may feature multiple bags added to a single drive unit, e.g., track (
It is further noted that a traveler may be equipped to include a tray carrying capability. The addition of such a tray-carrying attachment may allow the traveler to bring trays to each carousel.
It is noted that a traveler need not be equipped to carry a bag for receiving dispensed items. According to some embodiments, a traveler could be designed to carry tray(s) only.
Under certain implementations, there could be two robot types on the floor of the fulfillment center. One robot type carries customer bags between carousels, filling them with dispensed items as they go. The other robot type carries multiple (e.g., ˜5-15 full trays), loading them into carousels as they go. The robots could be environmentally controlled to ensure the condition chain is maintained.
Clause 1D. An apparatus comprising:
a first carousel storing a first product under a first set of environmental conditions, the first product corresponding to a first Stock Keeping Unit (SKU);
a second carousel storing a second product under a second set of environmental conditions different from the first set of environmental conditions, the second product corresponding to a second SKU different from the first SKU; and
packaging disposed on a traveler that is moveable to,
a first station proximate to the first carousel to receive the first product dispensed in a first direction into the packaging, and then
a second station proximate to the second carousel to receive the second product dispensed in the first direction into the packaging, and then
a shipping station proximate to a vehicle for receiving the packaging.
Clause 2D. An apparatus as in Clause 1D further comprising a first tray configured to support the first product and a third product in the first carousel, the third product corresponding to a third SKU different from the first SKU and the second SKU.
Clause 3D. An apparatus as in Clause 2D wherein:
the first tray is configured to support the first product in a first row; and
the apparatus further comprises a second tray moveable within the first carousel in a second direction proximate to the first dispensing station.
Clause 4D. An apparatus as in Clause 3D wherein the first direction is substantially orthogonal to the second direction.
Clause 5D. An apparatus as in Clause 4D wherein the first direction is horizontal.
Clause 6D. An apparatus as in Clause 5D wherein the second direction is vertical.
Clause 7D. An apparatus as in Clause 2D wherein the first item is dispensed to the packaging from the first tray located within the first carousel.
Clause 8D. An apparatus as in Clause 2D wherein the first item is dispensed to the packaging from the first tray moved outside the first carousel.
Clause 9D. An apparatus as in Clause 2D wherein the first tray defines an opening receiving a moveable member to contact the first product.
Clause 10D. An apparatus as in Clause 9D wherein the moveable member comprises a belt configured to move in the first direction.
Clause 11D. An apparatus as in Clause 10D wherein:
the first direction is horizontal; and
the first tray is configured to move vertically for the belt to project within the opening and contact the first product.
Clause 12D. An apparatus as in Clause 10D wherein:
the first direction is horizontal; and
the belt is configured to move vertically to project within the opening and contact the first product.
Clause 13D. An apparatus as in Clause 2D wherein:
the first direction is horizontal; and
the first tray is configured to drop the first product in a vertical direction off a tray end into the packaging.
Clause 14D. An apparatus as in Clause 13D further comprising a flexible member configured to cushion the first product falling into the packaging.
Clause 15D. An apparatus as in Clause 14D wherein the cushion is disposed within the packaging.
Clause 16D. An apparatus as in Clause 2D wherein the tray comprises a plurality of injection molded parts assembled based upon dimensions of the first product.
Clause 17D. An apparatus as in Clause 1D wherein the traveler includes a weight sensor.
Clause 18D. An apparatus as in Clause 1D wherein:
the first set of environmental conditions comprises a first condition selected from,
a first temperature,
a first humidity,
a first light level,
a first gas ambient,
a first mold spore level, or
a first ethylene level; and
the second set of environmental conditions comprises a second condition different from the first condition and selected from,
a second temperature,
a second humidity,
a second light level,
a second gas ambient,
a second mold spore level, or
a second ethylene level.
Clause 19D. An apparatus as in Clause 18D wherein the second carousel is configured to receive from the carousel, the first tray supporting the first product.
Clause 20D. An apparatus as in Clause 18D wherein:
the first set of environmental conditions is configured to prolong a shelf life of the first product; and
the second set of environmental conditions is configured to accelerate a ripening of the first product.
Clause 1E. An apparatus comprising:
a first carousel storing under a first set of environmental conditions, a first product supported on a first tray;
a second carousel storing under a second set of environmental conditions different from the first set of environmental conditions, a second product supported on a second tray; and
a non-transitory computer-readable storage medium in communication with the first carousel and with the second carousel to record data comprising,
a first identifier of the first product,
a first location of the first product,
a second identifier of the second product, and
a second location of the second product.
Clause 2E. An apparatus as in Clause 1E wherein the first location comprises:
a third identifier of the first carousel;
a fourth identifier of the first tray; and
a fifth identifier of a position of the first product within the first tray.
Clause 3E. An apparatus as in Clause 1E wherein the data further comprises status information of the first product.
Clause 4E. An apparatus as in Clause 3E wherein the status information comprises:
a weight of the first product;
a visual inspection result of the first product;
a non-visual inspection result of the first product;
an image of the first product;
a harvest date of the first product;
a shipping date of the first product;
a receipt date of the first product;
an expiration date of the first product;
a quality of the first product;
a name of the first product;
a brand of the first product;
a supplier of the first product;
a history of environmental storage conditions of the first product;
a moisture content of the first product;
a variety of the first product;
a Stock Keeping Unit (SKU) of the first product;
a regulatory classification of the first product;
a recipe including the first product;
dynamic pricing data; or
feedback data.
Clause 5E. An apparatus as in Clause 3E further comprising an engine configured to: receive a first input specifying a particular SKU;
match the particular SKU with the first product; and
provide at least a portion of the status information regarding the first product to a customer.
Clause 6E. An apparatus as in Clause 5E wherein the portion of the status information comprises an image of the first product.
Clause 7E. An apparatus as in Clause 5E wherein the portion of the status information comprises a quality of the first product.
Clause 8E. An apparatus as in Clause 5E wherein the engine is further configured to: receive a second input selecting the first product; and instruct the first carousel to dispense the first product into packaging.
Clause 9E. An apparatus as in Clause 8E wherein the engine is further configured to: receive a third input selecting the second product; and instruct the second carousel to dispense the second product into the packaging.
Clause 10E. An apparatus as in Clause 9E wherein the engine is configured to instruct the second carousel to dispense the second product into the packaging before the first product, based upon a path between the first carousel, the second carousel, and a shipping station.
Clause 11E. An apparatus as in Clause 9E wherein the engine is configured to instruct the second carousel to dispense the second product into the packaging before the first product, based upon a consideration comprising a weight, a size, or a fragility of the first product relative to the second product.
Clause 12E. An apparatus as in Clause 8E wherein the packaging comprises a bag.
Clause 13E. An apparatus as in Clause 8E wherein the packaging is disposed on a traveler moveable between the first carousel and the second carousel.
Clause 14E. An apparatus as in Clause 13E wherein the traveler is configured to move on a conveyor.
Clause 15E. An apparatus as in Clause 13E wherein the traveler is configured to move on a rail.
Clause 16E. An apparatus as in Clause 13E wherein the traveler is configured to carry the first tray.
Clause 17E. An apparatus as in Clause 13E wherein the traveler comprises wheels.
Clause 18E. An apparatus as in Clause 17E wherein the traveler is capable of semi-autonomous movement.
Clause 19E. An apparatus as in Clause 8E wherein the engine is further configured to instruct inspection of the first product prior to dispensing into the packaging.
Clause 20E. An apparatus as in Clause 1E wherein the first tray comprises a plurality of injection molded parts assembled based upon dimensions of the first product.
Clause 1F. A method comprising:
receiving a first product;
performing a first inspection of the first product;
an engine determining from the first inspection, a quality of the first product; the engine recording in a non-transitory computer readable storage medium, the quality related to an identifier of the first product;
based upon the quality, sorting the first product into a first tray stored in a first carousel under a first set of environmental conditions;
the engine recording in the non-transitory computer-readable storage medium, a location of the first item related to the identifier and the quality; and
in response to a first input, the engine offering the first item for a customer selection including display of the quality.
Clause 2F. A method as in Clause 1F wherein the quality comprises an image of the first product.
Clause 3F. A method as in Clause 1F wherein the quality is determined from a physical appearance of the first product.
Clause 4F. A method as in Clause 1F wherein the quality is determined from a taste of the first product.
Clause 5F. A method as in Clause 1F wherein the quality is determined from a ripeness of the first product.
Clause 6F. A method as in Clause 1F wherein the quality is determined from a nutritional value of the first product.
Clause 7F. A method as in Clause 1F wherein the quality is determined from a physical makeup of the first product.
Clause 8F. A method as in Clause 1F wherein the quality is determined from a lifetime of the first product.
Clause 9F. A method as in Clause 8F wherein the engine is further configured to predict the lifetime utilizing a model.
Clause 10F. A method as in Clause 9F wherein the model considers one of more conditions of the first set of environmental conditions.
Clause 11F. A method as in Clause 1F wherein the first inspection comprises an optical inspection.
Clause 12F. A method as in Clause 1F wherein the first inspection comprises spectroscopy.
Clause 13F. A method as in Clause 1F further comprising:
the engine determining a price for the first product based upon the quality; and
the engine displaying the price with the quality.
Clause 14F. A method as in Clause 1F wherein the first input comprises a Stock Keeping Unit (SKU) of the first product.
Clause 15F. A method as in Clause 1F further comprising:
based on the quality, the engine instructing transfer of the first product to a second carousel under a second set of environmental conditions different from the first set of environmental conditions.
Clause 16F. A method as in Clause 15F wherein:
the first set of environmental conditions is configured to prolong a shelf life of the first product; and
the second set of environmental conditions is configured to accelerate a ripening of the first product.
Clause 17F. A method as in Clause 1F further comprising:
in response to a second input comprising the customer selection, the engine instructing dispensing of the first product into packaging.
Clause 18F. A method as in Clause 17F further comprising:
the engine instructing movement of the traveler and the packaging to a shipping station.
Clause 19F. A method as in Clause 17F further comprising:
the engine instructing movement of the traveler to receive a second product dispensed from a second carousel under second environmental conditions different from the first environmental conditions.
Clause 20F. A method as in Clause 1F further comprising:
the engine instructing assembly of the first tray from a plurality of injection molded parts based upon dimensions of the first product.
Clause 1G. A method comprising:
an engine referencing data in a database to display a plurality of related products, the data comprising a product identifier and a product quality;
the engine receiving from a customer a single selection of a category of the related products; in response to the single selection, the engine instructing a system to dispense a subset of the related products matching the category.
Clause 2G. A method as in Clause 1G wherein the plurality of related products comprise ingredients of a meal recipe.
Clause 3G. A method as in Clause 2G wherein the category comprises a meal type.
Clause 4G. A method as in Clause 3G wherein meal type comprises breakfast, lunch, dinner, or dessert.
Clause 5G. A method as in Clause 1G further comprising:
the engine instructing the system to dispense the subset further based upon a second input removing certain products from the entire set of related products.
Clause 6G. A method as in Clause 1G further comprising:
the engine receiving a second input specifying a substitute product; and
the engine instructing the system to dispense the substitute product in place of one of the products of the subset.
Clause 7G. A method as in Clause 1G wherein the plurality of related products are based upon a purchase history of the customer stored in the database.
Clause 8G. A method as in Clause 1G wherein the plurality of related products are based upon a preference of the customer stored in the database.
Clause 9G. A method as in Clause 1G wherein the plurality of related products are based upon a budget of the customer stored in the database.
Clause 10G. A method as in Clause 1G wherein the plurality of related products are based upon an existing inventory of the customer stored in the database.
Clause 11G. A method as in Clause 1G further comprising:
sorting a plurality of related products based upon a quality assigned according to an initial inspection; and loading the plurality of related products into respective trays based upon the sorting.
Clause 12G. A method as in Clause 11G further comprising:
the engine instructing assembly of the plurality of trays from injection molded parts based upon dimensions of the plurality of related products.
Clause 13G. A method as in Clause 11G wherein the initial inspection comprises an optical inspection.
Clause 14G. A method as in Clause 13G wherein the optical inspection comprises a visual inspection.
Clause 15G. A method as in Clause 11G wherein the initial inspection comprises spectroscopy.
Clause 16G. A method as in Clause 11G further comprising:
the engine instructing storage of the respective trays under controlled environmental conditions based upon the inspection.
Clause 17G. A method as in Clause 11G further comprising:
the engine instructing a subsequent inspection prior to dispensing the plurality of related products.
Clause 18G. A method as in Clause 17G where the subsequent inspection is of a same type as the initial inspection.
Clause 19G. A method as in Clause 17G where the subsequent inspection is of a type other than the initial inspection.
Clause 20G. A method as in Clause 20G wherein the initial inspection comprises an optical inspection.
Clause 1H. An apparatus comprising:
a plurality of unique individual product items stored upon trays within a plurality of carousels under different environmental conditions;
a non-transitory computer-readable storage medium configured to record at least a location and a status of each unique individual product item;
a graphic user interface (GUI) configured to offer a customer selection between a first unique individual product item and a second unique individual product item; and
an engine in communication with the GUI and the non-transitory computer-readable storage medium, the engine configured to instruct dispensing of the first unique individual product item from a first carousel into a packaging in response to the customer selection.
Clause 2H. An apparatus as in Clause 1H wherein the first unique individual product item and the second unique individual product item differ from each other based upon an attribute.
Clause 3H. An apparatus as in Clause 2H wherein the attribute is stored in the non-transitory computer readable storage medium associated with the location and the status.
Clause 4H. An apparatus as in Clause 2H wherein the attribute is based upon a first inspection performed prior to the dispensing.
Clause 5H. An apparatus as in Clause 4H wherein the first inspection is performed prior to loading the first unique individual product item into the first carousel.
Clause 6H. An apparatus as in Clause 5H wherein the first inspection is performed with the first product item disposed on a tray.
Clause 7H. An apparatus as in Clause 5H wherein the first inspection is performed prior to the first product item being disposed on a tray.
Clause 8H. An apparatus as in Clause 7H further comprising an apparatus configured to assemble the tray from a plurality of injection molded pieces upon receipt of an instruction from the engine.
Clause 9H. An apparatus as in Clause 4H wherein the first inspection comprises rotating the first unique product item to obtain multiple views.
Clause 10H. An apparatus as in Clause 9H further comprising a conveyor configured to handle the first unique individual product item during the first inspection.
Clause 11H. An apparatus as in Clause 10H wherein the rotating is orthogonal to a direction of travel along the conveyor.
Clause 12H. An apparatus as in Clause 10H wherein the conveyor comprises a pivoting belt conveyor.
Clause 13H. An apparatus as in Clause 10H wherein the conveyor comprises a star wheel.
Clause 14H. An apparatus as in Clause 10H wherein the conveyor comprises a roller.
Clause 15H. An apparatus as in Clause 14H wherein the conveyor comprises a spinner and roller conveyor.
Clause 16H. An apparatus as in Clause 10H wherein the conveyor comprises a pop up conveyor.
Clause 17H. An apparatus as in Clause 4H further comprising a gantry to perform the rotating.
Clause 18H. An apparatus as in Clause 4H further comprising a camera to perform the first inspection.
Clause 19H. An apparatus as in Clause 2H wherein the attribute is further based upon a second inspection performed during the dispensing.
Clause 20H. An apparatus as in Clause 2H wherein the attribute comprises a current quality, a projected future quality, or a ripeness.
Clause 1I. An apparatus comprising:
a plurality of unique individual product items stored within a plurality of carousels under different environmental conditions;
a non-transitory computer readable storage medium recording, an inspection result of each of the plurality of unique individual product items, and
a location of each of the plurality of unique individual product items;
a processing engine in communication with the plurality of carousels and with the non-transitory computer readable storage medium; and
a plurality of stations associated respectively with the plurality of carousels and configured to dispense unique product items into packaging upon receipt of an instruction from the processing engine.
Clause 2I. An apparatus as in Clause 1I wherein the non-transitory computer-readable storage medium further records a Stock Keeping Unit (SKU) related to the inspection result and the location.
Clause 3I. An apparatus as in Clause 1I wherein the non-transitory computer-readable storage medium further records a product item quality related to the location and generated from the inspection result by the processing engine.
Clause 3I. An apparatus as in Clause 1I wherein the inspection result comprises an image.
Clause 4I. An apparatus as in Clause 3I wherein the image comprise visible wavelengths.
Clause 5I. An apparatus as in Clause 3I wherein the image comprises other than visible wavelengths.
Clause 6I. An apparatus as in Clause 1I wherein the inspection result comprises a spectrograph.
Clause 7I. An apparatus as in Clause 1I wherein the inspection result comprises chemical sensing.
Clause 8I. An apparatus as in Clause 7I wherein the chemical sensing detects spoilage.
Clause 9I. An apparatus as in Clause 7I wherein the chemical sensing detects a mold spore.
Clause 10I. An apparatus as in Clause 1I wherein sorting the plurality unique individual product items into the plurality of carousels is based upon the inspection result.
Clause 11I. An apparatus as in Clause 1I further comprising a mechanism configured to handle the plurality of unique individual product items during inspection for disposing on a tray.
Clause 12I. An apparatus as in Clause 11I wherein the inspection comprises rotating a unique product item to obtain multiple views.
Clause 13I. An apparatus as in Clause 12I wherein the mechanism comprises a conveyor.
Clause 14I. An apparatus as in Clause 13I wherein the rotating is orthogonal to a direction of travel along the conveyor.
Clause 15I. An apparatus as in Clause 13I wherein the conveyor comprises a pivoting belt conveyor.
Clause 16I. An apparatus as in Clause 13I wherein the conveyor comprises a star wheel.
Clause 17I. An apparatus as in Clause 13I wherein the conveyor comprises a roller.
Clause 18I. An apparatus as in Clause 13I wherein the conveyor comprises a spinner and roller conveyor.
Clause 19I. An apparatus as in Clause 13I wherein the conveyor comprises a pop up conveyor.
Clause 20I. An apparatus as in Clause 12I wherein the mechanism further comprises a gantry to perform the rotating.
The platform is in communication with a facility 7610 via a communications network 7611. In particular, the engine is configured to provide instructions 7612 to cause the facility to intake 7614 a plurality product items in bulk form, to perform an inspection 7616 (e.g., utilizing camera 7617) of those bulk items, and to sort 7618 those bulk items on the basis of the inspection.
Further on the basis of the inspection, the platform is further configured to instruct an apparatus 7620 to assemble trays 7622 from individual parts 7624 (e.g., injection molded pieces).
The trays then receive the sorted product items, and are in turn loaded 7626 into carousels 7628. As has been described extensively above, those carousels are maintained under particular set(s) of environmental condition(s).
The platform is further in communication with non-transitory computer-readable storage medium 7630 having data stored thereon. For example, database 7632 may comprise data regarding various aspects of the system and the products stored therein.
Data stored in the database may be referenced by the platform to fulfill 7633 a customer order. In particular, the customer may issue to the platform, a request 7640 specifying a particular item type 7642 (e.g., a specific apple varietal such as golden delicious). The platform may receive the request, and reference that item according to a table storing SKU information.
Then, with reference to data in the database, the platform may return to display to the customer in the GUI, data regarding a plurality of unique individual product items 7644, 7646, and 7648 matching the request. The data may comprise specific information 7650 regarding those individual product items, including but not limited to an image, a lifetime, and others.
Then, the GUI is configured to receive a user selection 7652 of one of the particular product items. In response, the engine is configured to instruct a dispensing station 7653 of the facility to dispense that unique individual item from the tray and carousel, into packaging 7654 supported in a moveable traveler 7656.
While the particular embodiment shown in
In some embodiments, it is noted that the processing engine may communicate to the facility, an instruction to transfer 7657 a product item from one storage carousel to another. Such a transfer instruction can be based upon considerations such as an expected ripeness of the product item, a storage capacity of a particular carousel, and an expected future order of a product item (e.g., as may be predicted by machine learning). This transfer can be accomplished utilizing a traveler (as shown), or alternatively utilizing a different system component.
According to certain embodiments, the product dispensing and/or transfer may be accompanied by an inspection 7680 (e.g., as performed by a camera 7682). This dispensing/transfer inspection may be the same, or may be different, from the original inspection forming the basis for the initial sorting.
Finally, once all of the selected customer items have been collected in the packaging, the engine is configured to instruct the traveler to move along path 7658 to shipping station 7660. There, the packaging is offered for delivery 7662 to the customer.
According to certain embodiments, this delivery may comprise the customer receiving the product items in the packaging as conveyed via a vehicle (e.g., drone, truck, car, bicycle). According to alternative embodiments, however, delivery could not feature an automated drop, but rather pickup by a human customer. Such manual delivery options could utilize visual cues such as indicator lights or screen cartoons.
It is noted that the non-transitory computer readable storage medium of
For example, according to some embodiments a customer may be able to provide feedback about specific items. That feedback can be used to better predict preferences and other quality attributes.
Furthermore, information regarding item availability able to be combined with expected demand and expected inventory replenishment, could be utilized to incentivize specific consumption trends. This can be accomplished by, amongst other techniques, placement in web browsers, relative pricing, promotions, and suggestions of recipes that include such items or otherwise.
The item gets lifted by the rollers, the rollers spin and cameras take pictures from above.
The items is dropped back onto the conveyors and the items advances forward with its classification.
To locate the tray, a click in place feature was added to the carrier and tray. Specifically, when the tray is inserted, it moves past four clasps 7804 (two on each side of the tray) that are shown in the wall edge view of
The carrier embodiment shown may offer benefits by avoiding the consumption of space otherwise needed to fit the forks into the carrier to remove the tray. Now, without that consumed space, the tabs can be smaller.
Also, because the dispensing tool does not need to fit into the carrier, a higher number of slots can be designed for trays. In this particular embodiment, 2× the number of slots have been added. This affords flexibility when inserting trays into the carrier, to achieve maximum density.
According to some embodiments, an interior of a carousel may be lined with foam panels.
These foam panels attached to the walls with a hooking bracket, illustrated in perspective and side views in
Here, the hooking bracket 7900 has teeth 7902 that sink into the foam of the panels. The bracket may or may not be glued on, too. Also, the bracket may be covered with some of the same coating that the foam is covered in. The bracket may or may not be inset into a pocket in the foam.
The frame of the carousel could have a projecting stud 7904 with which the bracket could mate. So, to install the panel, the foam gets lifted so the stud goes through the large holes. Then the panel is lowered down and two or more brackets get secured to the studs as they slide into the key hole shape.
Such an approach may offer certain benefits. For example, it avoids tolerance considerations attributable to the carrier tilting back forward. There will be a tolerance in the location of each carrier relative to each other.
With this design, the hook fits into a large area and has a big tolerance itself. Now, the position can be off by >5 mm and still engage correctly. This increases tool robustness, and may reduce the cycle time of the dispensing process because less precision is needed.
This embodiment also features a left-to-right belt drive design 8104 that has a fixed timing belt 8106 and pulleys 8108 that move through it. Again, this reduces cost and complexity, disposing the motors onto the pop through conveyor and making the tool easier to manage.
This approach allows larger-sized items to be stored and dispensed. Such items may not normally fit into a bag, but this design saves the manual effort of walking down an aisle and picking them from a shelf.
This could be used when items are too fragile to be dropped onto other, harder items in a bag. A peach, for example, might get damaged falling onto a box corner or the stem of a butternut squash.
Clause 1J. An apparatus comprising:
a frame translatable to a staging position in front of a carousel;
a fork supported by the frame and configured to extend in a direction toward the carousel to engage a tray holding an item; and a moveable member configured to, project vertically into a first opening of the tray, contact the item at a first point, and dispense the item from the tray in a direction away from the carousel, into packaging.
Clause 2J. An apparatus as in clause 1J wherein the moveable member comprises a conveyor.
Clause 3J. An apparatus as in claim 2J wherein the conveyor comprises:
a first conveyor belt configured to project through the first opening and contact the item at a first point; and
a second conveyor belt configured to project through a second opening in the tray and contact the item at a second point.
Clause 4J. An apparatus as in clause 3J wherein the first conveyor belt and the second conveyor belt are parallel.
Clause 5J. An apparatus as in clause 2J further comprising a backstop configured to prevent an item from falling off a back side of the tray opposite to the staging position.
Clause 6J. An apparatus as in clause 1J further comprising a load cell.
Clause 7J. An apparatus as in clause 1J wherein the moveable member is configured to translate in a horizontal direction toward the staging position after projecting through the first opening.
Clause 8J. An apparatus as in clause 1J further comprising an item location sensor.
Clause 9J. An apparatus as in clause 8J wherein the item location sensor comprises an optical sensor.
Clause 10J. An apparatus as in clause 9J further comprising an optical fiber.
Clause 11J. An apparatus as in clause 1J wherein the moveable member is configured to move in a vertical direction to project into the first opening.
Clause 12J. An apparatus as in clause 1J wherein the tray is configured to move in a vertical direction to allow the moveable member to project into the first opening.
Clause 13J. An apparatus as in clause 1J wherein the tray comprises an injection molded part.
Clause 14J. An apparatus as in clause 13J wherein the tray comprises a plurality of injection molded parts assembled by a machine.
Clause 15J. An apparatus as in clause 1J wherein the fork comprises a hook configured to engage a hole of the tray.
Clause 16J. An apparatus as in clause 1J wherein the fork is configured to extend along a side of the tray.
Clause 17J. An apparatus as in clause 1J wherein the fork is configured to align with the tray utilizing a pin.
Clause 18J. An apparatus as in clause 1J further comprising a pulley.
Clause 19J. An apparatus as in clause 1J further comprising a spline.
Clause 20J. An apparatus as in clause 1J further comprising a belt.
As discussed at length above, embodiments of product handling systems may feature traveler(s) that are configured to receive dispensed items and to transport same for delivery (e.g., to a customer's vehicle or a delivery vehicle—manned or unmanned). It is noted that a variety of different mechanisms in the form of a Line Following Robot (LFR) may be utilized by embodiments for product handling purposes.
Examples of such mechanisms are summarized below in the following table.
Here, the term “Line Following” does not necessarily require guidance that is based upon line following principles. Rather other forms of movement are possible, including but not limited to the use of markers, active radar, and others.
According to certain embodiments, product handling approaches may feature item storage on fixed shelving rather than in a moveable carousel.
This product handling apparatus 8400 comprises moving carousel 8402.
By contrast,
The product handling apparatus of
In particular, the lift mechanism may comprise a shuttle that can move up and down to contact and remove a tray from the fixed shelving storage. The tray is then brought downward to the dispense station, where product items are dispensed into the LFB.
The view of
Returning to
However, this is not required and alternative embodiments may employ a dispensing mechanism that is itself vertically moveable, in conjunction with products that are supported on fixed shelving rather than in a moveable carousel.
Returning to
As shown in
Moreover the further alternative embodiment 8560 shown in
While the embodiment of
The left portion 8600 of
The right portion 8620 of
Lift 8626 replaces used trays. Lift 8628 fetches new trays. Tray 8630 brings trays from deep storage into the lower area.
To reduce cost, it may be desirable for a product handling system to maximize storage density. Environmental control (e.g., temperature, humidity, gas species and concentration control, particulate filtering) in such a product handling system allows significant shelf life extension for perishable items. However, environmental control may call for sealing and/or insulating storage, reducing density.
Accordingly, embodiments may integrate sealing, insulation, and ducting to structural members. This allows for the benefit of environmental controls with little detriment to storage density.
Perforated duct plate 8714 turns structural members into a designated refrigeration air path. The air exits holes 8716 and refrigerates items on the shelving slots
Under certain circumstances, a product item may need to be retrieved from a center of a tray. Such a center-tray-located item can be delivered by adding a second set of conveyors behind the dispense area. This provides a low cost way to dispense an item from any location in the tray.
While these particular figures show a specific embodiment that engages in product handling to access center tray items within the dispense area, this is not required. Alternative embodiments featuring the dispense conveyor located on the lift, could perform product handling above the dispense area somewhere in the storage area. A secondary conveyor could be in the back or front of the lift.
Embodiments as just described can be utilized when selecting specific cuts of meat, cheese, produce, etc. Such embodiments can also be employed where multiple, low sale SKUs (like bottles of wine or spices) are present in one lane of the tray. Such embodiments can be used as a method of combining inventory on less than full rows and trays.
In particular, motors for the elevator and tray retrieval are located near the ground. Power is transmitted to the elevator carriage via timing belts. This configuration keeps the motors and electronics away from the potentially wet environment (e.g., condensation) of the vertical lift system.
It is noted that certain tray designs may lend themselves for beneficial use in connection with a vertical lift system.
Software may be utilized to implement certain features in product handling. For example, software may optimize a speed of fulfillment of an order.
In the case of inventory management considerations, the most commonly purchased items may be stored in a few adjacent carriers within each carousel, or at proximate locations within the fixed shelving. Rarer items may be stored in the remaining carriers or fixed shelving.
Most of the time, the carousel will likely be moving between these few “high runner” carriers. Ultimately, this arrangement will reduce the time it takes to fill each order.
In the high runner carriers, only a limited quantity of the high runner items may be stored. Others items are located on less frequently used carriers, accessed if the high runner carriers are empty or if the carousel happens to be closer to that carrier.
Where fixed shelving is used, high runner storage may be located at lower locations relative to the dispense mechanism. This can reduce the time required for repeated vertical movement in order to access the high runner storage locations.
There can be empty tray slots in carriers within each carousel. Accordingly, after a dispense routine, a tray can be returned to a different location than the location it was originally taken from. This imparts flexibility in optimizing fulfillment speed. A routing manager can optimize each LFB route to minimize interference with other LFBs and reduce fulfillment time.
LFBs have the ability to easily pass by other active LFBs. In this way, LFBs can pick items from carousels and bays that are not busy.
A routing manager knows the status of each carousel and bay. The manager can implement planning to minimize time that LFBs wait in line behind other LFBs.
Where possible, each SKU is distributed throughout the store in multiple locations. SKUs can be distributed in one or more of multiple:
This imparts flexibility in optimization by the routing manager. Larger orders can be prioritized over smaller order in order to ensure orders are fulfilled in an acceptable time.
The routing manager can signal to a carousel and/or bay to prepare a tray for dispensing before an LFB even arrives at the bay. This can further reduce wait times.
These considerations allow for faster fulfillment time estimates being offered to customers when they place an order.
It is noted that carousels can be designed with different numbers of bays. For a typical carousel with three bays, the actions of all three bays may be coordinated to the movement of one carousel motor. This can result in unwanted interference and wait times between the bays. However, carousels designed to control only a single bay avoid this problem, thus reducing interference and fulfillment times.
Idle LFBs can be stationed throughout the facility in order to reduce travel time to a first fulfillment bay. Carousels can be arranged on paths of different lengths and shapes to optimize speed of LFBs through facility. LFBs can travel in a variety of directions and paths to best navigate the facility based on the items to be retrieved from the carousels.
Once an order is finished, the LFB will travel directly to a waiting driver in the delivery area. The driver will remove the bags from each LFB and put them in their vehicle. This reduces time and labor. There is no need to stop the LFB somewhere, to remove the bag, and then to move it again out to the vehicle.
Where a vehicle has not yet arrived at the delivery area, the LFBs can drive into a temperature controlled chamber until the content of their bags is ready to be picked up.
Once an LFB is filled, it can drive to a parking lot or delivery area of the building to meet a vehicle. The bag can then be removed from the LFB and placed in the vehicle.
In certain embodiments, a human may remove the bag from the line following robot (LFR). In some embodiments, a tool may put the bag into a (self-driving) vehicle or drone.
Such delivery to a vehicle is not limited to a bag of a LFB. Alternative embodiments could have the delivered item in a box (LFBX) or sitting on a platform (LFP). Such a procedure may conserve labor as no human being is required to prepare the order.
The LFR can wait in a queue for the vehicle to arrive. If item(s) need to be refrigerated or to remain frozen, the LFR can move to a cooler or freezer while waiting for the vehicle to arrive.
Embodiments may split up individual orders amongst multiple LFBs. Each such LFB may then navigate independently throughout the store, receiving items in parallel.
Dynamic pricing and/or incentives can be used to reduce traffic during peak hours. This can more evenly distribute orders over the course of a day.
Software may also be employed to enhance a product inspection process for quality. Certain embodiments may employ software for quality inspection in connection with:
As previously disclosed, machine learning models may be trained to detect defects on visual and NIR images of produce. One approach may be based upon object detection varieties of machine learning.
However, embodiments are not limited to this or any other type of machine learning approach. Alternatives which may be used can include but are not limited to, anomaly detection (auto-encoders or GANs) or segmentation techniques.
Prediction models can be developed per individual product SKU. Alternatively, prediction models can encompass multiple types of SKUs. Accordingly a single prediction model can be used to predict quality for ten different apple varieties, or a single prediction model can be used for all stone fruits.
Three-dimensional images of produce, or three-dimensional reconstructions of produce from a series of two-dimensional images, could also be used as the basis for training predictive models and detecting defects. Such an approach can add a variety of benefits, including the ability to judge defect depth and topography.
Machine learning methods according to embodiments can reveal multiple types of information about each detected defect. Such defect information types can include:
According to particular embodiments, a second machine learning model can take some or all of these inputs (e.g., quantity, type, size, confidence) and infer a resulting output grade for the overall produce quality. Possible grades may be A/B/C, for example.
This machine learning model can be trained based on a human quality expert applying a grade to each item. This machine learning model can also be trained based on consumer feedback to match consumer preference.
According to embodiments, machine learning models can also be used to predict a simple pass/fail quality grade on items, without first identifying specific individual defects. In such an implementation, the machine learning model can be trained on a human quality expert applying a pass/fail grade to each item.
Alternatively, or in conjunction with human training, the machine learning model can be trained on sets of images that are individually labeled as pass/fail. Thus simple pass/fail grading could also be replaced by multiple levels of categorization (A/B/C, etc.)
Such approaches could also be extended to items in the store beyond spoilable produce. For example, embodiments could identify damaged cans, boxes, etc. utilizing machine learning prior to storage and/or delivery to a customer.
Software may also be employed to perform inspection for Quality Analysis (QA) and/or presentation to customer. Such inspection may take place at induction, with 2D or 3D rendering allowing a user to select their preferred item. Such inspection may also occur during fulfillment, for purposes of Quality Control (QC).
According to certain embodiments, inspection may occur at the time of induction of the product to the store. Inspecting (with 2D or 3D imaging) products such as produce as it enters the store, followed by tracking of items in inventory, customers can be afforded an option to select specific produce items (e.g. for a cart) based on the saved mages. A series of 2D images can ultimately be used to create a 3D reconstruction of the produce.
The results of later (after intake) inspection of produce or other product items, can be used to update the 2D or 3D representation(s) that are offered to customers during the selection process. These images could also be used to update a customer after checkout, regarding the items that have been dispensed into their grocery bags.
Software may also be employed in connection with product inspection during fulfillment. For example, cameras attached to dispense stations or the carousel, can image:
According to some embodiments, a final image of each bag can be taken after the LFB has finished collecting all of its items, or after multiple bags from different LFBs are combined. These images can be sent to fulfil QA teams for monitoring, or to customers to provide feedback on the quality of their order during fulfillment.
As previously disclosed, embodiments may utilize software for purposes of produce ripeness prediction. This may involve predicting fruit ripeness stage including the use of tools, and optimizing inventory via storage control.
Environmentally controlled storage vessels (e.g., fixed, or moveable carousels) allow maintaining product items at different conditions (e.g., temperatures, levels of humidity and ethylene) in order to control their ripeness progression.
Fruits ripen faster when exposed to higher temperatures and ethylene. By contrast, colder temperatures generally slow down the ripening process.
LFTs can be used to shuttle ripening product items such as fruits, between different storage environments based upon conditions optimal for the fruit, in order to maintain optimal inventory for customer needs. In this manner, product handing approaches according to embodiments can provide fruits that are currently ripe, or will be ripe at a predicted future time range.
As one possible example, avocados can be kept at about 16° C. until they reach a ripe state. Then, they can be moved to 3° C. At this colder temperature, the ripe avocados can be expected to remain ripe for about seven days.
Predictive modeling (e.g., accomplished by machine learning or other methods) can be employed to predict a future progression of ripening of product items. Such models can ingest information from one or more sources including but not limited to:
Using automated, semi-automated, or manual tools for measuring a ripeness stage of a fruit or other product can be beneficial in providing a current state fruit ripeness in a manner that is accurate, repeatable, and predictable.
A variety of measurement methods, employed alone or in combination, may allow for assessment of a current ripeness stage of a product item. Such measurements can include but are not limited to:
Such measurements can replace or supplement manual human measurements, including but not limited to:
In certain cases, periodic (e.g. once a day) measurement of product items is important for tracking the ripeness stage over time. In other cases, metadata of the product item, along with typical trends for that item, may be sufficient for predicting the future ripeness trends. There, a single measurement of ripeness during induction to the store location, or even no measurement at all, may be sufficient.
Sourcing items that are pre-conditioned (e.g., ethylene and/or temperature conditioned) can create more homogeneous batches of fruits. This can allow for sub-sampling when measuring fruit ripeness, or permit fewer periodic measurements over time.
It is noted that tools for ripeness measurement (e.g., firmness, acoustic, spectrometer, etc.) can be located in a variety of different store locations to provide periodic assessments of product ripeness.
One possible position within the store is in a temperature controlled induction or pre-storage location for manual or semi-automated measurements. Ripeness measurement could be integrated into the automated inspection equipment for measurement during induction. Or, measurement could be performed when LFTs remove the produce from controlled climate storage and return to the station for periodic measurements.
Ripeness measurement may be incorporated into the LFT itself, or may be present at measurement stations distributed throughout the store for easy and quick LFT access. Ripeness measurement may be incorporated into dispense stations, or within a storage (e.g., carousel) bay itself.
In an alternative design to a moving carousel, product items could be stored in environmentally controlled individual vertical storage bays. Such storage does not move like a carousel. Rather, trays are removed and returned to fixed locations utilizing a tray transport mechanism. Such a configuration is shown in the simplified cross-section of
The storage temperature can be modular even within a single bay. This allows for higher and lower temperatures for granular control over ripeness stage. And, such an approach allows for easy transport of ripening produce between different temperature conditions based upon ripeness stage, without the need for a LFT to move the trays.
Ripeness measurement tools can be incorporated into this same vertical bay or dispense station for easy periodic measurements of fruits over time.
A variety of different designs of mechanical apparatuses for product handling, may be used to accomplish product inspection. Some embodiments may use tray loading shuttle conveyors for mechanical inspection design, as shown in
In particular, items 9100 can be loaded from an infeed conveyor 9101 into multiple trays 9102 using multiple shuttle conveyors 9104 and pop-through conveyors 9105 to move the items to the correct tray location. There can be one or more shuttles, which can move to one or more trays.
A mechanism similar to that used for dispensing items from trays, may be used for loading of items onto trays. One example is a dual-belt conveyor mechanism.
The trays do not move. Once intake inspection of an incoming item is concluded, the infeed conveyors move that item forward onto the shuttle. Then, the shuttle moves the item laterally until it is aligned with the proper row of the tray upon which the item is to be loaded.
Particular embodiment of designs for mechanical inspection may employ rollers with spirals that turn an item ninety degrees.
In particular, such rollers may handle both spherical and cylindrical objects. Spinning spiral roller(s) can rotate non-spherical objects to their equilibrium position. Spiral ridges on the rollers contact objects diagonally with respect to normal.
Objects can shift lengthwise along path of the conveyor. Roller-to-roller distance may be adjustable to account for different object size.
Size detection camera(s) can recognize outliers to determine whether rollers need to be adjusted to account for large variances in size. Asymmetric objects can also be detected and accounted for by positioning rollers relative to predicted center of gravity. The roller assembly can move along a rail in order to adjust for size.
One example of an embodiment of an inspection induction system is shown in
Loose produce corresponds to individual spherical and cylindrical produce. Packed produce corresponds to clamshells, boxes, and bags of multiple produce (e.g., lettuce, bags of carrots, etc.)
Some embodiments of mechanical designs may feature circular V belt loading stations. Operation of such a design for loading is shown in the simplified side view of
In particular, this station is a high density loading station for both produce and dry goods. It allows a buffer to be loaded with items prior to entering either the inspection station and/or tray loading station.
One section of the conveyor belt is loaded up. Then, the station rolls, presenting a free conveyor section to an operator and a loaded section of conveyor to the conveyor leading to the next station. The conveyors can angle up to be a V for spherical and cylindrical items and flatten for packed items.
This buffer allows the operator to load the station, and then leave to accomplish another task.
Some embodiments of product handling mechanisms may feature a translating shuttle buffer. This offers a high density loading station for both produce and dry goods which allows a buffer to be loaded with items prior to entering either the inspection station and/or tray loading station.
As shown in the simplified view of
The conveyors can angle to assume a V-shape for handling spherical and cylindrical items, or flatten to handle for packed items. This buffer mechanism allows the operator to load the station and walk away to accomplish another task in the warehouse.
Some embodiments of product handling systems may utilized a stacked shuttle buffer. This station is a high density loading station for both produce and dry goods which allows a buffer to be loaded with items prior to entering either the inspection station and/or tray loading station.
As shown in the simplified side view of
Again, the conveyors can angle up to be a V for spherical and cylindrical items, and flatten for packed items. This buffer allows the operator to load the station and walk away to accomplish another task in the warehouse
Certain mechanical designs may allow for dynamic storage of cases of product. In particular, for a streamlined product handling system, cases of produce may be stored in a manner promoting autonomous dispensing and an ability to be brought directly to the tray loading station.
As shown in the simplified top view of
The product cases are then transferred into a bay in the shelving units 9704. As shown in the simplified cross-section of
The outbound conveyor carries the case to the tray loading station. The shelving is a conveyor that is driven by a connection to the gantry conveyor. In particular, geared connection between the motorized gantry conveyor and a non-motorized shelving unit allows transfer of cases between the two units
Embodiments may provide the functionality of permitting automatic re-inspecting. That is some SKUs may benefit from a second or third inspection on a specific schedule. Thus a LFT can
Alternatively, a camera above each pick station can perform inspection according to a schedule. If unacceptable items are found, they can be removed from the trays. If the ripeness or quality status has changed, the database of information can be updated accordingly.
Embodiments may offer the ability to automatically combine inventory. Specifically, throughout the fulfillment process each tray will have items dispensed from it for fulfillment to the customer.
Eventually, this repeated dispensing will result in trays being only partially filled. Such a condition may be undesirable because total inventory count is reduced.
At least two possible approaches may be used to address this issue. One approach is recombination.
Specifically at low demand times a recombination schedule may be created routing LFTs to each partially-filled tray. The trays will be removed and returned to the tray filling station. Trays will be removed from the LFT and placed on the loading station. Using the tray loading shuttle conveyors, for example, items can be removed from some trays and added to others until some trays are full and others are empty. At this point, the full trays can be replaced in the carousel.
Another approach involves addition. This proceeds similar to recombination, but rather than using the tray loading shuttle conveyors to recombine items, newly inspected items can be added to the partially filled trays.
Various features of tray designs are now described. It is noted that some product items do not interact as well with the dispense station. This can be due to one or more of:
A method allows such product SKUs to be dispensed in a similar manner to other items in the store. Specifically, each items gets placed on a small raft that fits with the tray. When the item is called for, the item and the raft are dispensed together.
In one embodiment the raft is captured before moving into the LFB. In an alternative embodiment the raft moves into the LFB and is removed at the end by a worker or a tool. In still another embodiment the raft is shipped to the customer for disposal.
Certain embodiments may utilize a tray design that allows dual side picking. This is shown in the simplified top view of
In particular, the tray includes pulling features that interface with the arms of the dispense station in order to contact and place the trays. The symmetrical design of the tray allows the tray to be used in either orientation, and allows the tray to be contacted and placed in either direction. This is discussed later below in connection with at least
Certain specialized tray designs may be employed for handling of heavy, bulk items. rays for heavy, bulk items. Examples can include but are not limited to Examples to 24-packs of bottled water, toilet paper, paper towels, bales of firewood, 24 packs of beer, cases of wine, and boxes of product that have not been unpacked.
Items would sit on this tray platform and be lifted off, conveyed forward, and then conveyed onto a Line-Following Platform (LFP). The LFP would then transport the item to the shipping area. Details regarding LFP embodiments are discussed further below.
Returning now to product storage within a moveable carousel, embodiments thereof can include various features. One such feature is a quick install carrier attachment.
Specifically, the storage carousel operates by moving shelving units. Mounting such shelving units to the components inside the carousel during installation, may be made more difficult by factors such as:
Accordingly,
As shown in
Embodiments may use a wedge pushed into a cone by screws to generate radial clamping force on the shaft. This fixes the joint radially and axially. The large mechanical advantage from screw+wedge, allows large clamping forces to be generated.
It is noted that certain carousel designs may include a single bay. Carousels can include various numbers of bays, all controlled by a single motor. Typical options include 1, 2, 3, 4 or 5 bays.
The more bays per motor, the larger the cost savings. This is because each bay shares the mechanics required for movement.
However, the fewer bays per motor, the faster the system can operate.
Each bay may be tied to an independently operating bay on the pick station. When more than one pick station bay is linked with a multi-bay carousel, conflicts may arise slowing the system and creating bottlenecks.
Embodiments may vary the number of different sized carousels in a facility in order to improve the fulfillment process. Thus for one particular facility it might make sense to have all single bay carousels. However, another facility may best utilize a combination of different sizes. This determination can balance speed requirements with cost.
Various features of a dispensing station according to certain embodiments, are now disclosed. In particular,
As indicated above, this dispense station embodiment adopts a dispense-next-to-lift approach. That is, the lift is not also leveraged to provide the vertical movement allowing the product conveyors to project through the tray slots.
Hence, the product dispensing mechanism is capable of vertical movement between a lowered and raised position.
As has been emphasized throughout, the dual conveyors of the product dispensing mechanism are independently actuable to move together or apart depending upon item size. That is, the conveyors can spread open/close depending upon item width.
According to this particular embodiment,
This is not required however, and alternative embodiments may employ other approaches for independently actuating dual conveyors for dispensing of product items. Specifically,
Here, motors 10060, 10062 are instead anchored to a fixed frame 10064 rather than to the carriages themselves. The first motor 10060 drives timing belt 10068 to actuate 10069 the carriage 10070.
The second motor 10062 drives timing belt 10072 to actuate 10073 the carriage 10072. This approach featuring a moving timing belt, was previously illustrated in the side view offered by
As mentioned earlier, a dispense mechanism may feature a hook design that allows for dual side dispensing (that is, dispensing from each of opposite sides).
The carousel and tray rover may not require complicated mechanisms. Rather, they may comprise shelving to hold a stored and dispensed item, respectively.
The arm of the dispensing station can reach in two directions. A first direction contacts and places from the carousel. This is shown in the perspective view of
A second, opposite direction contacts and places from the tray rover. This is shown in the perspective view of
While
It is noted that a combination color and depth cameras, may take 3D scans of a bag or box as it is filled with items. Fiducial markers (recognized by color cameras) may be located on a line-following robot, in order to allow transformation of the bag/box coordinate frame to the dispense robot coordinate frame. Using more markers than the minimum necessary, can increase accuracy and allow transformation in the event that some markers are obscured.
According to embodiments, a program may interpret a 3D scan before each dispense. The program may comprise hardware, software (e.g., instructions present in a non-transitory computer readable storage medium), or a combination thereof, executed by a processor.
The program may then select a dispense location based upon one or more of the following factors:
The program may also adjust the platform height of the line-following robot, in order to minimize a drop distance, while still ensuring successful dispensing while retaining items already present in the bag.
After each dispensing, the program may interpret another 3D scan, logging the location of new items. A 3D map of item locations evolves, allowing the association of various regions with features such as:
This map of features allows intelligent placement of new items into the bag (e.g., to avoid dispensing soft peaches onto hard cans). The post-dispense software program may also predict a dispensing location for the next item, thereby priming the line-following robot and the next dispense robot to meet more quickly at the next dispense location.
Fiducials located around a rim of the robot are recognized. The “droppable” area is shown as reference 10700. A dispense location 10702 is precisely selected based on a depth scan and existing data about the locations of items in the bag. The LFR moves horizontally to position the dispense location in front of the conveyor belt pair of the dispensing station.
Certain embodiments may follow specialized procedures for dispensing product items into a bag. Such approaches can be employed to pack bags with a degree of intelligence and care comparable to human effort.
According to some embodiments of procedures, packing density can be maximized, fitting more items into the same space. Damage to items may be minimized or eliminated, with items packed safely in the traveler so that they do not fall to the ground.
Some embodiments of packing procedures may rely upon physics simulation. Three-dimensional scans can be utilized in simulations that model the physical behavior of items falling off of the dispense conveyors and interacting with items already in the bag. Optimal drop impact and settling outcomes can be selected.
Particular embodiments of packing procedures may utilize reinforcement learning. A reinforcement learning model can be implemented using feedback on decisions made by the drop program. Such feedback can include but is not limited to:
Feedback could allow the model to improve success rate by adjusting certain numerical control parameters. Example parameters can include but are not limited to:
Certain embodiments may utilize a 3D camera for item height detection. Specifically, when a particular tray is stocked, the heights of items of that tray are known.
However in some instances items on the tray may have shifted. In other instances, the items could have variable sizes so that the maximum height is not known.
As has been discussed, a tray is loaded into storage (e.g., fixed, or moveable carousel). Each position within that storage may have a limited allowable height. Thus it may be beneficial to select a storage location accommodating the maximum height of items on a particular tray, so that storage density is optimized. Embodiments may measure the height of all items, determine the maximum height, and then select a storage location that is only slightly larger than that maximum height.
When a tray is first created, embodiments may employ a (stereoscopic) camera to create a heat map of the height of items in the tray. The maximum height can then be detected from the data. This information is used to select the location to which the tray will travel in the storage (e.g., LFT).
Once the tray is located at the picking station, a (stereoscopic) camera can be used to check the heights again, verifying that tray items did not shift during transport. This data can them be used to confirm the final location of the tray in the storage.
Each time a tray is removed from storage, there is some risk that the maximum height has changed. This can be rechecked each time, in order to ensure that the tray and the items it bears, will still fit.
If a camera or other sensed information reveals that the tray and items no longer fits into the storage, one or more of the following actions can be taken, as follows.
Various embodiments may employ a 3D camera to monitor fulfillment success. In particular, certain embodiments may fix a (stereoscopic) camera above the picking station.
That camera can monitor the low of items in and out of the bag. The camera can be used to determine the location of items in the tray.
This location information can be fed back to the control software, so that the correct number of items is dispensed. Such a configuration could replace or be used in combination with a fiber optic sensor.
Such a 3D camera can be used to detect when too many or too few items were dispensed. When this happens, an error procedure can be followed to correct the issue.
A 3D camera could be used to detect if an item did not enter the LFB. When such an event occurs, an error procedure can again be followed to fix the issue.
A 3D camera can be used to detect which product item moves from one row into another, when a row is being dispensed. When this movement between tray rows occurs, error procedure can be followed to correct the problem.
A 3D camera can be used to detect when an item is missing from the tray that should otherwise be there. Again, an error procedure can be initiated to correct the issue.
As indicated, the processor is configured to receive one or more inputs 11406. These inputs can include information received from a dispense station, from a traveler, from camera(s), from a tray handling apparatus, from storage, or from any other component of the product handling system.
In particular, input from the cameras can comprise images that indicate the position of the traveler, and the makeup of the goods inside. Camera input can form a basis for validation of how many items were dispensed properly.
In response to receiving such input(s), the processor is configured to reference 11407 additional information located in the non-transitory computer readable storage medium. Such additional information may be data stored in a database 11408, with any of the pieces of data discussed herein being possible candidates for storage in the database.
The computer-readable storage medium may also store instructions for performing certain specialized procedures according to various programs. Examples of such procedures can include but are not limited to one or more of:
Based upon this input and referenced stored data, the processor is configured to perform a large variety of calculations to produce output 11410 of different types. That output may be communicated to control one or more of the various components of the product handling system, including but not limited to the dispense station, storage, tray handling, intake, inspection, tray handling, and the others as have been described herein.
It is emphasized that the block diagram of
Moreover, a single processor may not be responsible for performing each and every calculation depicted. Rather processing may be distributed (in series, in parallel, or in some combination) amongst a plurality of processors for purposes of speed and efficient utilization of available computing resources.
In summary, embodiments for dispensing an item from a tray may utilize a depth camera to take a picture to find fiducials on a rim of a container (e.g., box, bag, platform) of a LFR. This image locates the LFR and container relative to the camera and the dispense conveyors.
The image may also inspect the heights of the items in the container. The image may allow locating an optimal dispense location, such that a box can be drawn around that dispense location (e.g., as shown and described above in connection with
Then, the LFR moves along the floor to position the dispense location to be in line with that dispense location box. The dispense conveyors extend in and out, relative to the container of the LFR, depending upon where the dispense location box is. If the dispense location box is in the back of the bag, the conveyors may extend out further.
A height of the container borne by the LFR may also adjust depending upon the object. If fragile items have already been dispensed to the container, and/or the item to be dispensed is fragile, then the LFR may move the container to a greater height to reduce a distance that the dispensed item will drop, and hence any impact force.
Where the item to be dispensed, and/or existing items already dispensed to the container are not fragile, then the dispensing may just drop the item to the correct location in an expedited manner. Thus certain approaches may direct packing of non-fragile items first, followed by packing of fragile items.
Additional details regarding possible LFB designs according to various embodiments, are now described. Briefly, the LFB is a device that holds a compliant bag from the top opening and supports it on a moving platform from below.
The upper portion of the LFB in
The LFB of
The platform height adjusts responding to commands and the volume of items inside of it and the desired space for the next item. The platform can have force sensors to determine bag weight
The LFB device can have sensors on the bag opening to determine item level in the bag. The LFB device can be either stationary, or have a motorized base.
In particular, a motorized version of a LFB (or LFT) can guide itself around a physical space utilizing one or more of the following:
According to certain embodiments, upper portions of a LFB device can feature an automated mechanism to offer bags to a human or to another automated mechanism. In some embodiments, an upper portion of a LFB device can feature an automated mechanism to grab bags from an automate feeder, and/or to preload the top of the bag in an open state.
Further details regarding a LFT design according to various particular embodiments, are now described. In particular, a LFT is a device that holds multiple trays separable from the device.
The LFT further includes a lifting platform mechanism 11006 to lift the tray stack. The LFT may additionally comprise proximity and load sensors to determine a presence and a mass of a lifted object. Sensors of the LFT may be used to determine a level and a height of the tray stack.
Also included in
A LFT device may include a locking or retention feature that prevents trays from popping out. The LFT device may include sensors/markers that allow alignment to a specific tray level. The LFT device can stand on its own or be carried by a motorized unit.
When lifted by a motorized unit a mechanism may lock the tray stack to the motorized unit.
The LFT device may have features in the bottom and/or side that allow kinematic alignment to a mating component.
Further details regarding a LFP design according to various particular embodiments, are now described. In particular, some items may be too large to big to fit in a grocery bag. Examples can include but are not limited to 24 packs of water, 12 packs of paper towels, and firewood.
According to embodiments, such items can be stored in a bulk carousel or fixed shelving, and then dispensed using the same pick station tool. However, instead of dispensing into a bag, they could be dispensed onto a platform.
In particular, the LFP comprises a line following robot having a platform on top that can move cases or bulk goods dispensed from a carousel or fixed shelving. In some embodiments this platform would match the pick station height. However, the platform could also lower for easier packing into a delivery vehicle.
According to certain embodiments a platform could have conveyors built-in, to aid in dispensing items onto the platform. The platform could alternatively just be a smooth surface that items slide onto. According to this approach, the handling of even large product items can be automated.
Further details regarding a Line Following Mini-Bag (LFM) design according to embodiments, is now provided. In particular, the LFM is a subset of the LFB design which has some number of bags installed open, on a frame.
The LFM robot would travel around to receive dispensed specific SKUs better suited to ship in plastic bags. Examples of SKUs could include but are not limited to, wet or dirty produce, bulk goods like nuts or granola, and meats that could drip juices upon other items. The LFM robot would travel to the storage bay to collect the dispensed items, and then travel to the delivery area to be consolidated with the rest of the order.
It is noted that a LFT design may include various inspection capabilities. An alternative to inspecting trays (mid-storage life), is to pull them out of the dispensing stations and onto the LFT.
The LFT could have an inspection camera present. That camera could inspect the quality or ripeness of the items, and then a processor could decide how to handle those items.
As has been previously described, the storage and handling of product items under climate controlled conditions, can be desirable. It is noted that embodiments of either of/both the LFB and the LFT could include on-board environmental control. This would allow maintenance of controlled climate conditions throughout the fulfillment process.
Certain embodiments could provide for a cold material (e.g., an ice pack or other chemical cooler) to be incorporated into the structure. Such a LFB and/or LFT could be stored in a cold area when not in use, and then move out to fulfil orders as needed.
According to some embodiment, LFT processes could be performed in climate-controlled (e.g., cold) areas, and LFTs could have cold waiting areas as needed, before they have the opportunity to place the trays into storage (e.g., fixed, or moveable carousels). Similarly LFBs could move into a climate controlled holding area after they have finished filling an order, but prior to arrival of the transport vehicle (e.g., customer or delivery van).
Still another possible design which could be employed in connection with item handling and delivery, is the LFBX. The LFBX design may comprise a line following robot featuring a box on top. This box could be any size or shape.
The box of the LFBX is specifically designed to interact with the dispensing system. This facilitates other item packing methods that are better suited for specific product types not otherwise working with a LFB design.
Clause 1K. An apparatus comprising:
fixed storage shelving maintained under climate controlled conditions;
a tray supported by the fixed storage shelving and bearing an item having a dimension;
an elevator moveable in a vertical direction from a first height proximate to the fixed storage shelving, to a second height proximate to a dispense mechanism; and
an element configured to move the tray from the fixed storage shelving to the elevator,
wherein the dispense mechanism comprises,
Clause 2K. An apparatus as in clause 1K wherein the second height is overlying the dispense station, such that lowering of the elevator causes the first conveyor belt and the second conveyor belt to project through the tray.
Clause 3K. An apparatus as in clause 1K wherein the second height is adjacent to the dispense mechanism.
Clause 4K. An apparatus as in clause 3K wherein the element is further configured to move the tray from the elevator to the dispense mechanism in order to allow the first conveyor belt and the second conveyor belt to project through the tray.
Clause 5K. An apparatus as in clause 1K wherein the element comprises a first hook configured to engage the tray and move the tray in a first direction.
Clause 6K. An apparatus as in clause 5K wherein the tray further comprises a pin engaged by the first hook.
Clause 7K. An apparatus as in clause 5K wherein the element further comprises a second hook configured to engage the tray and move the tray a second direction opposite to the first direction.
Clause 8K. An apparatus as in clause 7K wherein the first hook and the second hook are connected and driven together in the first direction and in the second direction.
Clause 9K. An apparatus as in clause 1K wherein both the first conveyor belt and the second conveyor belt are configured to extend in the first direction prior to dispensing the item.
Clause 10K. An apparatus as in clause 9K further comprising a traveler configured to receive the item from the dispense mechanism and transport the item for delivery to a vehicle.
Clause 11K. An apparatus as in clause 10K further wherein a bag of the traveler receives the item.
Clause 12K. An apparatus as in clause 1K wherein the fixed storage shelving comprises an insulating panel and defines a hole through which air is flowed.
Clause 13K. An apparatus as in clause 1K wherein the tray defines a first slot through which the first conveyor belt projects, and defines a second slot through which the second conveyor belt projects.
Clause 14K. An apparatus as in clause 1K wherein the elevator is disposed within a shaft, the apparatus further comprising:
an idler located at a top of the shaft above the first height; and a motor located at a bottom of the shaft below the second height.
Clause 15K. An apparatus as in clause 1K wherein the second conveyor belt is moveable relative to the first conveyor belt utilizing a timing belt.
Clause 16K. An apparatus as in clause 15K wherein the timing belt is fixed and a motor moves with the second conveyor belt along the timing belt.
Clause 17K. An apparatus as in clause 15K wherein the second conveyor belt moves with the timing belt.
Clause 18K. An apparatus as in clause 1K wherein the second conveyor belt is moved relative to the first conveyor belt according to the dimension stored in a non-transitory computer readable storage medium.
Clause 19K. An apparatus as in clause 1K wherein the tray comprises a plurality of injection molded parts assembled by a machine.
Clause 20K. An apparatus as in clause 19K wherein the machine references the dimension stored in a non-transitory computer readable storage medium in order to assemble the plurality of injection molded parts into the tray.
Clause 1L. An apparatus comprising:
a first conveyor belt configured to project through a tray to contact an item at a first point; and
a second conveyor belt moveable relative to the first conveyor belt and configured to project through the tray to contact the item at a second point,
wherein after a vertical position of the tray relative to the first conveyor belt and to the second conveyor belt is changed, the first conveyor belt and the second conveyor belt are configured to dispense the item from the tray in a first direction.
Clause 2L. An apparatus as in clause 1L further comprising:
a first element configured to move the first conveyor belt in a vertical direction to change the vertical position of the first conveyor belt relative to the tray; and
a second element configured to move the second carriage in the vertical direction to change the vertical position of the second conveyor belt relative to the tray.
Clause 3L. An apparatus as in clause 1L further comprising an element configured to move the tray in a vertical direction to change the vertical position of the tray relative to the first conveyor belt and to the second conveyor belt.
Clause 4L. An apparatus as in clause 3L wherein the element comprises an elevator.
Clause 5L. An apparatus as in clause 4L wherein the elevator is vertically moveable to receive the tray from controlled climate storage.
Clause 6L. An apparatus as in clause 5L wherein the controlled climate storage comprises a fixed shelf.
Clause 7L. An apparatus as in clause 1L wherein both the first conveyor belt and the second conveyor belt are configured to extend in the first direction prior to dispensing the item.
Clause 8L. An apparatus as in clause 1L wherein both the first conveyor belt and the second conveyor belt are configured to dispense the item by dropping into a bag.
Clause 9L. An apparatus as in clause 1L further comprising a third conveyor belt and a fourth conveyor belt configured to:
receive from the tray, another item conveyed by the first conveyor belt and the second conveyor belt in a second direction opposite to the first direction; and
after the item is dispensed, return the another item to the tray.
Clause 10L. An apparatus as in clause 1L further comprising a motor causing the second conveyor belt to move relative to the first conveyor belt using a timing belt.
Clause 11L. An apparatus as in clause 10L wherein the motor and the second conveyor belt move along the timing belt.
Clause 12L. An apparatus as in clause 10L wherein a location of the motor is fixed and the second conveyor belt moves with the timing belt.
Clause 13L. An apparatus as in clause 1L further comprising an element configured to remove the tray from a storage shelf prior to the first conveyor belt and the second conveyor belt projecting through the tray.
Clause 14L. An apparatus as in clause 13L wherein the element comprises a hook.
Clause 15L. An apparatus as in clause 13L wherein:
the storage shelving is fixed; and
the element is configured to move in a vertical direction together with another element that changes the vertical position of the tray relative to the first conveyor belt and to the second conveyor belt.
Clause 16L. An apparatus as in clause 15L wherein the another element comprises an elevator.
Clause 17L. An apparatus as in clause 13L wherein the storage shelving is part of moveable carousel.
Clause 18L. An apparatus as in clause 1L wherein:
a dimension of the item is stored in a non-transitory computer readable storage medium, and the second conveyor belt is moved relative to the first conveyor belt based upon the dimension.
Clause 19L. An apparatus as in clause 18L wherein the tray comprises a plurality of injection molded parts assembled by a machine.
Clause 20L. An apparatus as in clause 19L wherein the machine references the dimension to assemble the plurality of injection molded parts into the tray.
Clause 1M. An apparatus comprising:
a climate-controlled storage location;
a tray supporting an item in the climate-controlled storage location;
a dispense station configured to receive the tray supporting the item from the climate-controlled storage location;
a mobile robot configured to receive the item dispensed from the tray by the dispense station;
a processor configured to control operation of the dispense station; and
a first camera in optical communication with the dispense station and the mobile robot to send an image of the dispense station and the mobile robot to the processor.
Clause 2M. An apparatus as in clause 1M wherein if the image indicates the item has not been successfully dispensed to the mobile robot, the processor is configured to instruct the dispense station to take corrective action.
Clause 3M. An apparatus as in clause 1M wherein the processor determines a specific location of the traveler to receive the item based upon the image.
Clause 4M. An apparatus as in clause 3M wherein the specific location is within a bag, within a box, or upon a platform.
Clause 5M. An apparatus as in clause 3M wherein the traveler already includes other items.
Clause 6M. An apparatus as in clause 3M wherein the processor is configured to determine the specific location based upon a packing procedure.
Clause 7M. An apparatus as in clause 6M wherein the packing procedure considers one or more of the following:
item packing density within the traveler;
item damage;
successful dispensing;
physics simulation; and reinforcement learning.
Clause 8M. An apparatus as in clause 1M wherein the first camera is stereoscopic.
Clause 9M. An apparatus as in clause 1M wherein the image is a heat image.
Clause 10M. An apparatus as in clause 1M wherein the processor is configured to compare the image with another image of the tray including the item, taken by a second camera prior to the tray moving to the dispense station.
Clause 11M. An apparatus as in clause 10M wherein the processor comparing the image with the another image indicates that the item has shifted position on the tray.
This application is a continuation application of U.S. Nonprovisional patent application Ser. No. 16/418,901 filed on May 21, 2019, which application is a Continuation-in-Part (CIP) of U.S. Nonprovisional patent application Ser. No. 16/189,673 filed on Nov. 13, 2018 (issued as U.S. Pat. No. 10,543,942), which claims priority to U.S. Provisional Patent Application No. 62/589,409 filed on Nov. 21, 2017, and also to U.S. Provisional Patent Application No. 62/675,656 filed on May 23, 2018, which applications are incorporated entirely herein by reference for all purposes.
| Number | Date | Country | |
|---|---|---|---|
| 62675656 | May 2018 | US | |
| 62589409 | Nov 2017 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16418901 | May 2019 | US |
| Child | 16879192 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16189673 | Nov 2018 | US |
| Child | 16418901 | US |
