This invention relates, in general, to a machine for handling products and, more particularly, to a perception-based robotic manipulation system and method for automated unloading and unpacking which employs an automated truck unloader designed to unload and unpack product, such as boxes or cases, from trailers and containers.
Loading docks and loading bays are commonly found in large commercial and industrial buildings and provide arrival and departure points for large shipments brought to or taken away by trucks and vans. By way of example, a truck may back into a loading bay such that the bumpers of the loading bay contact the bumpers on the trailer and a gap is created between the loading bay and the truck. A dock leveler or dock plate bridges the gap between the truck and a warehouse to provide a fixed and substantially level surface. Power moving equipment, such as forklifts or conveyor belts, is then utilized to transport the cargo from the warehouse to the truck. Human labor is then employed to stack the cargo in the truck. This is particularly true of the unloading of product, such as boxes or cases, from a truck, or freight container, for example. These systems are designed to maximize the amount the cargo unloaded while minimizing the use of human labor to both protect and extend the life of the workforce. Reducing human labor, however, has proven difficult as the configuration and size of the boxes in the truck or freight container cannot be easily predicted in advance. Therefore, a need still exists for improved truck unloading systems that further reduce the use of human labor when unloading or unpacking product, such as cases and boxes, from trailers and containers.
It would be advantageous to achieve a system and method for automated unloading and unpacking of product, such as cases and boxes, that would enable a trailer or container to be fully unloaded using minimal or no human labor, thereby minimizing the time to unload the truck and the need for human capital. It would also be desirable to enable a robotic computer-based solution that would address this problem, and particularly issues with perception, by unloading and unstacking trailers and containers with boxes and cases of varying sizes. To better address one or more of these concerns, in one embodiment, an automated truck unloader for unloading/unpacking a plurality of product is disclosed. A mobile base structure includes ends and a drive subassembly coupled to the mobile base. The drive subassembly includes multiple for steering and driving the mobile base. A conveyance subassembly is disposed on the mobile base and includes a powered transportation path operable for transporting product between the ends.
An industrial robot is disposed on the mobile base to provide selective articulated movement of an end effector between the powered transportation path and a reachable space such that the industrial robot is operable to handle the product in the reachable space. A visual detection subsystem, which includes a camera, is associated with the industrial robot. In one embodiment, a distance measurement subassembly is provided to determine presence of objects within a detection space, wherein the detection space and the reachable space at least partially overlap. A control subassembly is mounted to the mobile base structure in communication with the drive subassembly, the industrial robot, the camera, and the distance measurement subassembly, in order to coordinate the selective articulated movement of the industrial robot and the activation of the drive subassembly based upon the distance measurement subassembly detecting objects within the detection space, and dimensions of the trailer provided to the control subassembly.
The control assembly includes a memory accessible to a processor and the memory includes processor-executable instructions that, when executed cause the processor to construct a 3-D point cloud model from a plurality of first data images collected by the camera. The 3-D point cloud model is a representation of a physical environment, which includes the product, of the industrial robot. The instructions also cause the process to transform the 3-D point cloud model into a 3-D voxel model and then specify a search operation within the 3-D voxel model to identify a candidate product corner, which may belong to a candidate product of the plurality of product. The processor-executable instructions further include an instruction for the camera to capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner. A dimensioning operation is specified that, based on the at least one second data image, dimensions the candidate product about the candidate product corner. Further instructions that are executed include a removal operation to unload the candidate product and calculate instructions for removing the candidate product. In a further embodiment, a method for unloading/unpacking a product is provided as is a non-transitory computer readable medium having the aforementioned instructions stored thereon.
In another embodiment, an automated truck unloader for unloading/unpacking a plurality of product is disclosed. An industrial robot, which includes a camera, is disposed configured to provide selective articulated movement of an end effector between a powered transportation path and a reachable space such that the industrial robot is operable to handle the product in the reachable space. A control subassembly is located in communication with the industrial robot and the camera, the control subassembly coordinating the selective articulated movement of the industrial robot. The control assembly includes a memory accessible to a processor and the memory including processor-executable instructions that, when executed cause the processor to construct a 3-D point cloud model from multiple first data images collected by the camera. The instructions cause the processor to then transform the 3-D point cloud model into a 3-D voxel model and specify a search operation within the 3-D voxel model to identify a candidate product corner, the candidate product corner belonging to a candidate product of the plurality of product. An instruction for the camera is provided that causes the processor to capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner. The processor then calculates, based on the at least one second data image, instructions for removing the candidate product, and updates the 3-D voxel model with the at least one second data image.
In a further embodiment, an automated truck unloader for unloading/unpacking a plurality of product is disclosed that includes an industrial robot, including a camera, is configured to provide selective articulated movement of an end effector between a powered transportation path and a reachable space such that the industrial robot is operable to handle the product in the reachable space. A control assembly includes a memory accessible to a processor and the memory including processor-executable instructions that, when executed cause the processor to construct a 3-D point cloud model from a plurality of first data images collected by the camera prior to transforming the 3-D point cloud model into a 3-D voxel model. The processor-executable instructions cause the processor to identify segmented planes within the 3-D voxel model using connected component plane segmentation, identify sufficiently sized segmented planes by fitting a minimum area bounding rectangle to each set of inliers for the previously identified segmented planes, and identify a candidate product by selecting one of the sufficiently sized segmented planes with closed edge point to forecasted box corner dimensions. The processor then calculates instructions for removing the candidate product.
These systems and methodologies utilizing the present automated truck unloader therefore maximize the amount the product and cargo unloaded while minimizing the use of human labor to both protect and extend the life of the workforce. These and other aspects of the invention will be apparent from and elucidated with reference to the embodiments described hereinafter.
For a more complete understanding of the features and advantages of the present invention, reference is now made to the detailed description of the invention along with the accompanying figures in which corresponding numerals in the different figures refer to corresponding parts and in which:
While the making and using of various embodiments of the present invention are discussed in detail below, it should be appreciated that the present invention provides many applicable inventive concepts, which can be embodied in a wide variety of specific contexts. The specific embodiments discussed herein are merely illustrative of specific ways to make and use the invention, and do not delimit the scope of the present invention.
Referring initially to
As will be described in further detail hereinbelow, under the supervision of distance measurement subassembly or subassemblies that are components of the automated truck unloader 10, the automated truck unloader 10 maneuvers and drives automatically into the trailer 16 to a position as proximate as possible to the front wall 18. It should be appreciated that although an operator is not depicted as operating the automated truck unloader 10, an operator may be at location 40, an operator platform, although unnecessary. The automated truck unloader 10 operates independently of an operator and an operator is only necessary for certain types of troubleshooting, maintenance, and the like. A telescoping conveyor unit 42 is connected to the automated truck unloader 10. A stream of product 46, in the form standard cases or boxes 46A-46H, which may be of any dimension, is being supplied by the automated truck unloader upon removal thereof, as shown by arrow 48. In particular, the automated truck unloader 10 has already unloaded boxes 46A through 46E, and others, for example, at the intersection of the product 46 proximate the front wall 18 and the floor 22. As shown, the automated truck unloader 10 is unloading box 46f, which will be followed by boxes 46g, 46h and other boxes 46. The automated truck unloader 10 alternates between unloading the product 46 and driving forward to create more opportunities to grip the product 46 between the front wall 18 and the automated truck unloader 10 until the trailer 16 is at least partially unloaded of product 46.
The drive subassembly 52 is coupled to the undercarriage 74 of the mobile base 50 to provide mobility. As will be discussed in further detail hereinbelow, drive wheel assemblies 78, 80, are disposed on the undercarriage 74 proximate to the sides 70, 68 respectively. A universal wheel assembly 82 is disposed on the undercarriage 74 more proximate to the rear end 66 and centered between the sides 68, 70, respectively. In combination, wheel assemblies 78, 80, 82 provide forward and reverse drive and steering. Retractable stabilization assemblies 84, 86 are also disposed on the undercarriage 74 proximate to the intersection of the end 64 and side 68, the intersection of end 66 and the side 70, respectively. As alluded to, in a forward or reverse drive and steering operation, such as moving into or out of the trailer 16, drive wheel assemblies 78, 80 and the universal wheel assembly 82 are actuated and in contact with the deck 38 of the loading dock 32 while the retractable stabilization assemblies 84, 86 are withdrawn from contact with the deck 38 in a position close to the undercarriage 74. On the other hand, when the automated truck unloader 10 is conducting a product loading or unloading operation, such as during the use of the industrial robot 56, the retractable stabilization assemblies 84, 86 are positioned in contact with the deck 38 to anchor the automated truck unloader 10. It should be appreciated that although the automated truck unloader 10 is being described relative to unloading and unpacking, the automated truck unloader 10 may also be used to load and pack product, including boxes and cases, into a trailer.
The conveyance subassembly 54 is disposed on the surface 72 of the mobile base 50 to provide a powered transportation path 88 operable for measuring, separating, carrying, and stacking, as required by the application and job assignment of the automated truck unloader 10, boxes from the rear end 66 to the front end 64 proximate to the industrial robot 56. As shown, the powered transportation path 88 includes a powered roller conveyor having roller elements 92 which transfer the product or boxes 46 from the industrial robot 56 to a downstream location. The conveyance subassembly 54 as well as the telescoping conveyor unit 42 may also each be equipped with a series of end stop photo eyes to adjust the rate of automatic flow of product through the telescoping conveyor unit 42 and the conveyance subassembly 54. Such an implementation provides a steady and continuous flow of product, maintains proper box or product separation, and prevents unnecessary gaps between the product and product backups and jams.
The industrial robot 56 is disposed at the front end 64 and adapted to provide selective articulated movement of an end effector 130 between the landing platform 94 of the powered transportation path 88 and a reachable space 132 such that the industrial robot 56 is operable to place the product 46 in the reachable space 132. The end effector 130 includes a gripper arm 134 adapted for manipulating product with cooperating and complementary grapplers 136A, 136B. It should be appreciated that any type of end effector 130 may be employed the industrial robot and the choice of end effector 130 will depend upon the product 46 and specific automated truck unloader 10 application. By way of example, the gripper arm 134 with grapplers 136A, 138B is preferred for unloading and unpacking boxes 46A-46H. It should be understood, however, that the product 46 may be any type of good such as other cased or non-cased objects requiring loading.
In one implementation, the industrial robot 56 includes seven segments 130, 138, 140, 142, 144, 146, 148 joined by six joints 150, 152, 154, 156, 158, 160 to furnish selective articulated movement having six degrees of freedom. More particularly, the referenced reachable space 132, as best seen in
The positioning subassembly 58 is dispersed throughout the mobile base 50. A distance measurement subassembly 170 disposed at the front end 64 of the mobile base 50 measures distance and determines the presence of objects within a detection space which is located in front of the front end 64. In one embodiment, the detection space and the reachable space 132 at least partially overlap. The distance measurement subassembly 170 assists the automated truck unloader 10 with forward and reverse movement and the repositioning of the automatic case loader 10 to create additional empty reachable space 132 for the placement of the product 46. Further, the distance measurement subassembly 170 assists with the coordination and operation of the industrial robot 56. Distance and measurement information gathered by the distance measurement subassembly 170 is provided to the control subassembly 62.
The distance measurement subassembly 170 may be a laser range finding apparatus operating on a time-of-flight measurement basis or principle or a camera system operating on an adaptive depth principle. It should be appreciated, however, that other types of distance measurements are within the teachings of the present invention. By way of example, and not by way of limitation, the distance measurement subassembly may include a laser range finding apparatuses, cameras, ultrasonic measurement apparatuses, inclinometers, and combinations thereof. Similar to distance measurement subassembly 170, distance measurement subassemblies 174, 176 are respectively disposed at the sides 68, 70. The distance measurement subassemblies 174, 176 each may include in one embodiment, detection spaces (not illustrated) to provide measurement and distance information to the control subassembly 62 during traverse movement operations of the automated truck unloader 10.
The safety subsystem 60 is distributed and mounted to the mobile base 50. The safety subsystem 60 may include a light tower which provides a quick indication of the current status of the automated truck unloader 10 to an operator and a wireless operator alert system which contacts pagers or cellular devices of individuals through a wireless network. Also a cage and railing may be included around the operator platform 40 to provide additional safety to the operator. Emergency buttons may be located throughout the automated truck unloader 10 to provide for instant and immediate power down. Front safety scanners 188 and rear safety scanners 190 may be positioned at the front end 64 and the rear end 64 to protect the automated truck unloader 10, people, and product during a collision with an obstacle. Additionally, the front safety bumpers 188 and the rear safety bumpers 190 may include detectors that detect the presence of an object and cause an automatic power down during a collision. Side safety scanners, although not illustrated, may also be utilized. It should be appreciated that other safety features, such as safety bumpers, may be integrated into the automated truck unloader 10.
The control subassembly 62, which is also distributed and mounted to the mobile base 50, may include control station having a user interface disposed at the side 70 near the operator platform 76. As discussed, the drive subassembly 52, the conveyance subassembly 54, the industrial robot 56, the positioning subassembly 58, and the safety subassembly 60 are interconnected and in communication with the control subassembly via a network of concealed and sheathed cables and wires. With this arrangement, the control subassembly 62 may coordinate the manual and automatic operation of the automated truck unloader 10. Further, a visual detection subsystem 162 is associated with the end effector such that the visual detection subsystem 162 captures an image of a product space for processing by the control subassembly, as will be discussed further hereinbelow.
A main frame 200 is constructed of welded steel tubing includes tubular sections which provide a rectangular framework. With reference to the operation of the drive subassembly 52 in conjunction with the mobile base 50, the drive wheel assemblies 78, 80 and universal wheel assembly 82 provide mobility along the length of the automated truck unloader 10. In addition to providing forward and reverse capability, the one embodiment, the drive subassembly 52 may furnish a traverse drive system providing the capability to move the entire automated truck unloader 10 perpendicular to a trailer or fixed object at the loading dock 32.
Referring now to
Multiple suction cups 302 are associated with a face 303 of the main frame 280 and, similarly, multiple suction cups 304 are associated with a face 305 of the moving frame half 282. Bulk head fittings 306 secure the multiple suction cups 304 to the moving frame half 282. Within the main frame 280, a paddle cylinder 308, straight fitting 310, and an elbow fitting 312 secure vacuum manifolds 314, 316 within the main frame 280 in pneumatic communication with the suction cups 302, 304. Vacuum sensors 318, 320 are mounted in the main frame 280 proximate to the suction cups 302. A visual detection subsystem 162 is housed in the main frame and includes a camera assembly 322, lighting assembly 324, and a camera mount 326.
Referring now to
The processor-executable instructions further include an instruction for the camera to capture at least one second data image of a local area of the physical environment corresponding to the candidate product corner. Then, the processor calculates, based on the at least one second data image, instructions for removing the candidate product. More specifically, a dimensioning operation is specified that, based on the at least one second data image, dimensions the candidate product about the candidate product corner. Further instructions that are executed include a removal operation to unload the candidate product and calculate instructions for removing the candidate product. Lastly, the processor-executable instructions update the 3-D voxel model with the at least one second data image. These instructions may iteratively continue.
The user interface 194 provides user control and interaction with the automated truck unloader 10. The user interface 194 may utilize icons in conjunction with labels and/or text to provide navigation and a full representation of the information and actions available to the operator. In addition to loading operations, user interactions may be related to maintenance, repair and other routine actions which keep the automated truck unloader 10 in working order or prevent trouble from arising.
The operational data environment database 370 includes data about the reachable space 132 of the industrial robot 56, stacking methodology data, product information as well as information about the standard sizes of trailers. The product information may be stored in the operational data environment database 350, gathered by the conveyance subassembly 54 as previously discussed, or gained by a combination thereof. By having the standard sizes of trailers pre-loaded, operator time is saved from having to enter this data and performance of the automated truck unloader 10 is improved with this additional information. By way of example, Tables I & II present exemplary examples of type of trailer data that the automatic ruck unloader 10 may utilize in determining position and product placement. It should be appreciated, however, that the automated truck unloader presented herein may operate within pre-loaded information such as the trailer data presented in Tables I & II.
The programmable logic controller 372 coordinates overall operation and switches between various modes of operation including manual and automatic. The programmable logic controller 372 also provides for the high-level calculation and coordination required during automatic operation for items such as the current unload quadrant unloading and steering angel calculations during automatic navigation.
The robot controller 374 controls the motions of the industrial robot 56 through built in inputs and outputs wired through the industrial robot 56 and the end effector 130. It should be appreciated that although a particular architecture is presented for the control of the automatic case loader, other architectures are within the teachings of the present invention. By way of example, any combination of hardware, software, and firmware may be employed. By way of further example, the distribution of control may differ from that presented herein.
In one operation embodiment, the programmable logic controller 372 accesses the dimensions of the trailer 16 from the operational environment database 372. The operator 40 has indicated through the user interface 194 which type of trailer has arrived at the docking bay 30. Alternatively, the distance measurement subassembly 170 is operable to detect this information. The distance measurement subassemblies 170, 174, 176 relay distance and position data to the programmable logic controller 352 which uses this information to send control signals to the robot controller 374, the drive subassembly 52, the controller 372, and the safety controller 378. Additionally, the programmable logic controller 372 receives control signals, which are inputs into the behavior process, from each of these components. Constant updates and status information are provided to the operator 40 by the programmable logic controller 352 through the user interface 194.
According to one embodiment, between the driver 384 and the control program 380 is provided an independent data processing layer in the form of a frame program 386, which controls the robot movements, and a unit 388 for automated or event-controlled strategy or behavioral selection on the basis of the states and signals which occur. User application programs, event-controlled strategy selections and sensor programs in the frame program 386 can be programmed by the operator 40 and directed by a robot program 390, which monitors the balance and implementation of manual and automatic control of the industrial robot 56.
A homogenous transformation matrix is defined from the camera to tool coordinate systems, ctH, and from the tool to robot coordinate systems, rtH. The manipulator provides rtH using encoder feedback on the manipulator joints, and ctH is may be obtained using a hand-eye calibration procedure. This may be performed offline when the camera is installed on the system, and the resulting transformation is stored for future use. A point originally expressed in the camera coordinate system, cp, can then transformed to the robot base using Equation 1.
rp=ctHtrHcp
At block 462, plane segmentation occurs, wherein after the points in the input point cloud are transformed into the robot base, planes are segmented using connected component based plane segmentation. This approach takes advantage of the pixel based organizational structure of RGB-D data to efficiently segment multiple planes in a single image. The segmented plane models can then be represented as:
Pi={ai,bi,ci,di}
where ai, bi, ci, and di are the coefficients of the ith plane when expressed in Hessian normal form ax+by +cz+d=0. Additionally, the jth plane inlier and kth edge point of the ith plane are defined as:
Pi,jinliner={xi,j,yi,j,zi,j}
Pi,kedge={xi,k,yi,k,zi,k}
Once the sets P, pinlier, and pedge, are acquired, each segmented plane is evaluated for target box face candidacy.
At block 464, plane boundary and elimination occurs wherein a fitting of a minimum area bounding rectangle to each set of inliers for the previously segmented planes occurs. The purpose of this step is to aid the target plane selection process by identifying planes that cover an insufficient area (i.e., the plane is too small to contain the smallest possible box face that we wish to detect). In one embodiment, the method of rotating calipers may be used to compute the minimum area bounding rectangles in linear time, after which we remove from consideration any plane with a bounding rectangle side length less than the smallest possible box dimension. For purposes of this step, the lth bounding rectangle coordinate of the ith plane in clockwise order are defined as:
pi,lrectangle={xi,l,yi,l,zi,l}
The smallest bounding rectangle side si can then be computed using:
si=min{∥Pi,1rectangle−Pi,2rectangle∥,∥Pi,2rectangle−Pi,3rectangle∥}
In addition to eliminating planes of insufficient size, also the methodology may choose to eliminate planes with an orientation that deviates more than
radians from an ideal target orientation. This target orientation depends on the camera pose, since we expect to see mostly ‘top’ box faces when looking from above and ‘front’ box faces when looking forward toward the stacked boxes. In the case when the camera is looking down upon potential target boxes, the resulting image is expected to contain the top box face with a normal vector pointing in the general direction of the floor bounding plane normal (i.e., in an upward direction). Conversely, when the camera is looking forward toward potential target box, the image is expected to contain the front box face with a normal pointing in the general direction of the rear bounding plane (i.e., pointing straight out of the container). In both cases, removed from further consideration are any planes that sufficiently deviate from the ideal normal. The angular deviation of plane Pi from the target plane normal Ni is computed as:
Using Equations 6 and 7, and the minimum box side length Smin, the final target plane candidacy is computed with the function:
At block 466, target plane selection occurs where once a set of candidate box face planes has been identified using the preceding steps, a final target plane and its corresponding set of inliers is chosen for further processing. In one embodiment of this step, the euclidean distance of the forecasted box corner location to each of the planar edge points is computed. Of the set of candidate planes not eliminated in the previous steps, the final target plane is chosen to be that which contains the closest edge point to the forecasted box corner. Once the target face plane is selected, we are able to proceed with box vertex detection and orientation estimation. At block 468, vertex segmentation occurs and the final product detection is at block 470.
At block 490, from the remaining candidate corners, the minimum distance to the left wall is determined. Candidate corners are kept that are within a certain distance. At block 492, the bounding cuboid of the remaining corner candidate is determined. The candidate corner selected has the smallest Euclidian distance to the top left corner of the bounding cuboid.
Returning to block 494, if the highest corner candidate is not in the top area threshold, then from the remaining corners the minimum distance to the front plane is determined and corners are kept that are within the maximum distance of the given box or product dimensions. At block 496, the highest remaining corner candidates are kept and all other corner candidates are removed that are more than a threshold away, such as 2 inches (5.08 cm). At block 498, from the remaining candidate corners, the minimum distance to the left wall is determined. Candidate corners are kept that are within a certain distance. At block 500, the bounding cuboid of the remaining corner candidate is determined. The candidate corner selected has the smallest Euclidian distance to the top left corner of the bounding cuboid.
The order of execution or performance of the methods and process flows illustrated and described herein is not essential, unless otherwise specified. That is, elements of the methods and process flows may be performed in any order, unless otherwise specified, and that the methods may include more or less elements than those disclosed herein. For example, it is contemplated that executing or performing a particular element before, contemporaneously with, or after another element are all possible sequences of execution.
While this invention has been described with reference to illustrative embodiments, this description is not intended to be construed in a limiting sense. Various modifications and combinations of the illustrative embodiments as well as other embodiments of the invention, will be apparent to persons skilled in the art upon reference to the description. It is, therefore, intended that the appended claims encompass any such modifications or embodiments.
This application is a continuation of U.S. patent application Ser. No. 16/506,710 entitled “Perception-Based Robotic Manipulation System and Method for Automated Truck Unloader that Unloads/Unpacks Product from Trailers and Containers” filed on Jul. 9, 2019, in the names of Christopher D. McMurrough et al., now U.S. Pat. No. 10,633,202, issued on Apr. 28, 2020; which is a continuation of U.S. patent application Ser. No. 15/949,872 entitled “Perception-Based Robotic Manipulation System and Method for Automated Truck Unloader that Unloads/Unpacks Product from Trailers and Containers” filed on Apr. 10, 2018, in the names of Christopher D. McMurrough et al., now U.S. Pat. No. 10,343,858, issued on Jul. 9, 2019; which is a continuation of U.S. patent application Ser. No. 15/516,277 entitled “Perception-Based Robotic Manipulation System and Method for Automated Truck Unloader that Unloads/Unpacks Product from Trailers and Containers” filed on Mar. 31, 2017, in the names of Christopher D. McMurrough et al., now U.S. Pat. No. 9,938,092, issued on Apr. 10, 2018; which is a 371 of Application No. PCT/US2015/054079 entitled “Perception-Based Robotic Manipulation System and Method for Automated Truck Unloader that Unloads/Unpacks Product from Trailers” and filed on Oct. 5, 2015, in the names of Christopher D. McMurrough et al.; which claims priority from U.S. Patent Application Ser. No. 62/059,515 entitled “3D Perception-based Robotic Manipulation System for Automated Truck Unloading” and filed on Oct. 3, 2014, in the names of Christopher D McMurrough et al.; all of which are hereby incorporated by reference, in entirety, for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
7317456 | Lee | Aug 2008 | B1 |
8469261 | Bonner et al. | Jun 2013 | B2 |
8847954 | Lin | Sep 2014 | B1 |
9102055 | Konolige | Aug 2015 | B1 |
9589353 | Halata | Mar 2017 | B2 |
9746852 | Watts | Aug 2017 | B1 |
9754413 | Gray | Sep 2017 | B1 |
9938092 | McMurrough | Apr 2018 | B2 |
10343858 | McMurrough et al. | Jul 2019 | B2 |
10455212 | Konolige | Oct 2019 | B1 |
10633202 | McMurrough | Apr 2020 | B2 |
20050151734 | Gubkin | Jul 2005 | A1 |
20090110522 | Criswell | Apr 2009 | A1 |
20120194516 | Newcombe | Aug 2012 | A1 |
20130312371 | Ambrose et al. | Nov 2013 | A1 |
20140002597 | Taguchi | Jan 2014 | A1 |
20140184749 | Hilliges | Jul 2014 | A1 |
20140205403 | Criswell | Jul 2014 | A1 |
20160016311 | Konolige | Jan 2016 | A1 |
20160063309 | Konolige | Mar 2016 | A1 |
20160132059 | Mason | May 2016 | A1 |
20160288330 | Konolige | Oct 2016 | A1 |
20170116779 | Lalish | Apr 2017 | A1 |
20170278298 | Hurter | Sep 2017 | A1 |
Number | Date | Country |
---|---|---|
2011043674 | Apr 2011 | WO |
Entry |
---|
International Search Report, PCT/US2015/054079, dated Dec. 22, 2015. |
Supplementary European Search Report, Application No. 15847944.4-1017/3201112 PCT/US2015/054079, dated Aug. 31, 2018. |
Number | Date | Country | |
---|---|---|---|
20200324986 A1 | Oct 2020 | US |
Number | Date | Country | |
---|---|---|---|
62059515 | Oct 2014 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16506710 | Jul 2019 | US |
Child | 16860731 | US | |
Parent | 15949872 | Apr 2018 | US |
Child | 16506710 | US | |
Parent | 15516277 | US | |
Child | 15949872 | US |