Patent Application
20240109203
Embodiments of the present invention are in the field of robotic systems that use artificial intelligence, computer vision, and/or mechanical systems to manipulate (e.g., pick, sort, and place) objects, and pertain particularly to tool changers for robotic systems.
The statements in the background of the invention are provided to assist with understanding the invention and its applications and uses, and may not constitute prior art.
There are several approaches that have been used to change end effector tools for robotic systems to pick objects of varying dimensions, weights, materials, and levels of fragility. However, many of these approaches either require human intervention, are too slow, do not retain the new tool firmly, or are unreliable. For example, some conventional tool changers have two or more magnets that are magnetized and are axially symmetric. Such a tool changer has a low radial strength, and the tool portion can become unintentionally disconnected from the robotic arm portion with even a minor transverse force. Such a tool changer can also easily rotate, even when such rotation is undesired, and there is no control over the axial rotation.
Therefore, it would be an advancement in the state of the art to provide an end effector tool changer for various robotic systems (e.g., pick, sort, and place robotic systems) that is physically compact, mechanically robust, has high radial strength, prevents undesirable rotations, uses minimal sensory input, is suitable for a wide variety of tools, and operates quickly.
It is against this background that the present invention was developed.
The present invention relates to an end effector tool changer for various robotic systems, such as pick, sort, and place robotic systems.
More specifically, in various embodiments, the present invention is an end effector tool changer device for a robotic system, including a robotic arm attachment portion, including a first magnetic part and a plurality of first engagement parts; and a tool attachment portion, including a second magnetic part and a plurality of second engagement parts; where the plurality of first engagement parts is configured to engage with the plurality of second engagement parts, where each of the plurality of first engagement parts and each of the plurality of second engagement parts are selected from the group consisting of a pin and a pin groove, and where the first magnetic part spatially and magnetically corresponds to the second magnetic part.
In some embodiments, a first engagement part of the plurality of first engagement parts comprises a pin with a first cross-sectional shape, a second engagement part of the plurality of second engagement parts comprises a pin groove with a second cross-sectional shape, and the first cross-sectional shape and the second cross-sectional shape are configured to prevent the robotic arm attachment portion and the tool attachment portion from rotating relative to each other when the first engagement part and the second engagement part are engaged.
In some embodiments, at least one of the plurality of first engagement parts includes a pin with a tapered tip.
In some embodiments, at least one of the plurality of second engagement parts includes a beveled pin groove.
In some embodiments, the robotic arm attachment portion and the plurality of first engagement parts are constructed as a single piece.
In some embodiments, the tool attachment portion and the plurality of second engagement parts are constructed as a single piece.
In some embodiments, the tool attachment portion further includes a plurality of rack grooves.
In some embodiments, one or more of the plurality of rack grooves are beveled.
In some embodiments, the plurality of rack grooves is a pair of rack grooves.
In some embodiments, the end effector tool changer device includes a tool plate comprising a slot, where dimensions of the slot correspond to the plurality of rack grooves.
In some embodiments, the slot is tapered.
In some embodiments, the robotic arm attachment portion further comprises a robotic arm attachment portion through-hole and the tool attachment portion further comprises a tool attachment portion through-hole.
In some embodiments, the robotic arm attachment portion through-hole is adjacent to a leakage prevention device.
In some embodiments, the robotic system is configured for object manipulation.
Other aspects and embodiments of the present invention include the methods and processes including the steps described herein, and also include the processes and modes of operation of the systems and devices described herein. Yet other aspects and embodiments of the present invention will become apparent from the detailed description of the invention when read in conjunction with the attached drawings.
Embodiments of the present invention described herein are exemplary, and not restrictive. Embodiments will now be described, by way of examples, with reference to the accompanying drawings, in which:
With reference to the figures provided, embodiments of the present invention are now described in detail. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention can be practiced without these specific details. In other instances, structures, devices, activities, and methods are shown using schematics, use cases, and/or flow diagrams in order to avoid obscuring the invention. Although the following description contains many specifics for the purposes of illustration, anyone skilled in the art will appreciate that many variations and/or alterations to suggested details are within the scope of the present invention. Similarly, although many of the features of the present invention are described in terms of each other, or in conjunction with each other, one skilled in the art will appreciate that many of these features can be provided independently of other features. Accordingly, this description of the invention is set forth without any loss of generality to, and without imposing limitations upon, the invention.
There are several approaches that have been used to change end effector tools for a sort and place robotic system to pick objects of varying dimensions, weights, materials, and levels of fragility. However, many of these approaches either require human intervention, are too slow, do not retain the new tool firmly, or are unreliable. For example, some conventional tool changers have two or more magnets that are magnetized and are axially symmetric. Such a tool changer has a low radial strength, and the tool changer can become unintentionally disconnected from the robotic arm with even a minor transverse force. Such a tool changer can also easily rotate, even when such rotation is undesired, and there is no control over the axial rotation.
Therefore, it would be an advancement in the state of the art to provide an end effector tool changer for a robotic system that is mechanically robust, has high radial strength, prevents undesirable rotations, uses minimal sensory input, is suitable for a wide variety of tools, and operates quickly.
Context of the End Effector Tool Changer within a Robotic System
In some embodiments, robotic system 100 includes an end effector, which further includes a tool that manipulates objects. In some embodiments, the end effector and the tool remain stationary relative to the robotic system while an object's position is controlled. For example, the robotic system may alter the slope or other characteristics of the surface on which the object is sitting to slide it into a stationary tool, such as a gripper or a suction.
In other embodiments, the end effector and the tool are attached to a motion device, such as a moveable robotic arm 102. The tool may include, for example, a knife that may slice or poke an object, a key that may open a lock, or a center punch that may mark a plate. In some embodiments, the tool is useful in a pick, sort, and place robotic system. Such tools may include grippers and suctions. Although the description below discloses tools and related systems for pick, sort, and place robotic systems, embodiments useful for other types of tools are readily apparent to those skilled in the art.
Pick, sort, and place robotic system 100 includes a robotic arm 102, various input and output components and structures such as a sorting stand 150 and a receptacle stand 180. An operator 101 may supervise or assist the robotic arm (see
Pick, sort, and place robotic system 100 also includes a control system 170 to monitor and manage robot motion. The control system provides instructions and/or command signals for moving (e.g., rotating, extending, retracting) the various components of robotic arm 102. The control system 170 includes a processor 171, memory 172 (e.g., a non-transitory computer-readable storage medium), data links 173, communications interfaces, and other components. The control system may also include an optional cloud component 174 with processors 175 and databases 176 accessible over a local or remote network (e.g., Internet).
Pick, sort, and place robotic system 100 also includes a vision system with a vision processor 169, sensor devices 160, and other components. Each sensor device 160 may have one or more cameras 162, a variety of sensors 163 (e.g., image, depth, visible light, and/or infrared sensors), barcode readers 164, or other components. In some cases, cameras 162 capture image data that includes visible light data (e.g., RGB data) and/or depth information (e.g., how far objects in the image are from the camera). The captured image data is sent to the control system for processing. The vision system can have any number of sensors and cameras. Its components can be supported by any robotic, input/output component or structure, and be located in other locations.
Pick, sort, and place robotic system 100 also includes a motion controller 177. The vision processor 169 and motion controller 177 may be external or located within the control system 170.
Pick, sort, and place robotic system 100 may also include a light curtain system that includes multiple sensors 165 generating a light curtain 166. Pick, sort, and place robotic system 100 may also include a LIDAR 167. The light curtain system and the LIDAR may be used either for safety purposes (e.g., monitor human movement around the robotic system) or for operations (e.g., detect the movement of objects or system components). Pick, sort, and place robotic system 100 may also include lighting devices 168 that can be dimmed depending on tote color or other environmental and operational factors.
Robotic arm 102 includes base 104 for mounting to a support surface (e.g., the floor or some other support structure). Frame 106 is rotatably connected to base 104. Lower arm 108 is rotatably connected to frame 110. Upper arm 112 is rotatably connected to lower arm 108. End effector 114 is rotatably connected to upper arm 112. End effector 114 includes one or more tools 116 as well as a tube 115. The end effector 114 and each tool 116 have tool changer 117 parts allowing various tools to be compatible with the end effector 114. A tool rack 118 is used for storing and accessing the various tools. Each tool slot on the tool rack 118 has a tool sensor 119 to indicate the presence or absence of a tool.
Pick, sort, and place robotic system 100 may also include a vacuum source 120 (e.g., pump) or compressed air source 121 to provide the pressure necessary to use the tools, where vacuum denotes negative pressure and compressed air denotes positive pressure. Each source is controlled by a source switch 122 operable by the control system. A source selection switch 123 allows the control system to select the adequate source to operate the tool that is currently in use. A hose 124 runs through the robotic arm from the end effector to the source. A valve 125 allows the control system to select a pressure source, or to connect the hose to the atmosphere (i.e., no positive or negative pressure applied). A pressure sensor 126 allows the control system to monitor the pressure level within the hose. A component of a tool changer 127 permits efficient and reliable tool-changing. A weight sensor 131 located on the end effector allows the control system to monitor the weight of the tool and its load (see
All components of the control system 170 and vision system (e.g., cameras and sensors), are connected through data links 173. Furthermore, all components of the robotic system involved in motion or monitoring (e.g., motion controller 177, pump/selection switches 122/123, valve 125, pressure sensor 126, tool sensors 119, lighting devices 168) have data links 173 to the control system 170.
When the robotic system picks up a load using an end effector 114 attached with a load cell, the load cell detects the load's weight, by acting as a transducer that converts the gravitational force of the load, i.e., its weight, applied onto the load cell into an electrical output that the robotic system collects. However, most commercially available load cells are designed to handle and measure force in one dimension only. When the load cell is subjected to forces in other directions, the resulting torsion or bending may damage the load cell and distort the weight measurement.
When the robotic system identifies a load to be picked up, it may also determine a particular end effector tool among a selection of tools that is most appropriate for picking the load. However, in some cases, the robotic system may need to frequently change tools between objects. In such cases, there is a great need for a tool changer that operates automatically without human intervention. Current solutions include end effectors with magnetically coupled components that attach and detach seamlessly. However, some such devices are axially magnetized and have low radial strength. They may also rotate easily, making the system difficult to use and control. Moreover, magnetic couplings may not be sufficiently robust: tools may fall off due to collision or sudden movements. A new solution is to add mechanical pins to the magnetically coupled components in order to maintain the orientation of the two components and to add radial strength. The end effector tool changer described in this disclosure includes a system designed for such a purpose.
Interaction between System Components of a Robotic System
In some embodiments, the pick and place robotic system includes a robotic arm with an end effector configured to have an attached tool at its distal end; a tool changing device; a tool rack; a vision system; and a control system. The tool changing device further includes a robotic arm attachment portion and a tool attachment portion. The tool rack includes one or more tool plates and a plurality of tools. The control system includes a processor, a non-transitory computer-readable storage medium, and a plurality of communication interfaces.
In some embodiments, at least one tool includes a tool attachment portion of the tool changing device at its proximal end. The robotic arm attachment portion is configured to attach to the tool attachment portion, and at least one tool plate of the one or more tool plates includes a tool slot.
In some embodiments, each tool plate has exactly one tool slot. In other embodiments, a tool plate may have more than one tool slot. In one embodiment, the tool rack further includes one or more sensors associated with a tool slot, where the one or more sensors are configured to indicate the presence of a tool in the tool slot.
In one embodiment, the control system receives data from each of the sensors on the tool rack and can determine, at any time, whether a given tool is located at its slot in the tool rack.
In yet another embodiment, the tool attachment portion further includes a plurality of grooves, and the plurality of grooves spatially corresponds to a tool slot on a tool plate.
The grooves enable tools to slide into the tool rack and to be retrieved from the tool rack in a robust and timely manner.
In one embodiment, the pick and place robotic system further includes a weight sensor at the distal end of the end effector, where the weight sensor is configured to measure the weight of an attached tool and its load (e.g., one or more objects).
A weight sensor may allow the control system to detect the presence or absence of a tool, the number of objects carried by it.
In one embodiment, the pick and place robotic system further includes an electric circuit, where the electric circuit is configured to indicate the presence of a tool attached to the end effector. In one embodiment, the electric circuit is a tool wire.
A tool wire may be configured to be in contact with a tool when a tool is attached to the end effector so as to convey to the control system whether a tool is attached. In one embodiment, the presence of a tool is determined electrically (e.g., through detecting a change in wire impedance, current intensity, voltage, etc.).
In one embodiment, the tool wire is run from the tool, through the pass-through end effector tube, to a proximal part of the robotic arm such as its base or its frame, where the information is conveyed to the control system through a data link.
In some embodiments, the pick and place robotic system further includes a plurality of input and output components, where at least one output component corresponds to an object type, and where the plurality of input and output components are selected from the group consisting of a sorting stand, a tote, a receptacle stand, a bin, a tote conveyor, an object conveyor, a put wall, an automated guided vehicle (AGV), and a shelf.
Objects may be classified by type. Object types may involve their shape (e.g., round vs. elongated objects), the material they are made of (e.g., plastic vs. metal objects), their color, or their nature (e.g., fruits vs. vegetables, apples vs. oranges). In one embodiment, objects having the same barcode or the same destination (e.g., shipping address, destination department in an office or plant, etc.) belong to the same object type. In one embodiment, objects belonging to the same order (e.g., they have the same order number) belong to the same object type. In one embodiment, each of the various output components (e.g., the bins in a sorting stand) are associated with distinct object types.
In some embodiments, the robotic arm and tool attachment portions further include a through-hole. In some embodiments, the through-hole transmits vacuum or compressed air between an attached tool and its corresponding source pump. In some embodiments, the through-hole includes a mechanical pass-through. In some embodiments, the through-hole includes an electrical pass-through. In further embodiments, the robotic arm and tool attachment portions include a plurality of through-holes, each of which may be any of the aforementioned types.
In one embodiment, the pick and place robotic system further includes a first hose, where the through-hole of the robotic arm attachment portion is connected to a distal end of the first hose.
In one embodiment, the pick and place robotic system further includes a pressure sensor, where the pressure sensor is located on the first hose. Data from the pressure sensor (e.g., a pressure reading) may indicate the presence or absence of an attached tool or a picked object.
In one embodiment, the pick and place robotic system further includes a source pump, where the source pump is connected to the proximal end of the first hose, and the source pump is selected from the group consisting of a vacuum pump and a compressed air pump.
In systems operating using a single source pump, the first hose is the hose 124, represented in
In another embodiment, the pick and place robotic system further includes a valve and one or more second hoses, where the valve connects the proximal end of the first hose to one valve output selected from the group consisting of the atmosphere and the one or more second hoses.
In one embodiment, the pick and place robotic system further includes one or more source pumps, where at least one of the one or more second hoses connects a valve output to one of the one or more source pumps, at least one tool of the plurality of tools corresponds to one of the one or more source pumps, and a source pump of the one or more source pumps is selected from the group consisting of a vacuum pump and a compressed air pump.
In systems operating using more than one source pump (e.g., one vacuum pump and one compressed air pump), a valve is required to switch between pumps or to connect the tool to the atmosphere (i.e., disconnect from all pumps). In this case, the first hose is the distal segment of the hose 124 shown in
In one embodiment, each source pump has a pump switch to activate it, and a pump selection switch is used by the control system to activate the required pump switch through data links or any other form of control signaling (e.g., an electrical ON/OFF signal).
In other embodiments, a fluid pump is used to control a tool. In this case, the through-hole, hoses, pressure sensors, and valve, are configured to operate with a fluid.
In one embodiment, the vision system includes a vision processor, a plurality of vision communication interfaces, and one or more vision components selected from the group consisting of a camera, a barcode reader, a depth sensor, an infrared sensor, a light curtain system, and a LIDAR; and where at least one component of the vision system is connected to the vision processor through a data link, and the vision processor is connected to the control system through a data link.
In one embodiment, the pick and place robotic system further includes a lighting source, where the lighting source is configured to emit multiple light intensities.
In one embodiment, the control system controls robotic arm movements through a motion controller. In one embodiment, the motion controller also controls the valve.
In one embodiment, data from the pressure sensor, the weight sensor, the tool wire, the vision system sensors, the tool sensors, or any other component with a communication interface, is transmitted at regular time intervals to the control system (i.e., a data push). In another embodiment, such data is transmitted only upon request from the control system (i.e., a data pull).
In one embodiment, the tool wire is configured to provide the control system with information on the presence of an attached tool at the end effector continuously and instantaneously, through an electrical signal.
The arm attachment portion 291 and the tool attachment portion 292 are mated or engaged through one or more engagement mechanisms, and are held together by a magnetic system. In some embodiments, the magnetic system includes a first magnetic part 207, which is associated with or is part of the arm attachment portion 291, and a second magnetic part 211, which is associated with or is part of the tool attachment portion 292. In some embodiments, the engagement mechanism includes a first engagement part 203, which is associated with or is part of the arm attachment portion 291, and a second engagement part 219, which is associated with or is part of the tool attachment portion 292. In some embodiments, the first engagement part of the engagement mechanism is or includes an indicator pin 203, as shown in
In some embodiments, the arm attachment 291 (also called the “upper assembly”) includes the first magnetic part 207 of the magnetic system and a main body 209 (also called an “upper main body”), which further includes a plurality of indicator pins 203. The first magnetic part 207 of the magnetic system corresponds, spatially and magnetically, with the second magnetic part 211 of the magnetic system, as described in greater detail with reference to
In some embodiments, the indicator pins 203 and the rest of the upper main body 209 are constructed together as a single piece, such as by molding, machining, or 3D printing. Some advantages of single-piece construction include the ease and speed of mass production and closer consistency within a batch of upper main bodies 209.
The arm attachment portion optionally includes a through-hole 205 to access the various capabilities of a tool 116. In some embodiments, the through-hole 205 is capable of maintaining a vacuum for a vacuum-driven tool 116. In some embodiments, the through-hole 205 is capable of carrying compressed air for a compressed air-driven tool 116. In other embodiments, the through-hole 205 is capable of carrying mechanical or electrical connections between the arm 102 and the tool 116, such as wires carrying signals. In some embodiments, the arm attachment portion 291 includes a plurality of through-holes 205, each of which may be any of the aforementioned types.
The locations, orientations, and magnetic strengths of the first magnetic part 207 of the magnetic system and of the second magnetic part 211 of the magnetic system provide sufficient force to hold a load 103 on a tool 116, but still enable the robotic system 100 to easily change tools 116 by sliding off the tool attachment portion 292 in accordance with methods described in this disclosure. If the magnetic force were too strong, for example, the robotic system 100 would be unable to remove the tool attachment portion 292. When the first magnetic part 207 of the magnetic system and the second magnetic part 211 of the magnetic system are firmly attracted to each other via a magnetic force, the magnetic system is considered to be engaged. Otherwise, the magnetic system is considered to be disengaged. In some embodiments, the first magnetic part 207 of the magnetic system is a magnet or a plurality of magnets and the second magnetic part 211 of the magnetic system is a magnet or a plurality of magnets. In some embodiments, the first magnetic part 207 of the magnetic system is a magnet or a plurality of magnets and the second magnetic part 211 of the magnetic system is a component containing a non-magnetized ferromagnetic material. In some embodiments, the first magnetic part 207 of the magnetic system is a component containing a non-magnetized ferromagnetic material and the second magnetic part 211 of the magnetic system is a magnet or a plurality of magnets. In some embodiments, any of the aforementioned magnets is an electromagnet. In some embodiments, any of the aforementioned magnets is a permanent magnet. In some embodiments, the first magnetic part 207 of the magnetic system and the second magnetic part 211 of the magnetic system are each magnet of similar shape, so that there is sufficient attraction between them. For example, both magnets are ring magnets, or both magnets are square magnets.
The plurality of pin grooves 219 corresponds spatially with the plurality of indicator pins 203 of the arm attachment portion 291. In some embodiments, the plurality of pin grooves 219 is a set of two grooves at opposite sides of the lower main body 213. In some embodiments, the pins 203 and the pin grooves 219 are chamfered, which enables the robotic system 100 to successfully attach the upper portion 291 and lower portion 292 even in the presence of spatial misalignment, which provides enhanced operational stability and repeatability.
In the embodiments shown in
The plurality of rack grooves 215 is designed to complement the slots of the tool plate described in reference to
In some embodiments, there is error tolerance built into the size, shape, and locations of the indicator pins 203 and corresponding pin grooves 219. This way, slight misalignments when adding a tool are tolerated. In some embodiments, the tips 201 of the indicator pins 203 are tapered and/or the pin grooves 219 are beveled with wider lips 217 to enable misalignments to be automatically corrected when the indicator pins 203 attempt to enter corresponding pin grooves 219.
In some embodiments, the upper main body 209, including the indicator pins 203, is manufactured from aluminum alloy. In some embodiments, lower main body 213 is manufactured from a softer material, such as nylon or polyoxymethylene (POM). Aluminum alloy sliding on a softer material may support millions of attachments and separations with very little wear and tear.
As would be apparent to a person skilled in the art, various configurations of indicator pins 203 and pin grooves 219 are possible. For example, the two components may be switched, so that the tool attachment portion 292 contains indicator pins 203 and the arm attachment portion 291 contains pin grooves 219. Alternatively, each of the portions 291, 292 may contain both at least one indicator pin 203 as well as at least one pin groove 219. For example, in some embodiments, the arm attachment portion contains both one indicator pin and one pin groove, and the tool attachment portion also contains both one indicator pin and one pin groove, where the indicator pin of the arm attachment portion corresponds spatially with the pin groove of the tool attachment portion, and where the indicator pin of the tool attachment portion corresponds spatially with the pin groove of the arm attachment portion. In other embodiments, the arm attachment portion contains both two indicator pins and two pin grooves, and the tool attachment portion also contains both two indicator pins and two pin grooves, where the two indicator pins of the arm attachment portion correspond spatially with the two pin grooves of the tool attachment portion, and where the two indicator pins of the tool attachment portion correspond spatially with the two pin grooves of the arm attachment portion.
The tool attachment portion 292 optionally includes a through-hole 221 to interact with a tool 116. The through-hole 221 may align with the through-hole 205 from the arm attachment portion 291 to create a sealed channel for the tool 116 when the arm attachment portion 291 and the tool attachment portion 292 come into contact with each other. A leakage prevention device (e.g., an O-ring, ring foam) may be used to seal the interface between the through-holes 205, 221 of the arm attachment portion 291 and of the tool attachment portion 292. The sealed channel may, in some embodiments, be capable of carrying a vacuum, compressed air, mechanical connections, or electrical connections. In some embodiments, the through-holes 205, 221 are centered on the faces of the tool attachment portion 292 and the arm attachment portion 291, respectively. In some embodiments, the tool attachment portion 292 includes a plurality of through-holes 221, each of which may be any of the aforementioned types.
In some embodiments, the tool plate 231 is made of mild steel or other ferromagnetic material. When the tool attachment portion 292 is docked in the tool plate 231, the lower magnet 211 in the tool attachment portion 292 attracts the tool plate 231, which ensures that the tool attachment portion 292 returns to a repeatable position in the tool plate 231.
In some embodiments, the tool plate 231 has hole features 227 that allow for mounting presence sensors 119, which monitor whether or not a tool attachment portion 292 is in the docked position. Some examples of these sensors 119 are hall-effect sensors, infrared sensors, ultrasonic sensors, and capacitive sensors. This monitoring function allows the system to detect whether a tool attachment portion 292 has fallen off during operations, or has been removed by an operator 101. In some embodiments, the tool plate 231 also has hole features 227 for a magnet assembly. The magnet assembly mounts a permanent magnet close to the lower tool changer assembly docking position, increasing the force retaining the tool attachment portion 292 in the docked position.
With reference to
With reference to
Source Pumps and Sensors Interacting with the End Effector Tool Changer
In one embodiment, the control system sends a signal to the valve to switch from one valve output to another through a data link. In one embodiment, the plurality of available tools are categorized by their corresponding source pump. For example, suction tools may require a compressed air pump whereas gripping tools may require a vacuum pump.
In one embodiment of step 1403, the system compares a weight reading from the weight sensor with a known weight of an attached tool, where a weight reading close to the known weight indicates that there is a tool attached to the end effector.
In one embodiment of step 1433, the system compares a pressure reading from the pressure sensor with atmospheric pressure, where a pressure reading close to the atmospheric pressure indicates that the attached tool is damaged. In another embodiment, a pressure reading close to the atmospheric pressure indicates that there is no tool attached to the end effector.
In some embodiments, the illustrative flow diagram shown in
In some embodiments, the illustrative flow diagram shown in
In one embodiment, the data received from the vision system is used to determine the visibility of objects in the input components. In one embodiment, low object visibility triggers the system to increase a light intensity for a lighting source.
In another embodiment, the system may begin by detecting a plurality of objects to be picked and determining one or more picking areas on a surface of each of the plurality of objects to be picked. In another embodiment, the system may estimate a picking score associated with each determined picking area and select one picking area per detected object based on the estimated picking scores. In another embodiment, the system may prioritize picking objects having picking areas associated with the highest picking score. In yet another embodiment, the system may further compute a group picking score for a group of detected objects (e.g., objects belonging to the same type) based on the picking scores associated with a picking area of each of the objects. In another embodiment, the system may prioritize picking object groups (e.g., types) having picking areas associated with the highest group picking score.
In one embodiment, the system may group detected objects (e.g., according to their object type) and select a tool based on the size of a detected object group.
Using Data from Vision Systems and Sensors for the End Effector Tool Changer
In one embodiment of step 2203, the system compares the pressure reading with atmospheric pressure, where a pressure reading close to the atmospheric pressure indicates that there is no object attached to the end effector tool.
In one embodiment, arm movement is halted when an operator or an unrecognized object obstructs a trajectory of the robotic arm. In another embodiment, the vision system includes a light curtain that is used to detect obstructions to robotic arm movement. In one embodiment, the light curtain is used as a safety measure to protect operators.
In various embodiments, the system combines data from one or more of the vision system, the weight sensor, the tool wire, and the pressure sensor, to determine whether a tool is attached at the end effector, whether an object was successfully picked by the robotic arm, whether an object was released or dropped by the robotic arm, whether more than one object is grasped by an attached tool, whether a tool is damaged, and whether a tool has fallen or become detached.
An exemplary embodiment of the present disclosure may include one or more servers (management computing entities), one or more networks, and one or more clients (user computing entities). For example, a server device may be used to remotely control the pick and place robotic system from a remote location, and a client device may be used to remotely monitor the operations of the robotic system. Similarly, various computing components of the robotic system may be implemented using either the server or client device architecture described herein. Each of these components, entities, devices, and systems (similar terms used herein interchangeably) may be in direct or indirect communication with, for example, one another over the same or different wired or wireless networks. Additionally, while
As indicated, in one embodiment, the management computing entity 2601 may also include one or more communications interfaces 2620 for communicating with various computing entities, such as by communicating data, content, and/or information (similar terms used herein interchangeably) that can be transmitted, received, operated on, processed, displayed, stored, and/or the like.
As shown in
In one embodiment, the management computing entity 2601 may further include or be in communication with non-volatile media (also referred to as non-volatile storage, memory, memory storage, and/or circuitry—similar terms used herein interchangeably). In one embodiment, the non-volatile storage or memory may include one or more non-volatile storage or memory media 2610, including but not limited to hard disks, ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrack memory, and/or the like. As will be recognized, the non-volatile storage or memory media may store databases, database instances, database management systems, data, applications, programs, program modules, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like. The term database, database instance, and/or database management system (similar terms used herein interchangeably) may refer to a collection of records or data that is stored in a computer-readable storage medium using one or more database models, such as a hierarchical database model, network model, relational model, entity-relationship model, object model, document model, semantic model, graph model, and/or the like.
In one embodiment, the management computing entity 2601 may further include or be in communication with volatile media (also referred to as volatile storage, memory, memory storage, and/or memory circuitry—similar terms used herein interchangeably). In one embodiment, the volatile storage or memory may also include one or more volatile storage or memory media 2615, including but not limited to RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, and/or the like. As will be recognized, the volatile storage or memory media may be used to store at least portions of the databases, database instances, database management systems, data, applications, programs, program modules, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like being executed by, for example, the processing element 2605. Thus, the databases, database instances, database management systems, data, applications, programs, program modules, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like may be used to control certain aspects of the operation of the management computing entity 2601 with the assistance of the processing element 2605 and operating system.
As indicated, in one embodiment, the management computing entity 2601 may also include one or more communications interfaces 2620 for communicating with various computing entities, such as by communicating data, content, and/or information (similar terms used herein interchangeably) that can be transmitted, received, operated on, processed, displayed, stored, and/or the like. Such communication may be executed using a wired data transmission protocol, such as fiber distributed data interface (FDDI), digital subscriber line (DSL), Ethernet, asynchronous transfer mode (ATM), frame relay, data over cable service interface specification (DOCSIS), or any other wired transmission protocol. Similarly, the management computing entity 2601 may be configured to communicate via wireless external communication networks using any of a variety of protocols, such as general packet radio service (GPRS), Universal Mobile Telecommunications System (UMTS), Code Division Multiple Access 2000 (CDMA2000), CDMA2000 1× (1×RTT), Wideband Code Division Multiple Access (WCDMA), Time Division-Synchronous Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), Evolved Universal Terrestrial Radio Access Network (E-UTRAN), Evolution-Data Optimized (EVDO), High-Speed Packet Access (HSPA), High-Speed Downlink Packet Access (HSDPA), IEEE 802.11 (Wi-Fi), Wi-Fi Direct, 802.16 (WiMAX), ultra-wideband (UWB), infrared (IR) protocols, near field communication (NFC) protocols, Wibree, Bluetooth protocols, wireless universal serial bus (USB) protocols, and/or any other wireless protocol.
Although not shown, the management computing entity 2601 may include or be in communication with one or more input elements, such as a keyboard input, a mouse input, a touch screen/display input, motion input, movement input, audio input, pointing device input, joystick input, keypad input, and/or the like. The management computing entity 2601 may also include or be in communication with one or more output elements (not shown), such as audio output, video output, screen/display output, motion output, movement output, and/or the like.
As will be appreciated, one or more of the components of the management computing entity 2601 may be located remotely from other management computing entity 2601 components, such as in a distributed system. Furthermore, one or more of the components may be combined and additional components performing functions described herein may be included in the management computing entity 2601. Thus, the management computing entity 2601 can be adapted to accommodate a variety of needs and circumstances. As will be recognized, these architectures and descriptions are provided for exemplary purposes only and are not limiting to the various embodiments.
A user may be an individual, a company, an organization, an entity, a department within an organization, a representative of an organization and/or person, and/or the like.
The signals provided to and received from the transmitter 2704 and the receiver 2706, respectively, may include signaling information in accordance with air interface standards of applicable wireless systems. In this regard, the user computing entity 2701 may be capable of operating with one or more air interface standards, communication protocols, modulation types, and access types. More particularly, the user computing entity 2701 may operate in accordance with any of a number of wireless communication standards and protocols, such as those described above with regard to the management computing entity 2601. In a particular embodiment, the user computing entity 2701 may operate in accordance with multiple wireless communication standards and protocols, such as UMTS, CDMA2000, 1×RTT, WCDMA, TD-SCDMA, LTE, E-UTRAN, EVDO, HSPA, HSDPA, Wi-Fi, Wi-Fi Direct, WiMAX, UWB, IR, NFC, Bluetooth, USB, and/or the like. Similarly, the user computing entity 2701 may operate in accordance with multiple wired communication standards and protocols, such as those described above with regard to the management computing entity 2601 via a network interface 2720.
Via these communication standards and protocols, the user computing entity 2701 can communicate with various other entities using concepts such as Unstructured Supplementary Service Data (USSD), Short Message Service (SMS), Multimedia Messaging Service (MMS), Dual-Tone Multi-Frequency Signaling (DTMF), and/or Subscriber Identity Module Dialer (SIM dialer). The user computing entity 2701 can also download changes, add-ons, and updates, for instance, to its firmware, software (e.g., including executable instructions, applications, program modules), and operating system.
According to one embodiment, the user computing entity 2701 may include location determining aspects, devices, modules, functionalities, and/or similar words used herein interchangeably. For example, the user computing entity 2701 may include outdoor positioning aspects, such as a location module adapted to acquire, for example, latitude, longitude, altitude, geocode, course, direction, heading, speed, universal time (UTC), date, and/or various other information/data. In one embodiment, the location module can acquire data, sometimes known as ephemeris data, by identifying the number of satellites in view and the relative positions of those satellites. The satellites may be a variety of different satellites, including Low Earth Orbit (LEO) satellite systems, Department of Defense (DOD) satellite systems, the European Union Galileo positioning systems, the Chinese Compass navigation systems, Indian Regional Navigational satellite systems, and/or the like. Alternatively, the location information can be determined by triangulating the user computing entity's 2701 position in connection with a variety of other systems, including cellular towers, Wi-Fi access points, and/or the like. Similarly, the user computing entity 2701 may include indoor positioning aspects, such as a location module adapted to acquire, for example, latitude, longitude, altitude, geocode, course, direction, heading, speed, time, date, and/or various other information/data. Some of the indoor systems may use various position or location technologies including RFID tags, indoor beacons or transmitters, Wi-Fi access points, cellular towers, nearby computing devices (e.g., smartphones, laptops) and/or the like. For instance, such technologies may include the iBeacons, Gimbal proximity beacons, Bluetooth Low Energy (BLE) transmitters, NFC transmitters, and/or the like. These indoor positioning aspects can be used in a variety of settings to determine the location of someone or something to within inches or centimeters.
The user computing entity 2701 may also comprise a user interface (that can include a display 2716 coupled to a processing element 2708) and/or a user input interface (coupled to a processing element 2708). For example, the user interface may be a user application, browser, user interface, and/or similar words used herein interchangeably executing on and/or accessible via the user computing entity 2701 to interact with and/or cause display of information from the management computing entity 2601, as described herein. The user input interface can comprise any of a number of devices or interfaces allowing the user computing entity 2701 to receive data, such as a keypad 2718 (hard or soft), a touch display, voice/speech or motion interfaces, or other input device. In embodiments including a keypad 2718, the keypad 2718 can include (or cause display of) the conventional numeric (0-9) and related keys (#, *), and other keys used for operating the user computing entity 2701 and may include a full set of alphabetic keys or set of keys that may be activated to provide a full set of alphanumeric keys. In addition to providing input, the user input interface can be used, for example, to activate or deactivate certain functions, such as screen savers and/or sleep modes.
The user computing entity 2701 can also include volatile storage or memory 2722 and/or non-volatile storage or memory 2724, which can be embedded and/or may be removable. For example, the non-volatile memory may be ROM, PROM, EPROM, EEPROM, flash memory, MMCs, SD memory cards, Memory Sticks, CBRAM, PRAM, FeRAM, NVRAM, MRAM, RRAM, SONOS, FJG RAM, Millipede memory, racetrack memory, and/or the like. The volatile memory may be RAM, DRAM, SRAM, FPM DRAM, EDO DRAM, SDRAM, DDR SDRAM, DDR2 SDRAM, DDR3 SDRAM, RDRAM, TTRAM, T-RAM, Z-RAM, RIMM, DIMM, SIMM, VRAM, cache memory, register memory, and/or the like. The volatile and non-volatile storage or memory can store databases, database instances, database management systems, data, applications, programs, program modules, scripts, source code, object code, byte code, compiled code, interpreted code, machine code, executable instructions, and/or the like to implement the functions of the user computing entity 2701. As indicated, this may include a user application that is resident on the entity or accessible through a browser or other user interface for communicating with the management computing entity 2601 and/or various other computing entities.
In another embodiment, the user computing entity 2701 may include one or more components or functionality that are the same or similar to those of the management computing entity 2601, as described in greater detail above. As will be recognized, these architectures and descriptions are provided for exemplary purposes only and are not limiting to the various embodiments.
The present invention may be implemented in a client server environment.
One of ordinary skill in the art knows that the use cases, structures, schematics, and flow diagrams may be performed in other orders or combinations, but the inventive concept of the present invention remains without departing from the broader scope of the invention. Every embodiment may be unique, and methods/steps may be either shortened or lengthened, overlapped with the other activities, postponed, delayed, and continued after a time gap to practice the methods of the present invention.
Although the present invention has been described with reference to specific exemplary embodiments, it will be evident that the various modifications and changes can be made to these embodiments without departing from the broader scope of the invention. Accordingly, the specification and drawings are to be regarded in an illustrative sense rather than in a restrictive sense. It will also be apparent to the skilled artisan that the embodiments described above are specific examples of a single broader invention which may have greater scope than any of the singular descriptions taught. There may be many alterations made in the descriptions without departing from the scope of the present invention.
| Number | Date | Country | |
|---|---|---|---|
| 63196707 | Jun 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/US22/20085 | Mar 2022 | US |
| Child | 18526248 | US |
