The invention generally relates to robotics and more specifically to a system and method for under-constraining a robot or robotic system.
Tasking a robot is difficult, and this difficulty is exacerbated by tasking robots in an unnatural, over constrained way. Natural tasking is a comprehensive approach to specifying robotic tasks easily without over- or under-constraint.
A gymnast, for example, does not model the exact position of hand placements on bars, but rather models the bars as having one degree of freedom in placement. Similarly, tasks like welding and painting have an extra degree of freedom in how the electrode or nozzle is rotated about its primary axes, and the operator tasking a robot benefits by working with these natural constraints. Similarly, a workpiece to be grasped by an industrial robot may have symmetry that allows grasping in multiple ways. Natural tasking specifies the constraint on the robot in its minimal and least restrictive, way.
In one or more embodiments of the present disclosure, a system is provided. The system may include a robot configured to receive an initial constrained approach for performing a robot task. The system may further include a graphical user interface in communication with the robot. The graphical user interface may be configured to allow a user to interact with the robot to determine an allowable range of robot poses associated with the robot task. The allowable range of robot poses may include fewer constraints than the initial constrained approach. The allowable range of poses may be based upon, at least in part, one or more degrees of symmetry associated with a workpiece associated with the robot task or an end effector associated with the robot. The system may also include a processor configured to communicate the allowable range of robot poses to the robot.
One or more of the following features may be included. The initial constrained approach for performing the robot task may be received from at least one of a user input, a program, a simulation, or a user demonstration. The one or more degrees of symmetry may include a degree of symmetry of a pick feature of a workpiece associated with the robot task. The degree of symmetry of the pick feature of the workpiece may be continuous, discrete, or asymmetrical. The one or more degrees of symmetry may include a degree of symmetry of a robot end effector. The degree of symmetry of the robot end effector may be continuous, discrete, or asymmetrical. The one or more degrees of symmetry may include a degree of symmetry of the placement target associated with the robot task. The degree of symmetry of the placement target may be continuous, discrete, or asymmetrical. The processor may be further configured to determine a first valid pose for a workpiece relative to a robot end effector associated with the robot. The processor may be further configured to identify an axis of a coordinate frame associated with the workpiece that is coincident with an axis of the robot end effector's coordinate frame when the workpiece is in the first valid pose. The processor may be further configured to determine a range of valid robot end effector poses by allowing rotation of the robot end effector based upon, at least in part, a degree of symmetry determined for either the workpiece around the axis of the workpiece's coordinate frame or for the robot end effector around the axis of the robot end effector's coordinate frame. The graphical user interface may be configured to display the robot end effector, a workpiece, and at least one of an axis of a coordinate frame associated with the robot end effector or an axis of a coordinate frame associated with the workpiece and wherein the graphical user interface is configured to allow a user to specify whether the robot end effector has substantially equivalent performance for picking up a workpiece when the robot end effector is rotated around one of the axis of the coordinate frame associated with the robot end effector or the axis of the coordinate frame associated with the workpiece. The processor may be further configured to determine a placement for a workpiece relative to a placement target associated with the robotic system. The processor may be further configured to determine a range of valid robot end effector poses by allowing rotation of the robot end effector based upon, at least in part, a degree of symmetry determined for either the workpiece around the axis of the workpiece's coordinate frame or for the placement target. The graphical user interface may be configured to display the robot end effector, a workpiece, and at least one of an axis of a coordinate frame associated with the robot end effector or an axis of a coordinate frame associated with the workpiece and wherein the graphical user interface is configured to allow a user to specify whether the robot end effector has substantially equivalent performance for placing the workpiece at the placement target when the robot end effector is rotated around the axis of the coordinate frame associated with the robot end effector or the axis of the coordinate frame associated with the workpiece. The processor may be further configured to determine a minimum number of degrees of freedom (DOFs) required for the robot to perform the task associated with the range of poses. The processor may be further configured to determine an additional number of degrees of freedom required satisfy one or more additional constraints. The processor may be further configured to combine the minimum number of degrees of freedom with the additional number of degrees of freedom to determine a total number of degrees of freedom, wherein the processor may be further configured to compare the total number of degrees of freedom with a maximum number of allowable degrees of freedom associated with the robot. The processor may be further configured to determine a difference between the maximum number of allowable degrees of freedom and the total number of degrees of freedom and to optimize performance of the robot, based upon, at least in part, a range of poses associated with the difference. The allowable range of poses may be based upon, at least in part, one or more degrees of symmetry associated with a placement target associated with the robot task.
In another embodiment of the present disclosure, a method is provided. The method may include providing a robot configured to receive an initial constrained approach for performing a robot task. The method may further include allowing a user to interact with the robot using a graphical user interface in communication with the robot. The graphical user interface may be configured to allow a user to interact with the robot to determine an allowable range of robot poses associated with the robot task. The allowable range of robot poses may include fewer constraints than the initial constrained approach, and the allowable range of poses may be based upon, at least in part, one or more degrees of symmetry associated with at least one of a workpiece associated with the robot task or an end effector associated with the robot. The method may also include communicating, via a processor, the allowable range of robot poses to the robot.
For a better understanding of the nature and objects of the present disclosure, reference is made to the following detailed description taken in conjunction with the following drawings, in which:
Embodiments of the subject application may include concepts from U.S. Pat. Nos. 6,757,587, 7,680,300, 8,301,421, 8,408,918, 8,428,781, 9,357,708, U.S. Publication No. 2015/0199458, U.S. Publication No. 2016/0321381, and U.S. Publication No. 2018/0060459, the entire contents of each are incorporated herein by reference in their entirety.
Referring now to
In a robotic system, the phrase “degrees of freedom” may refer to specific, defined modes in which a mechanical device or system can move. The number of degrees of freedom may be equal to the total number of independent displacements or aspects of motion. For example, a six degrees of freedom (“6DOF”) scenario may refer to the freedom of movement of a rigid body in three-dimensional space. Specifically, the body may be free to change position as forward/backward (surge), up/down (heave), left/right (sway) translation in three perpendicular axes, combined with changes in orientation through rotation about three perpendicular axes, often termed yaw (normal axis), pitch (transverse axis), and roll (longitudinal axis). In contrast, placing a point in space may correspond three degrees of freedom, specifying a distance between two points on different links is one degree of freedom, etc.
In some embodiments, the phrase “robotic system”, as used herein, may include a system of one, two, and/or any number of robots. In this way, an entire robotic system DOF may refer to the sum of the DOFs on each of the individual robots. This may include one DOF for each single-axis joint, and six DOF for a free-moving base. For example, for a robotic system that includes two robots, one having 6DOF and the other having 5DOF, the available entire robotic system degrees of freedom may be 11DOF.
Embodiments included herein may be configured to explore an extra number of degrees of freedom of one or many robots to perform a robotic task naturally. In this example, an entire robotic system may include a maximum number of degrees of freedom associated therewith DOF (“N”) and may also include a minimum number of DOF (“M”) subtracted therefrom.
For example, and referring now to
In some embodiments, and referring now to
In some embodiments, and referring now to
In some embodiments, the core velocity framework may be based on the manipulator Jacobian equation:
V=J(q){dot over (q)} (1)
Here V is an m-length vector representation of the motion of the end effector (usually some combination of linear and angular velocity referenced to points rigidly attached to parts of the manipulator); q is the n-length vector of joint positions (with {dot over (q)} being its time derivative); and J is the m×n manipulator Jacobian, a function of q. (For spatial arms with a single end effector, V is often the frame velocity with three linear and three angular components. As used herein, it takes on a larger meaning that includes the concatenation of point, frame, or other motion of multiple end-effectors.) The Jacobian J(q) is the matrix that makes ( ) true for all possible values of {dot over (q)}. Note V can represent a concatenation of values for multiple end effectors, enabling coordinated motion of multiple points on the manipulator.
The velocity control question is the following: given a desired end effector motion V, what are the joint rates {dot over (q)} that best achieve this motion? To answer this, the framework is based, in part, on the approach below, which uses a scalar α, a matrix function W(q) , and a scalar function ƒ(q) to solve for {dot over (q)} given V through the following formula:
Where ∇ƒ is the gradient of ƒ and NJ is an n×(n×m) set of vectors that spans the null space of J. That is, JNJ=0, and NJ has rank (n×m). Both ∇ƒ and NJ are generally functions of q. By changing the values of α, W, and ƒ, many new and most established velocity-control techniques can be implemented.
Embodiments of the present disclosure, however, go beyond the formulation above to create a more general framework. Instead of insisting on the use of the gradient of a function, a general column vector F(q) is used. Not all vector functions are gradients. This minor, but important, modification yields the following formula:
This achieves the desired V while minimizing:
½{dot over (q)}TW{dot over (q)}+αFT{dot over (q)} (4)
The parameters α, W and F can be defined using XML to give many different types of velocity control.
In some embodiments, position control system 402 may build upon the velocity control system 404 to give robust position control to the end effectors. A position control module may be supplied as an interface to the position control system 402 to enable the addition of new position control algorithms. Position control system 402 may provide basic position control for the end effectors by building upon velocity control system 404. Given the joint velocities from velocity control system 404, position control system 402 may use Euler integration to calculate the joint positions. Once the joint positions are calculated, the end effector positions are known. The control system may check for joint limit exceedance and collisions as it is iterated forward in time. It may also zero the joint rate commands if a singularity is detected.
In some embodiments, the control methodologies discussed above may be used, at least in part, to allow for the controller to relate the entire robotic system DOF (e.g., the available DOF of the system, “N”) to the required minimum number of degrees of freedom identified based on the workpiece constraints alone (“M”). In other words, the robotic system and method described herein may only need workpiece data in order to execute a natural robot task. In some cases, the DOF may be based upon other factors as well, including but not limited to, other factors or constraints, however this information may not be necessary. Embodiments of the robotic system and methods provided herein may be available to provide better control and/or increased performance of the robotic system, more feasible and/or optimized solutions for a robotic task, etc.
The phrase “maximum number of degrees of freedom” may refer to a maximum number of specific, defined modes in which a mechanical device, robot(s), and/or system may move. The phrase “minimum number of degrees of freedom” may refer to a minimum number of specific, defined modes in which a mechanical device, robot(s), and/or system move to execute a task naturally. The phrase “natural robot task” may refer to a task that may be performed using a reduced and/or simplified amount of motion. In some cases specifying a natural robot task may be contrasted with a math intensive way of specifying a task that may require a user to specify in robotic system's coordinate/frame The natural robot task may be specified/demonstrated to the robotic system using one or more robots or end effectors and the minimum number of degrees of freedom may be determined based upon, at least in part, the natural robot task. The minimum number of degrees of freedom may be determined through analysis of end effector, robot, and/or workpiece geometry. It may be specified using a demonstration, teach to learn robotic techniques, using a GUI, etc.
In some embodiments, the naturalness of the task may refer to the degree of alignment between 1) the commanded robot task and 2) the actual desired thing to be done. Some examples are provided below:
It should be noted that although these unnatural constraints may seem odd in how they over restrict robot motion, they are actually how these kind of tasks are typically performed. Embodiments of the robotic system included herein allow using and solving for the natural representation.
In some embodiments, the phrase “natural workpiece constraint” may relate to geometrical features, and/or any other aspect associated with a particular workpiece. The phrase “natural environmental constraint” may relate to geometrical features, obstacles, and/or any other aspect associated with an environment that may be associated with a natural robot task.
In some embodiments, workpiece constraints, environmental constraints, and manipulator constraints may form a continuum in this approach, and multiple natural constraints can be combined to define a comprehensive tasking movement. Some of these may include, but are not limited to, directional constraints, orientation constraints, assignment to a curve or surface, distance constraints, joint positions, linear and nonlinear functional combinations of joint positions, center of mass, and linear and angular momentum. These constraints may vary as a function of robot configuration. End effector offsets, static workpieces or obstacles, and dynamic workpieces or obstacles may be used to define and configure the constraints. In some embodiments, the identification of the minimum number of degrees of freedom may be based on the workpiece constraints geometry and/or its features, some of which may include, but are not limited to, holes, shapes, sizes, etc.
In some embodiments, one or more constraints may be combined and/or specified by the user using a visual editor to create complex combinations of constraints that change over time to best achieve a real-world task. It should be noted that some or all of these constraints may be changed during execution of a robot task or operation, even time step by time step, and may be combined with optimizations such as collision and joint-limit avoidance. In this way, the number and type of constraints may be updated, dynamically, in real-time. For example, in some embodiments, some or all of the constraints may be automatically determined and/or identified using a scanner that may be configured to scan the robotic environment to identify, allow for the necessary processing, and subsequent determinations. Various sensors and devices may be used to obtain this data, some of which may include, but are not limited to, data from lidar, stereo cameras, color cameras, multispectral cameras, etc. Additionally and/or alternatively, some or all of the constraints may be updated remotely over a communication channel (e.g., radio, laser, or network communication, etc.). The constraints are set using a 2D graphical user interface or a 3D interface (e.g., augmented reality, etc.). The constraints may be of the “less than or equal to” variety, not just “equal to”. Accordingly, the robotic system included herein may allow a user to constrain a point to be above the floor not just on it, for example.
Accordingly, embodiments of the robotic system described herein may include one or more robots having a maximum number of degrees of freedom “N”. In some embodiments, the system may allow a user to specify (e.g., teaching the robotic system by example, performing a robot “watch” and learn using one or more sensors, using a GUI such as a teach pendant to perform a demo, etc.) a task naturally by indicating only what needs to be done, such as insert a peg in a hole (using five degrees of freedom in constraint), follow a line with the tip of a welding tool (using three degrees of freedom in constraint), or point a camera at a location in space (using two degrees of freedom in constraint), etc. Specifying a task naturally may include specifying the task using a demonstration, for example, teaching or learning the robot how to perform the task. Additionally and/or alternatively, the user may not specify how the robotic system may perform the task and the system may determine how to perform the task automatically or using minimal inputs.
In some embodiments, the robotic system described herein may be configured to determine the minimum number of degrees of freedom “M” that may be required to perform the natural robot task, while M<=N. The robotic system may be configured to determine, via exploring the extra (N−M) extra degree of freedom, the most natural or efficient manner in which to perform the task. This may involve optimizing secondary criteria using the extra degrees of freedom. For example, a controller (e.g. Actin) may help to enable exploring the extra degrees of freedom (N−M) as discussed above. The extra number of degrees of freedom may be used to determine more convenient, faster, energy efficient, and/or feasible, “ways” to perform the task. The way or manner of performing may be based upon, at least in part, the determination of collision free paths, coordination of robots (if more than one), etc. In some embodiments, executing or performing the natural robot task may be at least partly employed using various software methodologies (e.g., using Energid's Actin® software). Determining the way or manner of performing the task naturally may also include using one or more simulators that may be configured to perform a simulation. The results of such a simulation may be used to determine how to perform the task.
In some embodiments, exploring the extra degrees of freedom (N−M) may include completely relaxing some DOF (e.g., the “Z” dimension). Additionally and/or alternatively, embodiments included herein may be configured to relax a DOF partially, for example, allow Z to free spin within +−15 degrees. As used herein, the phrase “free spin” may refer to allowing for unconstrained robot motion in at least one axis (e.g., end effector axis, etc.). The same concept may be applied to any DOF to provide flexibility when specifying a natural task. Accordingly, this may allow the robot system to find a solution in a more efficient manner.
In some embodiments, the robotic system described herein may be configured to specify one or more robot poses. Commonly, robots may be programmed as a series of poses, i.e. particular orientations of the robot's joints and rigid elements. The robot may then cycle through these poses under the command of a robot controller to accomplish some task. For example, picking up a workpiece with its end effector and transporting it to a new location, moving a paint sprayer through a series of positions, etc.
In some cases, sensory input to the robot may be limited to error signals, pause signals, or speed signals. These signals only stop, pause, or slow the movement of the robot through its series of poses, but do not affect the poses themselves. In more advanced cases, sensory input to the robot may be used to affect the poses themselves. For example, a 2D camera may be used to identify the position of a workpiece on a flat surface in a known position. From this, a robot pose to pick up the workpiece may be calculated. Generally, only a single robot pose may be calculated for any given sensory input. This minimizes calculation time, but does not take in to account the most efficient path for the robot to achieve the pose, whether the robot pose at workpiece pick-up also yields a correct path for the robot to put the workpiece down again, or the most efficient path between those points.
Allowing robot pose calculation and motion control based on multiple possible robot poses is desirable for more efficient motion, and discovery of more realizable pose combinations. Unfortunately, specifying the possible robot poses requires understanding robot kinematics, and applying them to every specific case in the robot picking and placing task. It has heretofore been impractical or impossible for an unskilled user to specify these variables. Accordingly, embodiments included below may be used to specify these variables indirectly, simply by describing a few characteristics of the workpiece, end effector, and workpiece placement target.
In some embodiments, and referring now to
Referring now to
In operation, the user may demonstrate to the robotic system how to perform a particular task. This demonstration may be received using any suitable approach, including, but not limited to, via a graphical user interface, the user physically guiding the robot, using a simulation, using a pre-generated program, using a program generated in real-time, etc. The demonstration may provide an initial constrained approach that shows the robotic system exactly how to perform the particular task. In previous systems, once the initial constrained approach has been learned/input the system may use the same pick pose, place pose, path, etc. to perform the task.
In some embodiments, the robotic process described herein may allow for a relaxation of the initial constrained approach by determining an allowable range of poses that may reduce the number of constraints from the initial constrained approach. Accordingly, embodiments of robotic process may identify and subsequently allow for a range of poses for picking and/or placing a workpiece any of which being acceptable for accomplishing the task. In some embodiments, the least constraining option may be provided and/or selected (e.g. automatically).
In some embodiments, the allowable range of poses may be based upon, at least in part, a workpiece geometry, an end effector geometry, a combination of both geometries, a portion of either geometry, a portion of both geometries, etc. It should be noted that these geometries may be received using any suitable approach, including, but not limited to, scanning, user input via graphical user interface, accessing one or more pre-determined or real-time generated programs, etc.
In some embodiments, one or more degrees of freedom associated with the robot may be partially or completely relaxed. The process may then determine and/or plan the appropriate path and/or poses for the robot to implement the task. This may include, but is not limited to, determining redundant degrees of freedom, considering environment obstacles, system performance optimization (e.g., energy efficiency), specific metric optimization, etc.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
The embodiments described above illustrate that picking and placing performance may be enhanced by allowing for more options in the position of the robot end effector relative to the workpiece and workpiece placement target, but at the cost of a difficult process to understand and specify the limitations within these options. This complexity is increased if, unlike in the examples, workpiece picking is 3-dimensional, such as when workpieces are randomly distributed in a bin.
Embodiments described herein include a process for synthesizing the motion limitations based upon a set of characteristics of the components of the system. The characteristics may be specified in a graphical user interface used as part of programming the robot or the application as is discussed in further detail hereinbelow.
As used herein, the phrase “pick feature” may refer to some section of the workpiece that may be used as the point of contact or mating contact for a robot end effector. A “place feature” may refer to some section of the workpiece or of the place target that may be used as the point of contact or mating contact between the workpiece and the place target for the workpiece. The phrase “workpiece feature” may refer to either a pick feature or a place feature or both. The phrase “coordinate frame”, as used herein, generally refers to three orthogonal axes that may correspond to each robot, workpiece feature, robot end effector, and workpiece placement target. The term “symmetry” of a workpiece feature, robot end effector, and workpiece placement target, as used herein, may be described in terms of whether that feature, robot end effector, or workpiece placement target has identical performance for its intended picking or placing operation, when rotated around one or more of the three axes of its coordinate frame. When the workpiece feature, robot end effector, or workpiece placement target may be rotated any degree around one of its axes, with identical, or substantially identical, performance for picking or placing, it has “continuous symmetry” around that axis. When the workpiece feature, robot end effector, or workpiece placement target may be rotated any whole number of increments around one of its axes, but more than one and less than infinite increments, with identical performance for picking or placing at each increment, it has “discrete symmetry” around that axis. When the workpiece feature, robot end effector, or workpiece placement target may be rotated only one complete revolution (i.e. an identity operation) around one of its axes, to have identical performance for picking or placing, it has “asymmetry” around that axis.
Accordingly, unconstraining robot axes and exploiting symmetries in the workpiece, end effector, and/or place target, may lead to smoother paths, more possible picks, and better bin emptying (e.g., in a situation where the workpiece may be randomly placed in a bin). Embodiments included herein may institute a default of constrained motion. If a user demonstrates or programs a pick or a place, the system may duplicate that exact alignment of workpiece, end effector, and place target on every pick & place. All unconstrained motions may be displayed using a graphical user interface (e.g., in advanced tabs) such as those discussed below. Accordingly, this may simplify the creation of a user's first program and therefore match his/her intuitive understanding of how most systems work, which is suitable for debugging a program using hand-arranged workpieces.
In some embodiments, unconstraining motion may be performed incrementally. For example, unconstraining the end effector rotation during a pick has an incremental benefit. Moreover, unconstraining the workpiece pose during placement has an independent, but additive benefit.
In some embodiments, unconstraining robot axes may not be specified directly as it may require an advanced understanding of kinematics and robots in general. Instead, embodiments included herein may provide a wizard or a means of entering characteristics that may query the user about characteristics of the workpiece, the end effector, and/or the place target. The system may use this information to calculate and apply motion constraints. This may simplify user training and free the user from having to consider, for example, the combinations of end effector and workpiece characteristics that affect unconstraining the robot. The salient workpiece, end effector, and place target characteristics are symmetry.
As discussed above, embodiments of the present disclosure may allow for the analysis of various types of symmetry before, during, and/or after aspects of the process. In some embodiments, the only symmetry information that may be needed to unconstrain robot axes are those of the pick feature on the workpiece, the place feature on the workpiece, the end effector, and/or the place target. In some embodiments, the system may only need to know about rotational symmetry. Rotational symmetry is always relative to some axis of rotation. In some embodiments, all rotational symmetry may be around one of the three axes of the associated frame, e.g., the x, y, or z axis.
Referring now to
In some embodiments, for unconstraining the robot axes for pick and place, the symmetry of the pick feature and the place feature may be considered separately from each other, and from the overall workpiece symmetry.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
In some embodiments, the system may be configured to calculate motion constraints from symmetry information. The unconstrained motion may be easily calculated by examining the following combinations: the workpiece pick feature and the end effector and the workpiece place feature and the place target place feature. This all assumes an unconstrained environment as there may be further rules when the environment is constrained. The preferred rule is to operate according to the lowest constraint. For example, if an end effector has continuous symmetry around its z axis, and the pick feature is asymmetrical, the system may take advantage of the less constrained feature, the continuous symmetry of the end effector around its z axis, and allow the picking of the part using any valid end effector pose that varies only in the rotation around the end effector z axis. Allowing this freedom of end effector pose essentially allows unconstrained robot motion in one of the end effector axes, here in end effector z. Alternatively, if an end effector has 180° discrete symmetry around its z axis, and the pick feature has continuous symmetry around the workpiece x axis, this system may take advantage of the less constrained feature, the continuous symmetry of the pick feature around the workpiece x axis, and allow the picking of the part using any valid end effector pose that varies only in the end effector rotation around the workpiece x axis. Allowing this freedom of end effector pose may be known as free spin in workpiece x. If the workpiece place feature has 120° discrete symmetry around the associated workpiece axis, and the place target place feature has 60° discrete symmetry around the associated place target axis, the system may take advantage of the less constrained feature, the 60° discrete place target symmetry (the smaller of the two angles), and allow placing the part using any valid end effector pose in 60° increments of rotation around the place target reference axis. Allowing this freedom of end effector pose may be known as discrete spin around the place target reference axis x. In this example, the workpiece coordinate frame and the place target coordinate frame may need to be aligned to the same zero angle. Some possible rules examples are provided in
Referring now to
Referring now to
In some embodiments, all pick planning and motion constraints may be programmed as dialogues on the graphical user interface (e.g., Advanced tabs) for picking, placing, and the end effector. In some embodiments, the user may elect to specify anywhere from zero to all four of the symmetries. Every additional piece of information may improve performance, even when the others are left at the default (e.g., asymmetrical). In fact, when one of the two halves of the pick and place pairs (pick feature and end effector, or place feature and place target, place feature, respectively) have continuous symmetry, the other half of the pair's symmetry is irrelevant.
Referring now to
Referring now to
Referring now to
Referring now to
Referring now to
In some embodiments, the graphical user interfaces of the present disclosure may allow for workpiece and end effector symmetries to be defined using various user-editable options such as the installation tab described above. Accordingly, there is no need to edit symmetries in pick/place rules pages. Ease of use may be achieved through the visualization of symmetries in the viewer and in some embodiments the viewer display end effector rotation. In some embodiments it may not be necessary to explicitly define zero reference axis for rotational symmetry as a zero reference may be defined implicitly by pick and place rules.
In some embodiments, and as discussed above, the present disclosure may provide a method for specifying the allowable range of poses of a robot and/or robot end effector for the purpose of picking up a workpiece. The method may include graphically displaying the robot and/or robot end effector, the workpiece, and/or the axes of the workpiece coordinate frame. The method may also include querying a user whether the end effector has substantially equivalent performance for picking up the workpiece when the end effector is rotated around one or more axes of its coordinate frame.
In some embodiments, when the end effector is rotated around one or more axes of its coordinate frame, it may be determined that the end effector has substantially equivalent performance for picking up the workpiece. In this situation, the user may be allowed to specify each rotation as continuous, or in a number of discrete steps. In some cases, the specified rotation may be displayed as a series of images. For example, at a graphical display such as those described above (e.g., on a robot teach pendant, or any other suitable device).
In some embodiments, the present disclosure may provide a method for specifying the allowable range of poses of a workpiece feature for the purpose of being contacted by a robot and/or robot end effector. The method may include graphically displaying the workpiece, the axes of its coordinate frame, and/or the robot or the robot end effector, where the robot and/or robot end effector may be displayed in a pose for contacting the workpiece. The method may further include querying a user as to whether the end effector has substantially equivalent performance for contacting the workpiece when the workpiece is rotated around one or more axes in its coordinate frame.
In some embodiments, when the workpiece is rotated around one or more axes of its coordinate frame, the robot and/or robot end effector may have substantially equivalent performance for contacting the workpiece. Accordingly, the user may specify each rotation as continuous, or in a number of discrete steps.
In some embodiments, the present disclosure may provide a method for specifying the allowable range of poses of a workpiece for the purpose of placing it at a workpiece placement target. The method may include graphically displaying the workpiece, the axes of its coordinate frame, and/or the workpiece placement target. In some embodiments, the workpiece may be displayed in a pose for placing the workpiece at the workpiece placement target. The method may include querying a user as to whether the workpiece may be placed correctly when the workpiece is rotated around one or more axes in its coordinate frame.
In some embodiments, the present disclosure may provide a method for specifying the allowable range of poses of a robot and/or robot end effector for the purpose of placing a workpiece at a workpiece placement target, where the allowable range of poses being relative to the workpiece placement target. The method may include graphically displaying the robot and/or robot end effector, the workpiece, the axes of the workpiece coordinate frame, and/or the workpiece placement target. In some embodiments the robot and/or robot end effector may be displayed in a pose for placing a workpiece at the workpiece placement target. The method may include querying a user as to whether it may be possible to place the workpiece at the workpiece placement target correctly when the workpiece is rotated around one or more axes in its coordinate frame.
In some embodiments, when the workpiece is rotated around one or more axes in its coordinate frame, the workpiece may have at least one correct placement at the placement target. In this situation, the user may specify each rotation as continuous, or in a number of discrete steps. If the workpiece is rotated around one or more axes in its coordinate frame, and the workpiece has at least one correct placement at the placement target, the user may specify each rotation as a smaller angle than a complete rotation. In some cases, the range of possible rotations may be displayed as a series of images (e.g., at a graphical display associated with a robot teach pendant).
In some embodiments, the present disclosure may provide a method for calculating the greatest range of possible poses of a workpiece feature and of a robot and/or robot end effector, each relative to the other, for contacting the workpiece feature with the robot end effector. The method may include determining a first set of the allowable range of poses for the workpiece feature, where each of the allowable poses results in substantially equivalent performance for contacting the workpiece feature with a robot and/or robot end effector. The method may include determining a second set of the allowable range of poses for the robot and/or robot end effector, where each of the allowable poses results in substantially equivalent performance for contacting the workpiece feature with a robot and/or robot end effector. The method may also include selecting either the first set or the second set. This may be determined based upon which set allows the greatest range of poses with substantially equivalent performance for contacting the workpiece feature with the robot and/or robot end effector.
In some embodiments, the present disclosure may provide a method for calculating the greatest range of poses of a workpiece and of a workpiece placement target, each relative to the other, for placing the workpiece at a workpiece placement target. The method may include determining a first set of the allowable range of poses for the workpiece, where each of the allowable poses may result in substantially equivalent performance for placing the workpiece at the workpiece placement target. The method may further include determining a second set of the allowable range of poses for the workpiece placement target, where each of the allowable poses may result in substantially equivalent performance for placing the workpiece at the workpiece placement target. The method may also include selecting either the first set or the second set, determined by which set may allow for the greatest range of poses with substantially equivalent performance for placing the workpiece at a workpiece placement target.
In some embodiments, the present disclosure may provide a method for specifying the allowable range of poses of a robot end effector relative to a workpiece, for the purpose of contacting the workpiece to transport the workpiece with the robot end effector. The method may include graphically displaying the workpiece, the robot and/or the robot end effector, and/or the axes of the workpiece coordinate frame, where the robot end effector may be displayed in a pose for contacting the workpiece. The user may be allowed to specify the magnitude of an end effector translation along each axis of the workpiece coordinate frame for which the contact between the robot end effector and the workpiece has acceptable performance for the purpose of contacting the workpiece to transport the workpiece with the robot end effector. In some cases, the specified translation along each axis may be displayed as a series of images (e.g., at a graphical display associated with a robot teach pendant).
In some embodiments, the present disclosure may provide a method for specifying an allowable range of poses of a robot end effector relative to a workpiece, for the purpose of contacting a workpiece to transport the workpiece with the robot end effector. The method may include graphically displaying the workpiece, the robot end effector, and the axes of the robot end effector coordinate frame, where the robot and/or robot end effector may be displayed in a pose for contacting the workpiece. In some cases the user may be allowed to specify the magnitude of an end effector rotation around the origin of the robot end effector coordinate frame, relative to one or more axes of the robot end effector coordinate frame, for which the contact between the robot end effector and the workpiece has acceptable performance for the purpose of contacting the workpiece to transport the workpiece with the robot end effector. In some cases, one of the axes of the robot end effector coordinate frame may be normal to a contact point at the surface of the workpiece. In some cases, the specified rotation relative to each axis may be displayed as a series of images (e.g., at a graphical display associated with a robot teach pendant).
In some embodiments, the present disclosure may provide a method for specifying the allowable range of poses of a robot end effector for the purpose of picking up a workpiece. The method may include determining the degree of symmetry of the pick feature on the workpiece. The degree of symmetry may be continuous, discrete, and/or asymmetrical, for rotation of the workpiece around each of the axes of the workpiece's coordinate frame. If the determined symmetry is discrete, the method may include determining the increment of the discrete rotation around the corresponding axis. The method may also include determining the degree of symmetry of the robot end effector, where the degree of symmetry may be continuous, discrete, and/or asymmetrical, for rotation of the robot end effector around each of the axes of the robot end effector's coordinate frame. If the determined symmetry is discrete, the method may include determining the increment of the discrete rotation around the corresponding axis. The method may also include determining a first valid pose for the workpiece relative to the robot end effector for the purpose of picking up the workpiece. For an axis of workpiece's coordinate frame that is coincident with an axis of the robot end effector's coordinate frame when the workpiece is in the first valid pose, the method may include determining a range of valid robot end effector poses by allowing rotation of the end effector according to the degree of symmetry determined for either the workpiece around the an axis of the workpiece's coordinate frame or for the robot end effector around the an axis of the robot end effector's coordinate frame.
In some embodiments, the present disclosure may provide a method for specifying the allowable range of poses of a workpiece for the purpose of placing the workpiece at a placement target. The method may include determining the degree of symmetry of the place feature on the workpiece, where the degree of symmetry may be continuous, discrete, or asymmetrical, for rotation of the workpiece around each of the axes of the workpiece's coordinate frame, and where, if the determined symmetry is discrete, determining the increment of the discrete rotation around the corresponding axis. The method may also include determining the degree of symmetry of the placement target, where the degree of symmetry may be continuous, discrete, or asymmetrical, for rotation of the placement target around each of the axes of the place target's coordinate frame, and where, if the determined symmetry is discrete, determining the increment of the discrete rotation around the corresponding axis. The method may further include determining a first valid pose for the workpiece relative to the placement target for the purpose of placing the workpiece. For an axis of workpiece's coordinate frame that may be coincident with an axis of the placement target's coordinate frame when the workpiece is in the first valid pose, the method may include determining a range of valid workpiece poses by allowing rotation of the workpiece according to the degree of symmetry determined for either the workpiece around the an axis of the workpiece's coordinate frame or for the placement target around the an axis of the placement target's coordinate frame.
In some embodiments, the present disclosure may provide a system for picking up and placing workpieces. The system may include one or more of a robot, a robot end effector, a sensor, a robot controller, and/or a robot teach pendant in communication with the robot controller. In some embodiments, the sensor may be configured to detect the poses of one or more workpieces relative to the robot, and the robot controller may be configured to select a workpiece as detected by the sensor. The robot controller may be further configured to select from a range of possible end effector picking poses for the purpose of picking up the workpiece. The robot controller may be further configured to select from a range of possible end effector placing poses for the purpose of placing the workpiece at a workpiece placement target. The robot controller may be further configured to prompt a user for information concerning the symmetry of the end effector in order to generate the range of possible end effector picking poses. The user may be prompted for information concerning the symmetry of the end effector using a graphical user interface (e.g., on the robot teach pendant). The robot controller may be further configured to prompt a user for information concerning the symmetry of the workpiece pick features in order to generate the range of possible end effector picking poses. The user may be prompted for information concerning the symmetry of the end effector on the robot teach pendant. The robot controller may be further configured to select from a range of possible end effector placement poses for the purpose of placing the workpiece at a workpiece placement target.
In some embodiments, the robot controller may be further configured to prompt a user for information concerning the symmetry of the workpiece place features in order to generate the range of possible end effector placement poses. The user may be prompted for information concerning the symmetry of the end effector on the robot teach pendant. The robot controller may be further configured to prompt a user for information concerning the symmetry of the workpiece place target in order to generate the range of possible end effector placement poses. The user may be prompted for information concerning the symmetry of the end effector on the robot teach pendant. The robot controller may be further configured to calculate a path from a selected end effector picking pose to a selected end effector placing pose. The path between the selected end effector picking pose and the selected end effector placing pose may be optimized for the highest probability of a successful workpiece pick and a successful workpiece placement.
As will be appreciated by one skilled in the art, aspects of the present disclosure may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a circuit,” “module” or “system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having computer readable program code embodied thereon.
Any combination of one or more computer readable medium(s) may be utilized. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a nonexhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable signal medium may include a propagated data signal with computer readable program code embodied therein, for example, in baseband or as part of a carrier wave.
Such a propagated signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations for aspects of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
Aspects of the present disclosure are described below with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks.
The computer program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may, but not always, represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The terminology used herein is for the purpose of describing particular implementations only and is not intended to be limiting of the disclosure. As used herein, the singular forms “a”, “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps (not necessarily in a particular order), operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps (not necessarily in a particular order), operations, elements, components, and/or groups thereof.
The corresponding structures, materials, acts, and equivalents of all means or step plus function elements that may be in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of the present disclosure has been presented for purposes of illustration and description, but is not intended to be exhaustive or limited to the disclosure in the form disclosed. Many modifications, variations, substitutions, and any combinations thereof will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the disclosure. The implementation(s) were chosen and described in order to best explain the principles of the disclosure and the practical application, and to enable others of ordinary skill in the art to understand the disclosure for various implementation(s) with various modifications and/or any combinations of implementation(s) as are suited to the particular use contemplated.
A number of implementations of the invention have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the invention.
For example, in some implementations, the robot may be any mechanical system configured to pick up at least one object and move it from one place to another.
In some implementations, the graphical user interface may be implemented in places other than a robot teach pendant, including but not limited to a computer display, a mobile device, or a tablet computer. Moreover, while in some implementations “a processor” may be used, it should be noted that this may include, but is not limited to, a processor associated with or separate from a robot controller, for example, a single processor separate from or included within the controller, a plurality of processors separate from or included within the controller, etc. In some cases, one processor located in one device may perform certain operations and another processor located in a physically separate area may perform other operations. Additionally and/or alternatively, some or all of the operations may be performed by a processor or group of processors located in a single device, such as those associated with the robotic systems included herein.
Having thus described the disclosure of the present application in detail and by reference to implementation(s) thereof, it will be apparent that modifications, variations, and any combinations of implementation(s) (including any modifications, variations, substitutions, and combinations thereof) are possible without departing from the scope of the disclosure defined in the appended claims.