Patent Application
20220234209
The field of the invention relates, generally, to operation of potentially dangerous machinery and, in particular, to collaborative human-robot applications.
Traditional machinery for manufacturing and other industrial applications has been supplanted by, or supplemented with, new forms of automation that save costs, increase productivity and quality, eliminate dangerous, laborious, or repetitive work, and/or augment human capability. For example, industrial robots possess strength, speed, reliability, and lifetimes that may far exceed human potential. The recent trend toward increased human-robot collaboration in manufacturing workcells imposes particularly stringent requirements on robot performance and capabilities. Conventional industrial robots are dangerous to humans and are usually kept separate from humans through guarding—e.g., robots may be surrounded by a cage with doors that, when opened, cause an electrical circuit to place the machinery in a safe state. Other approaches involve light curtains or two-dimensional (2D) area sensors that slow down or shut off the machinery when humans approach it or cross a prescribed distance threshold. These systems disadvantageously constrain collaborative use of the workspace.
On the other hand, having humans and robots operate in the same workspace places additional demands on robot performance. Both may change position and configuration in rapid and unexpected ways, putting additional performance requirements on the robot's response times, kinematics, and dynamics. Typical industrial robots are fixed, but nonetheless have powerful arms that can cause injury over a wide “envelope” of possible movement trajectories; having knowledge of these trajectories in spaces where humans are present is thus fundamental to safe operation.
In general, robot arms comprise a number of mechanical links connected by revolute and prismatic joints that can be precisely controlled, and a controller coordinates all of the joints to achieve trajectories that are determined and programmed by an automation or manufacturing engineer for a specific application. Systems that can accurately control the robot trajectory are essential for safety in collaborative human-robot applications. However, the accuracy of industrial robots is limited by factors such as manufacturing tolerances (e.g., relating to fabrication of the mechanical arm), joint friction, drive nonlinearities, and tracking errors of the control system. In addition, backlash or compliances in the drives and joints of these robot manipulators can limit the positioning accuracy and the dynamic performance of the robot arm.
Kinematic definitions of industrial robots, which describe the total reachable volume (or “joint space”) of the manipulator, are derived from the individual robot link geometry and their assembly. A dynamic model of the robot is generated by taking the kinematic definition as an input, adding to it information about the speeds, accelerations, forces, range-of-motion limits, and moments that the robot is capable of at each joint interface, and applying a system identification procedure to estimate the robot dynamic model parameters. Accurate dynamic robot models are needed in many areas, such as mechanical design, workcell and performance simulation, control, diagnosis, safety and risk assessment, and supervision. For example, dexterous manipulation tasks and interaction with the environment, including humans in the vicinity of the robot, may demand accurate knowledge of the dynamic model of the robot for a specific application. Once estimated, robot model parameters can be used to compute stopping distances and other safety-related quantities. Because robot links are typically large, heavy metal castings fitted with motors, they have significant inertia while moving. Depending on the initial speed, payload, and robot orientation, a robot can take a significant time (and travel a great distance, many meters is not unusual) to stop after a stop command has been issued.
Dynamic models of robot arms are represented in terms of various inertial and friction parameters that are either measured directly or determined experimentally. While the model structure of robot manipulators is well known, the parameter values needed for system identification are not always available, since dynamic parameters are rarely provided by the robot manufacturers and often are not directly measurable. Determination of these parameters from computer-aided design (CAD) data or models may not yield a complete representation because they may not include dynamic effects like joint friction, joint and drive elasticities, and masses introduced by additional equipment such as end effectors, workpieces, or the robot dress package.
One important need for effective robotic system identification is in the estimation of joint acceleration characteristics and robot stopping distances for the safety rating of robotic equipment. As humans physically approach robotic arms, a safety system can engage and cut or reduce power to the arm, but robot inertia can keep the robot arm moving. The effective stopping distance (measured from the engagement of the safety system, such as a stopping command) is an important input for determining the safe separation distance from the robot arm given inertial effects. Similarly, all sensor systems include some amount of latency, and joint acceleration characteristics determine how the robot's state can change between measurement and application of control output. Robot manufacturers usually provide curves or graphs showing stopping distances and times, but these curves can be difficult to interpret, may be sparse and of low resolution, tend to reflect specific loads, and typically do not include acceleration or indicate the robot position at the time of engaging the stop. An improved approach to modeling and predicting robot dynamics under constraints and differing environmental conditions (such as varying payloads and end effectors) is set forth in U.S. Pat. No. 11,254,004, the entire disclosure of which is hereby incorporated by reference.
Even with robot behavior fully modeled, however, safe operation for a given application—particularly if that application involves interaction with or proximity to humans depends on the spatial arrangement of the workspace, the relative positions of the robot and people or vulnerable objects, the task being performed, and robot stopping capabilities. Moreover, overall responsibility for safe operation may be shared among safety-rated and non-safety-rated robot components (since as a practical matter not all components can be safety-rated, nor do they need to be) as well as the workspace-level monitoring system described, for example, in U.S. Patent Publ. No. 2021/0053227. As explained in that application, a non-safety-rated speed governor may be used in conjunction with safety-rated monitoring, which is triggered only when the robot operating velocity exceeds a threshold. Safety-rated monitoring may be implemented via the workspace-level monitoring system, with the objective of predicting possible violations of predefined protective safety distances and avoiding them in the least disruptive way—e.g., slowing the robot rather than shutting it down.
While effective, this configuration places demands on the workspace-level monitoring system that may be more efficiently or conveniently assigned to robot-level control. Doing so, however, may not achieve the desired efficiency benefits if an entire safety-rated monitoring system must be built into the robot. Accordingly, there is a need for systems and techniques that achieve compliance with rigorous safety standards while affording flexibility in terms of using safety-rated and non-safety-rated components.
Embodiments of the present invention utilize robot controllers with safety-rated and non-safety rated components in a manner that achieves compliance with industrial safety standards. In particular, embodiments described herein permit use of non-safety-rated speed or velocity governors so long as the robot is configured to engage safety-rated monitoring when a predetermined speed or velocity is reached. For example, safety-rated monitoring may be implemented by a workspace-level system with which the robot controller communicates using a safety-rated signal. In some embodiments, it is not necessary to actually register that the predetermined speed or velocity is reached or receive a communication from the robot; rather, it may be more operationally convenient to simply wait a certain amount of time (e.g., the smallest interval following robot activation after which the robot could have reached the predetermined speed or velocity given the task being performed) and then trigger safety-rated monitoring, e.g., at the workspace level. Although the ensuing discussion focuses on industrial robots, it should be understood that the present invention and the approaches described herein are applicable to any type of controlled industrial machinery whose operation occurs in the vicinity of, and can pose a danger to, human workers.
In various embodiments, the spatial regions potentially occupied by any portion of the robot (or other machinery) and the human operator within a defined time interval or during performance of all or a defined portion of a task or an application are generated, e.g., calculated dynamically and, if desired, represented visually. These “potential occupancy envelopes” (POEs) may be based on the states (e.g., the current and expected positions, velocities, accelerations, geometry and/or kinematics) of the robot and the human operator (e.g., in accordance with the ISO 13855 standard, “Positioning of safeguards with respect to the approach speeds of parts of the human body”). POEs may be computed based on a simulation of the robot's performance of a task, with the simulated trajectories of moving robot parts (including workpieces) establishing the three-dimensional (3D) contours of the POE in space. Alternatively, POEs may be obtained based on observation (e.g., using 3D sensors) of the robot as it performs the task, with the observed trajectories used to establish the POE contours.
In some embodiments, a “keep-in” zone and/or a “keep-out” zone associated with the robot can be defined, e.g., based on the POEs of the robot and human operator. In the former case, operation of the robot is constrained so that all portions of the robot and workpieces remain within the spatial region defined by the keep-in zone. In the latter case, operation of the robot is constrained so that no portions of the robot and workpieces penetrate the keep-out zone. Based on the POEs of the robot and human operator and/or the keep-in/keep-out zones, movement of the robot during physical performance of the activity may be restricted in order to ensure safety.
In addition, the workspace parameters, such as the dimensions thereof, the workflow, the locations of the resources (e.g., the workpieces or supporting equipment), etc. can be modeled based on the computed POEs, thereby achieving high productivity and spatial efficiency while ensuring safety of the human operator. In one embodiment, the POEs of the robot and the human operator are both presented on a local display (a screen, a VR/AR headset, etc., e.g., as described in U.S. Patent Publ. No. 2020/0331155, filed on Jul. 2, 2020, the entire disclosure of which is hereby incorporated by reference) and/or communicated to a smartphone or tablet application for display thereon; this allows the human operator to visualize the space that is currently occupied or will be potentially occupied by the robot or the human operator, thereby enabling the operator to plan motions efficiently around the POE and further ensuring safety.
In various embodiments, one or more two-dimensional (2D) and/or three-dimensional (3D) imaging sensors are employed to scan the robot, human operator and/or workspace during actual execution of the task. Based thereon, the POEs of the robot and the human operator can be updated in real-time and provided as feedback to adjust the state (e.g., position, orientation, velocity, acceleration, etc.) of the robot and/or the modeled workspace. In some embodiments, the scanning data is stored in memory and can be used as an input when modeling the workspace in the same human-robot collaborative application next time. In some embodiments, robot state can be communicated from the robot controller, and subsequently validated by the 2D and/or 3D imaging sensors. In other embodiments, the scanning data may be exported from the system in a variety of formats for use in other CAD software. In still other embodiments, the POE is generated by simulating performance (rather than scanning actual performance) of a task by a robot or other machinery.
Additionally or alternatively, a protective separation distance (PSD) defining the minimum distance separating the robot from the operator and/or other safety-related entities can be computed. Again, the PSD may be continuously updated based on the scanning data of the robot and/or human operator acquired during execution of the task. In one embodiment, information about the computed PSD is combined with the POE of the human operator; based thereon, an optimal path of the robot in the workspace can then be determined.
Accordingly, in a first aspect, the invention pertains to a system for spatially modeling a workspace containing machinery (e.g., a robot). In various embodiments, the system comprises a controller for the machinery, the controller having a safety-rated component and a non-safety-rated component; an object-monitoring system configured to computationally generate a first potential occupancy envelope for the machinery and a second potential occupancy envelope for a human operator when performing a task in the workspace, where the first and second potential occupancy envelopes spatially encompass movements performable by the machinery and the human operator, respectively, during performance of the task. The non-safety-rated component of the controller is configured to establish a velocity of the machinery and the object-monitoring system is configured to update the first potential occupancy envelope in response to a safety-rated signal from the controller or an elapsed time.
In various embodiments, the object-monitoring system is further configured to computationally detect a predetermined degree of proximity between the second potential occupancy envelope and the updated first potential occupancy envelope and to thereupon cause the controller to put the machinery in a safe state. For example, the predetermined degree of proximity may correspond to a protective separation distance. The object-monitoring system may be configured to (i) detect a current state of the machinery, (ii) compute parameters for putting the machinery in the safe state from the current state, and (iii) communicate the parameters to the controller when the predetermined degree of proximity is detected.
In various embodiments, the object-monitoring system is configured to, prior to operation of the machinery, compute default parameters for putting the machinery in a safe state, the parameters comprising a safe velocity and spatially defining a keep-in zone. The object-monitoring system may be further configured to send a trigger signal to the controller to put the machinery into the safe state in accordance with the default parameters when the predetermined degree of proximity is detected.
In some embodiments, the safety-rated component of the controller is further configured to report when the machinery is in a safe state. For example, the safe state may correspond to a safe reduction in velocity or cessation of operation.
The system may further include a computer vision system for monitoring the machinery and the human operator, in which case the object-monitoring system may be configured to reduce or enlarge a size of the first potential occupancy envelope in response to movement of the operator detected by the computer vision system.
In some embodiments, the parameters are communicated to the non-safety-rated component of the controller. The object-monitoring system may be further configured to communicate to the controller that the machinery may be taken out of the safe state in accordance with an enlarged potential occupancy envelope. In some implementations, the safety-rated component of the controller is configured to enforce the reduced or enlarged first potential occupancy envelope as a keep-in zone.
In various embodiments, the object-monitoring system is configured to update the first potential occupancy envelope in response to an elapsed time corresponding to an expected time for the machinery to reach a predetermined velocity.
In another aspect, the invention relates to system for enforcing safety in a workspace containing machinery (e.g., a robot). In various embodiments, the system comprises a controller for the machinery, where the controller has a safety-rated component and a non-safety-rated component; and an object-monitoring system configured to computationally generate a first potential occupancy envelope for the machinery and a second potential occupancy envelope for a human operator when performing a task in the workspace, where the first and second potential occupancy envelopes spatially encompass movements performable by the machinery and the human operator, respectively, during performance of the task. The object-monitoring system is configured to detect an unsafe condition based on the first and second potential occupancy envelopes, and thereupon signal the safety-rated component of the controller to enforce a safety condition.
Signaling the safety-rated component of the controller may comprise communicating parameters for putting the machinery in the safe state from a current state or instructing the controller to enforce pre-stored default safety parameters. The object-monitoring system may be configured to further signal the safety-rated component of the controller after a delay. In some embodiments, the delay corresponds to the expected time for the machinery to enter a safe state. The further signaling may comprise communicating, to the safety-rated controller component, safety parameters and a command to operate the machinery within the parameters.
In various embodiments, the object-monitoring system is further configured to await an acknowledgment from the safety-rated controller component that the machinery is being operated in accordance with parameters corresponding to a safe state. The acknowledgment may include the parameters and the object-monitoring system may be further configured to responsively update the first potential occupancy envelope in accordance with the parameters. The object-monitoring system may be further configured to cause the machinery to safely cease operation if the acknowledgment is not received within the delay.
In still another aspect, the invention relates to a method of spatially modeling a workspace containing machinery. In various embodiments, the method comprises providing a controller for the machinery, the controller having a safety-rated component and a non-safety-rated component; computationally generating a first potential occupancy envelope for the machinery and a second potential occupancy envelope for a human operator when performing a task in the workspace, the first and second potential occupancy envelopes spatially encompassing movements performable by the machinery and the human operator, respectively, during performance of the task; causing the non-safety-rated component of the controller to establish a velocity of the machinery; and computationally updating the first potential occupancy envelope in response to a safety-rated signal from the controller or an elapsed time.
Still another aspect of the invention pertains to a method of enforcing safety in a workspace containing machinery. In various embodiments, the method comprises providing a controller for the machinery, the controller having a safety-rated component and a non-safety-rated component; computationally generating a first potential occupancy envelope for the machinery and a second potential occupancy envelope for a human operator when performing a task in the workspace, where the first and second potential occupancy envelopes spatially encompass movements performable by the machinery and the human operator, respectively, during performance of the task; and computationally detecting an unsafe condition based on the first and second potential occupancy envelopes, and thereupon electronically signaling the safety-rated component of the controller to enforce a safety condition.
In general, as used herein, the term “robot” means any type of controllable industrial equipment for performing automated operations—such as moving, manipulating, picking and placing, processing, joining, cutting, welding, etc.—on workpieces. The term “substantially” means±10%, and in some embodiments, ±5%. In addition, reference throughout this specification to “one example,” “an example,” “one embodiment,” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the example is included in at least one example of the present technology. Thus, the occurrences of the phrases “in one example,” “in an example,” “one embodiment,” or “an embodiment” in various places throughout this specification are not necessarily all referring to the same example. Furthermore, the particular features, structures, routines, steps, or characteristics may be combined in any suitable manner in one or more examples of the technology. The headings provided herein are for convenience only and are not intended to limit or interpret the scope or meaning of the claimed technology.
In the drawings, like reference characters generally refer to the same parts throughout the different views. Also, the drawings are not necessarily to scale, with an emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the present invention are described with reference to the following drawings, in which:
The following discussion describes an integrated system and methods for fully modeling and/or computing in real time the robot dynamics and/or human activities in a workspace for safety. In some cases, this involves semantic analysis of a robot in the workspace and identification of the workpieces with which it interacts. It should be understood, however, that these various elements may be implemented separately or together in desired combinations; the inventive aspects discussed herein do not require all of the described elements, which are set forth together merely for ease of presentation and to illustrate their interoperability. The system as described represents merely one embodiment.
Refer first to
In various embodiments, data obtained by each of the sensors 1021-3 is transmitted to a control system 112. Based thereon, the control system 112 may computationally generate a 3D spatial representation (e.g., voxels) of the workspace 100, recognize the robot 106, human operator and/or workpiece handled by the robot and/or human operator, and track movements thereof as further described below. In addition, the sensors 1021-3 may be supported by various software and/or hardware components 1141-3 for changing the configurations (e.g., orientations and/or positions) of the sensors 1021-3; the control system 112 may be configured to adjust the sensors so as to provide optimal coverage of the monitored area in the workspace 100. The volume of space covered by each sensor—typically a solid truncated pyramid or solid frustum may be represented in any suitable fashion, e.g., the space may be divided into a 3D grid of small (5 cm, for example) voxels or other suitable form of volumetric representation. For example, a 3D representation of the workspace 100 may be generated using 2D or 3D ray tracing. This ray tracing can be performed dynamically or via the use of precomputed volumes, where objects in the workspace 100 are previously identified and captured by the control system 112. For convenience of presentation, the ensuing discussion assumes a voxel representation, and the control system 112 maintains an internal representation of the workspace 100 at the voxel level.
The CPU 205 is typically a microprocessor, but in various embodiments may be a microcontroller, peripheral integrated circuit element, a CSIC (customer-specific integrated circuit), an ASIC (application-specific integrated circuit), a logic circuit, a digital signal processor, a programmable logic device such as an FPGA (field-programmable gate array), PLD (programmable logic device), PLA (programmable logic array), RFID processor, graphics processing unit (GPU), smart chip, or any other device or arrangement of devices that is capable of implementing the steps of the processes of the invention.
The system memory 210 may store a model of the machinery characterizing its geometry and kinematics and its permitted movements in the workspace. The model may be obtained from the machinery manufacturer or, alternatively, generated by the control system 112 based on the scanning data acquired by the sensor system 101. In addition, the memory 210 may store a safety protocol specifying various safety measures such as speed restrictions of the machinery in proximity to the human operator, a minimum separation distance between the machinery and the human, etc. In some embodiments, the memory 210 contains a series of frame buffers 235, i.e., partitions that store, in digital form (e.g., as pixels or voxels, or as depth maps), images obtained by the sensors 1021-3; the data may actually arrive via I/O ports 227 and/or transceiver 225 as discussed above.
The system memory 210 contains instructions, conceptually illustrated as a group of modules, that control the operation of CPU 205 and its interaction with the other hardware components. An operating system 240 (e.g., Windows or Linux) directs the execution of low-level, basic system functions such as memory allocation, file management and operation of the mass storage device 212. At a higher level, and as described in greater detail below, an analysis module 242 may register the images acquired by the sensor system 101 in the frame buffers 235, generate a 3D spatial representation (e.g., voxels) of the workspace and analyze the images to classify regions of the monitored workspace 100; an object-recognition module 243 may recognize the human and the machinery and movements thereof in the workspace based on the data acquired by the sensor system 101; a simulation module 244 may computationally perform at least a portion of the application/task performed by the machinery in accordance with the stored machinery model and application/task; a movement prediction module 245 may predict movements of the machinery and/or the human operator within a defined future interval (e.g., 0.1 sec, 0.5 sec, 1 sec, etc.) based on, for example, the current state (e.g., position, orientation, velocity, acceleration, etc.) thereof; a mapping module 246 may map or identify the POEs of the machinery and/or the human operator within the workspace; a state determination module 247 may determine an updated state of the machinery such that the machinery can be operated in a safe state; a path determination module 248 may determine a path along which the machinery can perform the activity; and a workspace modeling module 249 may model the workspace parameters (e.g., the dimensions, workflow, locations of the equipment and/or resources). The result of the classification, object recognition and simulation as well as the POEs of the machinery and/or human, the determined optimal path and workspace parameters may be stored in a space map 250, which contains a volumetric representation of the workspace 100 with each voxel (or other unit of representation) labeled, within the space map, as described herein. Alternatively, the space map 250 may simply be a 3D array of voxels, with voxel labels being stored in a separate database (in memory 210 or in mass storage 212).
In addition, the control system 112 may communicate with the robot controller 108 to control operation of the machinery in the workspace 100 (e.g., performing a task/application programmed in the controller 108 or the control system 112) using conventional control routines collectively indicated at 252. As explained below, the configuration of the workspace may well change over time as persons and/or machines move about; the control routines 252 may be responsive to these changes in operating machinery to achieve high levels of safety. All of the modules in system memory 210 may be coded in any suitable programming language, including, without limitation, high-level languages such as C, C++, C#, Java, Python, Ruby, Scala, and Lua, utilizing, without limitation, any suitable frameworks and libraries such as TensorFlow, Keras, PyTorch, Caffe or Theano. Additionally, the software can be implemented in an assembly language and/or machine language directed to the microprocessor resident on a target device.
When a task/application involves human-robot collaboration, it may be desired to model and/or compute, in real time, the robot dynamics and/or human activities and provide safety mapping of the robot and/or human in the workspace 100. Mapping a safe and/or unsafe region in human-robot collaborative applications, however, is a complicated process because, for example, the robot state (e.g., current position, velocity, acceleration, payload, etc.) that represents the basis for extrapolating to all possibilities of the robot speed, load, and extension is subject to abrupt change. These possibilities typically depend on the robot kinematics and dynamics (including singularities and handling of redundant axes, e.g., elbow-up or elbow-down configurations) as well as the dynamics of the end effector and workpiece. Moreover, the safe region may be defined in terms of a degree rather than simply as “safe.” The process of modeling the robot dynamics and mapping the safe region, however, may be simplified by assuming that the robot's current position is fixed and estimating the region that any portion of the robot may conceivably occupy within a short future time interval only. Thus, various embodiments of the present invention include approaches to modeling the robot dynamics and/or human activities in the workspace 100 and mapping the human-robot collaborative workspace 100 (e.g., calculating the safe and/or unsafe regions) over short intervals based on the current states (e.g., current positions, velocities, accelerations, geometries, kinematics, expected positions and/or orientations associated with the next action in the task/application) associated with the machinery (including the robot 106 and/or other industrial equipment) and the human operator. In addition, the modeling and mapping procedure may be repeated (based on, for example, the scanning data of the machinery and the human acquired by the sensor system 101 during performance of the task/application) over time, thereby effectively updating the safe and/or unsafe regions on a quasi-continuous basis in real time.
To model the robot dynamics and/or human activities in the workspace 100 and map the safe and/or unsafe regions, in various embodiments, the control system 112 first computationally generates a 3D spatial representation (e.g., as voxels) of the workspace 100 where the machinery (including the robot 106 and auxiliary equipment), workpiece and human operator are based on, for example, the scanning data acquired by the sensor system 101. In addition, the control system 112 may access the memory 210 or mass storage 212 to retrieve a model of the machinery characterizing the geometry and kinematics of the machinery and its permitted movements in the workspace. The model may be obtained from the robot manufacturer or, alternatively, generated by the control system 112 based on the scanning data acquired by the sensor system prior to mapping the safe and/or unsafe regions in the workspace 100. Based on the machinery model and the currently known information about the machinery, a spatial POE of the machinery can be estimated. As a spatial map, the POE may be represented in any computationally convenient form, e.g., as a cloud of points, a grid of voxels, a vectorized representation, or other format. For convenience, the ensuing discussion will assume a voxel representation.
In one embodiment, the POE 306 represents a 3D region which the robot and end-effector may occupy before being brought to a safe state. Thus, in this embodiment, the time interval for computing the POE 306 is based on the time required to bring the robot to the safe state. For example, referring again to
In addition, the POE 306 may be refined based on safety features of the robot 106; for example, the safety features may include a safety system that initiates a protective stop even when the velocity or acceleration of the robot is not known. Knowing that a protective stop has been initiated and its protective stop input is being held may effectively truncate the POE 306 of the robot (since the robot will only decelerate until a complete stop is reached). In one embodiment, the POE 306 is continuously updated at fixed time intervals (thereby changing the spatial extent thereof in a stepwise manner) during deceleration of the robot; thus, if the time intervals are sufficiently short, the POE 306 is effectively updated on a quasi-continuous basis in real time.
In various embodiments, the POE of the machinery is more narrowly defined to correspond to the execution of a task or an application, i.e., all points that the robot may or can reach during performance of the task/application. This “task-level” or “application-level” POE may be estimated based on known robot operating parameters and the task/application program executed by the robot controller. For example, the control system 112 may access the memory 210 and/or storage 212 to retrieve the model of the machinery and the task/application program that the machinery will execute. Based thereon, the control system 112 may simulate operation of the machinery in a virtual volume (e.g., defined as a spatial region of voxels) in the workspace 100 for performing the task/application. The simulated machinery may sweep out a path in the virtual volume as the simulation progresses; the voxels that represent the spatial volume encountered by the machinery for performing the entire task/application correspond to a static task-level or application-level POE. In addition, because the machinery dynamically changes its trajectory (e.g., the pose, velocity and acceleration) during execution of the task/application, a dynamic POE may be defined as the spatial region that the machinery, as it performs the task/application, may reach from its current position within a predefined time interval. The dynamic POE may be determined based on the current state (e.g., the current position, current velocity and current acceleration) of the machinery and the programmed movements of the machinery in performing the task/application beginning at the current time. Thus, the dynamic POE may vary throughout performance of the entire task/application—i.e., different sub-tasks (or sub-applications) may correspond to different POEs. In one embodiment, the POE associated with each sub-task or sub-application has a timestamp representing its temporal relation with the initial POE associated with the initial position of the machinery when it commences the task/application. The overall task-level or application-level POE (i.e., the static task-level or application-level POE) then corresponds to the union of all possible sub-task-level or sub-application-level POEs (i.e., the dynamic task-level or application-level POEs).
In some embodiments, parameters of the machinery are not known with sufficient precision to support an accurate simulation; in this case, the actual machinery may be run through the entire task/application routine and all joint positions at every point in time during the trajectory are recorded (e.g., by the sensory system 101 and/or the robot controller). Additional characteristics that may be captured during the recording include (i) the position of the tool-center-point in X, Y, Z, R, P, Y coordinates; (ii) the positions of all robot joints in joint space, J1, J2, J3, J4, J5, J6, . . . Jn; and (iii) the maximum achieved speed and acceleration for each joint during the desired motion. The control system 112 may then computationally create the static and/or dynamic task-level (or application-level) POE based on the recorded geometry of the machinery. For example, if the motion of the machinery is captured optically using cameras; the control system 112 may utilize a conventional computer-vision program to spatially map the motion of the machinery in the workspace 100 and, based thereon, create the POE of the machinery. In one embodiment, the range of each joint motion is profiled, and a safety-rated soft-axis limiting in joint space by the robot controller can bound the allowable range that each individual axis can move, thereby truncating the POE of the machinery as the maximum and minimum joint position for a particular application. In this case, the safety-rated limits can be enforced by the robot controller, resulting in a controller-initiated protective stop when, for example, (i) the robot position exceeds the safety-rated limits due to robot failure, (ii) an external position-based application profiling is incomplete, (iii) any observations were not properly recorded, and/or (iv) the application itself was changed to encompass a larger volume in the workspace without recharacterization.
A simple example of the task/application-level POE can be seen in
In
In various embodiments, the control system 112 facilitates operation of the machinery based on the determined POE thereof. For example, during performance of a task, the sensor system 101 may continuously monitor the position of the machinery, and the control system 112 may compare the actual machinery position to the simulated POE. If a deviation of the actual machinery position from the simulated POE exceeds a predetermined threshold (e.g., 1 meter), the control system 112 may change the pose (position and/or orientation) and/or the velocity (e.g., to a full stop) of the robot for ensuring human safety. Additionally or alternatively, the control system 112 may preemptively change the pose and/or velocity of the robot before the deviation actually exceeds the predetermined threshold. For example, upon determining that the deviation gradually increases and is approaching the predetermined threshold during execution of the task, the control system 112 may preemptively reduce the velocity of the machinery; this may avoid the situation where the inertia of the machinery causes the deviation to exceed the predetermined threshold.
To fully map the workspace 100 in a human-robot collaborative application, it may be desired to consider the presence and movement of the human operator in the vicinity of the machinery. Thus, in various embodiments, a spatial POE of the human operator that characterizes the spatial region potentially occupied by any portion of the human operator is based on any possible or anticipated movements of the human operator within a defined time interval or during performance of a task or an application; this region is computed and mapped in the workspace. As used herein, the term “possible movements” or “anticipated movements” of the human includes a bounded possible location within the defined time interval based, for example, on ISO 13855 standards defining expected human motion in a hazardous setting. To compute/map the POE of the human operator, the control system 112 may first utilize the sensor system 101 to acquire the current position and/or pose of the operator in the workspace 100. In addition, the control system 112 may determine (i) the future position and pose of the operator in the workspace using a well-characterized human model or (ii) all space presently or potentially occupied by any potential operator based on the assumption that the operator can move in any direction at a maximum operator velocity as defined by the standards such as ISO 13855. Again, the operator's position and pose can be treated as a moment frozen in space at the time of image acquisition, and the operator is assumed to be able to move in any direction with any speed and acceleration consistent with the linear and angular kinematics and dynamics of human motion in the immediate future (e.g., in a time interval, δt, after the image-acquisition moment), or at some maximum velocity as defined by the standards. For example, referring to
In some embodiments, the POE 602 of the human operator is refined by acquiring more information about the operator. For example, the sensor system 101 may acquire a series of scanning data (e.g., images) within a time interval Δt. By analyzing the operator's positions and poses in the scanning data and based on the time period Δt, the operator's moving direction, velocity and acceleration can be determined. This information, in combination with the linear and angular kinematics and dynamics of human motion, may reduce the potential distance reachable by the operator in the immediate future time δt, thereby refining the POE of the operator (e.g., POE 604 in
In addition, similar to the POE of the machinery above, the POE of the human operator can be established at an application/task level. For example, referring to
Further, the POE of the human operator may be truncated based on workspace configuration. For example, referring to
The robot-level POE (and/or application-level POE) of the machinery and/or the future-interval POE (and/or application-level POE) of the human operator may be used to show the operator where to stand and/or what to do during a particular part of the task using suitable indicators (e.g., lights, sounds, displayed visualizations, etc.), and an alert can be raised if the operator unexpectedly leaves the operator POE. In one embodiment, the POEs of the machinery and human operator are both presented on a local display or communicated to a smartphone or tablet application (or other methods, such as augmented reality (AR) or virtual reality (VR)) for display thereon. For example, referring to
In some embodiments, the machinery is operated based on the POE thereof, the POE of the human operator, and/or a safety protocol that specifies one or more safety measures (e.g., a minimum separation distance or a protective separation distance (PSD) between the machinery and the operator as further described below, a maximum speed of the machinery when in proximity to a human, etc.). For example, during performance of a particular task, the control system 112 may restrict or alter the robot operation based on proximity between the POEs of the robot and the human operator for ensuring that the safety measures in the protocol are satisfied. For example, upon determining that the POEs of the robot and the human operator in the next moment may overlap, the control system 112 may bring the robot to a safe state (e.g., having a reduced speed and/or a different pose), thereby avoiding a contact with the human operator in proximity thereto. The control system 112 may directly control the operation and state of the robot or, alternatively, may send instructions to the robot controller 108 that then controls the robotic operation/state based on the received instructions as further described below.
In addition, the degree of alternation of the robot operation/state may depend on the degree of overlap between the POEs of the robot and the operator. For example, referring again to
In various embodiments, based on the computed robot-level POE 804, future-interval POE 806 of the human operator, or dynamic and/or static application-level POEs 814, 816 of the machinery and human operator for performing a specific action or an entire task, the workspace parameter (such as the dimensions thereof, the workflow, the locations of the resources, etc.) can be modeled to achieve high productivity and spatial efficiency while ensuring safety of the human operator. For example, based on the static task-level POE 814 of the machinery and the largest computed POE 816 of the operator during execution of the task, the minimum dimensions of the workcell can be determined. In addition, the locations and/or orientations of the equipment and/or resources (e.g., the robot, conveyor belt, workpieces) in the workspace can be arranged such that they are easily reachable by the machinery and/or operator while minimizing the overlapped region between the POEs of the machinery and the operator in order to ensure safety. In one embodiment, the computed POEs of the machinery and/or human operator are combined with a conventional spatial modeling tool (e.g., supplied by DELMIA Global Operations or Tecnomatix) to model the workspace. For example, the POEs of the machinery and/or human operator may be used as input modules to the conventional spatial modeling tool so as to augment their capabilities to include the human-robot collaboration when designing the workspace and/or workflow of a particular task.
In various embodiments, the dynamic task-level POE of the machinery and/or the task-level POE of the operator is continuously updated during actual execution of the task; such updates can be reflected on the display 802. For example, during execution of the task, the sensor system 101 may periodically scan the machinery, human operator and/or workspace. Based on the scanning data, the poses (e.g., positions and/or orientation) of the machinery and/or human operator can be updated. In addition, by comparing the updated poses with the previous poses of the machinery and/or human operator, the moving directions, velocities and/or accelerations associated with the machinery and operator can be determined. In various embodiments, based on the updated poses, moving directions, velocities and/or accelerations, the POEs of the machinery and operator in the next moment (i.e., after a time increment) can be computed and updated. Additionally, as explained above, the POEs of the machinery and/or human operator may be updated by further taking into account next actions that are specified to be performed in the particular task.
In some embodiments, the continuously updated POEs of the machinery and the human operator are provided as feedback for adjusting the operation of the machinery and/or other setup in the workspace to ensure safety as further described below. For example, when the updated POEs of the machinery and the operator indicate that the operator may be too close to the robot (e.g., a distance smaller than the minimum separation distance defined in the safety protocol), either at present or within a fixed interval (e.g., the robot stopping time), a stop command may be issued to the machinery. In one embodiment, the scanning data of the machinery and/or operator acquired during actual execution of the task is stored in memory and can be used as an input when modeling the workflow of the same human-robot collaborative application in the workspace next time.
In addition, the computed POEs of the machinery and/or human operator may provide insights when determining an optimal path of the machinery for performing a particular task. For example, as further described below, multiple POEs of the operator may be computed based on his/her actions to be performed for the task. Based on the computed POEs of the human operator and the setup (e.g., locations and/or orientations) of the equipment and/or resources in the workspace, the moving path of the machinery in the workspace for performing the task can be optimized so as to maximize the productivity and space efficiency while ensuring safety of the operator.
In some embodiments, path optimization includes creation of a 3D “keep-in” zone (or volume) (i.e., a zone/volume to which the robot is restricted during operation) and/or a “keep-out” zone (or volume) (i.e., a zone/volume from which the robot is restricted during operation). Keep-in and keep-out zones restrict robot motion through safe limitations on the possible robot axis positions in Cartesian and/or joint space. Safety limits may be set outside these zones so that, for example, their breach by the robot in operation triggers a stop. Conventionally, robot keep-in zones are defined as prismatic bodies. For example, referring to
The machinery path determined based on prismatic volumes, however, may not be optimal. In addition, complex robot motions may be difficult to represent as prismatic volumes due to the complex nature of their surfaces and the geometry of the end effectors and workpieces mounted on the robot; as a result, the prismatic volume will be larger than necessary for safety. To overcome this challenge and optimize the moving path of the machinery for performing a task, various embodiments establish and store in memory the swept volume of the machinery (including, for example, robot links, end effectors and workpieces) throughout a programmed routine (e.g., a POE of the machinery), and then define the keep-in zone based on the POE as a detailed volume composed of, e.g., mesh surfaces, NURBS or T-spline solid bodies. That is, the keep-in zone may be arbitrary in shape and not assembled from base prismatic volumes. For example, referring to
As described above, the POE of the machinery may be static or dynamic, and may be robot-level or task-level. A static, robot-level POE represents the entire spatial region that the machinery may possibly reach within a specified time, and thus corresponds to the most conservative possible safety zone; a keep-in zone determined based on the static robot-level POE may not be truly a keep-in zone because the machinery's movements are not constrained. If the machinery is stopped or slowed down when a human reaches a prescribed separation distance from any outer point of this zone, the machinery's operation may be curtailed even when intrusions are distant from its near-term reach. A static, task-level POE reduces the volume or distance within which an intrusion will trigger a safety stop or slow down to a specific task-defined volume and consequently reduces potential robot downtime without compromising human safety. Thus, the keep-in zone determined based on the static, task-level POE of the machinery is smaller than that determined based on the static, robot-level POE. A dynamic, task-level or application-level POE of the machinery may further reduce the POE (and thereby the keep-in zone) based on a specific point in the execution of a task by the machinery. A dynamic task-level POE achieves the smallest sacrifice of productive robot activity while respecting safety guidelines.
Alternatively, the keep-in zone may be defined based on the boundary of the total swept volume 904 of the machinery during operation or slight padding/offset of the total swept volume 904 to account for measurement or simulation error. This approach may be utilized when, for example, the computed POE of the machinery is sufficiently large. For example, referring again to
In various embodiments, the actual or potential movement of the human operator is evaluated against the robot-level or application-level POE of the machinery to define the keep-in zone. Expected human speeds in industrial environments are referenced in ISO 13855:2010, ISO 61496-1:2012 and ISO 10218:2011. For example, human bodies are expected to move no faster than 1.6 m/s and human extremities are expected to move no faster than 2 m/s. In one embodiment, the points reachable by the human operator in a given unit of time is approximated by a volume surrounding the operator, which can be defined as the human POE as described above. If the human operator is moving, the human POE moves with her. Thus, as the human POE approaches the task-level POE of the robot, the latter may be reduced in dimension along the direction of human travel to preserve a safe separation distance. In one embodiment, this reduced task-level POE of the robot (which varies dynamically based on the tracked and/or estimated movement of the operator) is defined as a keep-in zone. So long as the robot can continue performing elements of the task within the smaller (and potentially shrinking) POE (i.e., keep-in zone), the robot can continue to operate productively; otherwise, it may stop. Alternatively, the dynamic task-level POE of the machinery may be reduced in response to an advancing human by slowing down the machinery as further described below. This permits the machinery to keep working at a slower rate rather than stopping completely. Moreover, slower machinery movement may in itself pose a lower safety risk.
In various embodiments, the keep-in and keep-out zones are implemented in the machinery having separate safety-rated and non-safety-rated control systems, typically in compliance with an industrial safety standard. Safety architectures and safety ratings are described, for example, in U.S. Patent Publ. No. 2020/0272123, entitled “Safety-Rated Processor System Architecture,” the entire contents of which are hereby incorporated by reference. Non-safety-rated systems, by contrast, are not designed for integration into safety systems (e.g., in accordance with the safety standard).
Operation of the safety-rated and non-safety-rated control systems is best understood with reference to the conceptual illustration of system organization and operation of
Typically, the robot controller 1004 itself does not have a safe way to govern (e.g., modify) the state (e.g., speed, position, etc.) of the robot; rather, it only has a safe way to enforce a given state. To govern and enforce the state of the robot in a safe manner, in various embodiments, an object-monitoring system (OMS) 1010 is implemented to cooperatively work with the safety-rated component 1006 and non-safety-rated component 1008 as further described below. In one embodiment, the OMS 1010 obtains information about objects from the sensor system 1001 and uses this sensor information to identify relevant objects in the workspace 1000. For example, OMS 1010 may, based on the information obtained from the sensor system (and/or the robot), monitor whether the robot is in a safe state (e.g., remains within a specific zone (e.g., the keep-in zone), stays below a specified speed, etc.), and if not, issues a safe-action command (e.g., stop) to the robot controller 1004.
For example, OMS 1010 may determine the current state of the robot and/or the human operator and computationally generate a POE for the robot and/or a POE for the human operator when performing a task in the workspace 1000. The POEs of the robot and/or human operator may then be transferred to the safety-rated component for use as a keep-in zone as described above. Alternatively, the POEs of the robot and/or human operator may be shared by the safety-rated and non-safety-rated control components of the robot controller. OMS 1010 may transmit the POEs and/or safe-action constraints to the robot controller 1004 via any suitable wired or wireless protocol. (In an industrial robot, control electronics typically reside in an external control box. However, in the case of a robot with a built-in controller, OMS 1010 communicates directly with the robot's onboard controller.) In various embodiments, OMS 1010 includes a robot communication module 1011 that communicates with the safety-rated component 1006 and non-safety-rated component 1008 via a safety-rated channel (e.g., digital I/O) 1012 and a non-safety-rated channel (e.g., an Ethernet connector) 1014, respectively. In addition, when the robot violates the safety measures specified in the safety protocol, OMS 1010 may issue commands to the robot controller 1004 via both the safety-rated and non-safety-rated channels. For example, upon determining that the robot speed exceeds a predetermined maximum speed when in proximity to the human (or the robot is outside the keep-in zone or the PSD exceeds the predetermined threshold), OMS 1010 may first issue a command to the non-safety-rated component 1008 via the non-safety-rated channel 1014 to reduce the robot speed to a desired value (e.g., below or at the maximum speed), thereby reducing the dynamic POE of the robot. This action, however, is non-safety-rated. Thus, after the robot speed is reduced to the desired value (or the dynamic POE of the robot is reduced to the desired size), OMS 1010 may issue another command to the safety-rated component 1008 via the safety-rated channel 1012 such that the safety-rated component 1008 can enforce a new robot speed, which is generally higher than the reduced robot speed (or a new keep-in zone based on the reduced dynamic POE of the robot). Accordingly, various embodiments effectively “safety rate” the function provided by the non-safety-rated component 1008 by causing the non-safety-rated component 1008 to first reduce the speed or dynamic POE of the robot in spatial extent in an unsafe way, and then engaging the safety-rated (e.g., monitoring) component to ensure that the robot remains in the now-reduced speed (or, within the now-reduced POE, as a new keep-in zone). Similar approaches can be implemented to increase the speed or POE of the robot in a safe manner during performance of the task. (It will be appreciated that, with reference to
Similarly, the keep-out zone may be determined based on the POE of the human operator. Again, a static future-interval POE represents the entire spatial region that the human operator may possibly reach within a specified time, and thus corresponds to the most conservative possible keep-out zone within which an intrusion of the robot will trigger a safety stop or slowdown. A static task-level POE of the human operator may reduce the determined keep-out zone in accordance with the task to be performed, and a dynamic, task-level or application-level POE of the human may further reduce the keep-out zone based on a specific point in the execution of a task by the human. In addition, the POE of the human operator can be shared by the safety-rated and non-safety-rated control components as described above for operating the robot in a safe manner. For example, upon detecting intrusion of the robot in the keep-out zone, the OMS 1010 may issue a command to the non-safety-rated control component to slow down the robot in an unsafe way, and then engaging the safety-rated robot control (e.g., monitoring) component to ensure that the robot remains outside the keep-out zone or has a speed below the predetermined value.
Once the keep-in zone and/or keep-out zone are defined, the machinery is safely constrained within the keep-in zone, or prevented from entering the keep-out zone, reducing the POE of the machinery as discussed above. Further, path optimization may include dynamic changing or switching of zones throughout the task, creating multiple POEs of different sizes, in a similar way as described for the operator. Moreover, switching of these dynamic zones may be triggered not only by a priori knowledge of the machinery program as described above, but also by the instantaneous detected location of the machinery or the human operator. For example, if a robot is tasked to pick up a part, bring it to a fixture, then perform a machining operation on the part, the POE of the robot can be dynamically updated based on safety-rated axis limiting at different times within the program.
Additionally or alternatively, once the machinery's current state (e.g., payload, position, orientation, velocity and/or acceleration) is acquired, a PSD (generally defined as the minimum distance separating the machinery from the operator for ensuring safety) and/or other safety-related measures can be computed. For example, the PSD may be computed based on the POEs of the machinery and the human operator as well as any keep-in and/or keep-out zones. Again, because the machinery's state may change during execution of the task, the PSD may be continuously updated throughout the task as well. This can be achieved by, for example, using the sensor system 101 to periodically acquire the updated state of the machinery and the operator, and, based thereon, updating the PSD. In addition, the updated PSD may be compared to a predetermined threshold; if the updated PSD is smaller than the threshold, the control system 112 may adjust (e.g., reduce), for example, the speed of the machinery as further described below so as to bring the robot to a safe state. In various embodiments, the computed PSD is combined with the POE of the human operator to determine the optimal speed or robot path (or choosing among possible paths) for executing a task. For example, referring to
In various embodiments, the measured separation distance relative to the PSD is utilized to govern the speed (or velocity or other states) of the machinery; this may be implemented in, for example, an application where the machinery path cannot deviate from its original programmed trajectory. In this case, the PSD between the POEs of the human and the machinery is dynamically computed during performance of the task and continuously compared to the instantaneous measured distance between the human and the machinery (using, e.g., the sensor system 101). However, instead of a system that alters the path of the machinery, or simply initiates a protective stop when the PSD is violated, the control system 112 may govern (e.g., modify) the current state of the machinery, e.g, reducing the velocity to a lower set point at a distance larger than the PSD. At the instant when the machinery reaches the lower set point, not only will the POE of the machinery be smaller, but the distance that the operator is from the new POE of the machinery will be larger, thereby ensuring safety of the human operator.
Interactions between control system 112 and robot controller 108 can take various forms with the overall objective of enforcing safety conditions. From the controller's perspective, robot controller 108 may report (e.g., with a safety-rated signal) to control system 112 that a monitoring component (such as a joint monitor) is active and that the safety-rated controller component will maintain operation of the robot 106 within the stored safe-state parameters. Here the joint monitor may be safety-rated, as discussed above, but in fact it may be acceptable for robot controller 108 to employ a non-safety-rated governor (e.g., a velocity governor) until a certain state, specified in the stored parameters, is attained. For example, the specified state may include a velocity within a certain percentage (e.g., 10%) of the maximum robot velocity and/or a keep-in zone defined spatially. At this point safety-rated monitoring is triggered to maintain operation of the robot 106 in the safe state dictated by the stored parameters. This approach allows use of non-safety-rated governors while clearly safe conditions prevail and safety-rated operation when progress toward potentially unsafe conditions is detected. Robot controller 108 may report activation of safety-rated monitoring to control system 112 (typically with a safety-rated signal), and de-activation of safety-rated monitoring if the current state no longer features any parameter values within the specified state.
Control system 112, is also monitoring the workspace for potentially unsafe conditions based on operator and equipment POEs, as described above. From the control system's perspective, detection of an unsafe condition (or, more typically, progress toward that condition with a prediction it may be reached shortly) may prompt action with respect to controller 108. For example, control system 112 may signal controller 108 to put robot 106 in a safe state—action that controller 108 may already be taking if it has detected the same condition based on its stored monitoring parameters. But because control system 112 rather than robot controller 108 has access to sensor information and the operator and equipment POEs, the safe-operation parameters that control system 112 sends to robot controller 108 may be more aggressive than the default parameters stored on controller 108, enabling more freedom of action and consequent workflow efficiency benefits.
Control system 112 may expect acknowledgment from controller 108 that robot 106 has entered a safe state within a prescribed interval, e.g., the worst-case stopping time for robot 106. If no acknowledgment is received by the end of the interval, control system 112 may enforce a safety stop (via robot controller 108 or by directly signaling robot 106). The acknowledgment, when furnished by controller 108, may indicate that, for example, the controller 108 has activated the non-safety rated governor and the safety-rated monitoring system. In response, control system 112 may update the operator and equipment POEs; for example, if the safety constraints enforced by robot controller 108 correspond to the stored default constraints, the POEs may be smaller and so robot 106 and human operators will have more safe freedom of movement within the workspace.
Further, the spatial mapping described herein (e.g., the POEs of the machinery and human operator and/or the keep-in/keep-out zone) may be combined with enhanced robot control as described in U.S. Pat. No. 10,099,372 (“'372 patent”), the entire disclosure of which is hereby incorporated by reference. The '372 patent considers dynamic environments in which objects and people come, go, and change position; hence, safe actions are calculated by a safe-action determination module (SADM) in real time based on all sensed relevant objects and on the current state of the robot, and these safe actions may be updated each cycle so as to ensure that the robot does not collide with the human operator and/or any stationary object.
One approach to achieving this is to modulate the robot's maximum velocity (by which is meant the velocity of the robot itself or any appendage thereof) proportionally to the minimum distance between any point on the robot and any point in the relevant set of sensed objects to be avoided. For example, the robot may be allowed to operate at maximum speed when the closest object or human is further away than some threshold distance beyond which collisions are not a concern, and the robot is halted altogether if an object/human is within the PSD. For example, referring to
In various embodiments, sufficient margin can be added to each of the zones 1302-1306 to account for movement of relevant objects or humans toward the robot at some maximum realistic velocity. Additionally or alternatively, state estimation techniques based on information detected by the sensor system 101 can be used to project the movements of the human and other objects forward in time. For example, skeletal tracking techniques can be used to identify moving limbs of humans that have been detected and limit potential collisions based on properties of the human body and estimated movements of, e.g., a person's arm rather than the entire person. The robot can then be operated based on the progressive safety zones 1302-1306 and the projected movements of the human and other objects.
In one embodiment, the mapping module 246 can receive data from a conventional computer vision system that monitors the machinery, the sensor system that scans the machinery and the operator, and/or the robot (e.g., joint position data, keep-in zones and/or or intended trajectory), in step 1432. The computer vision system utilizes the sensor system to track movements of the machinery and the operator during physical execution of the task. The computer vision system is calibrated to the coordinate reference frame of the workspace and transmits to the mapping module 246 coordinate data corresponding to the movements of the machinery and the operator. In various embodiments, the tracking data is then provided to the movement-prediction module 245 for predicting the movements of the machinery and the operator in the next time interval (step 1428). Subsequently, the mapping module 246 transforms this prediction data into voxel-level representations to produce the POEs of the machinery and the operator in the next time interval (step 1430). Steps 1428-1432 may be iteratively performed during execution of the task.
Additionally or alternatively, the control system (e.g., the path-determination module 248) may determine an optimal path of the machinery in the workspace for performing the task (e.g., without exiting the keep-in zone and/or entering the keep-out zone) based on the computed POEs of the machinery and human operator and/or keep-in/keep-out zones (e.g., by communicating them to a CAD system) and/or utilizing the conventional spatial modeling tool (step 1610). In some embodiments, the control system (e.g., the workspace-modeling module 249) computationally models the workspace parameter (e.g., the dimensions, workflow, locations of the equipment and/or resources) based on the computed POEs of the machinery and the human operator and/or the keep-in/keep-out zone (e.g., by communicating them to a CAD system) and/or utilizing the conventional spatial modeling tool so as to achieve high productivity and spatial efficiency while ensuring safety of the human operator (step 1612). For example, the workcell can be configured around areas of danger with minimum wasted space. In addition, the POEs and/or keep-in/keep-out zones can be used to coordinate multi-robot tasks, design collaborative applications in which the operator is expected to occupy some portion of the task-level POE in each robot cycle, estimate workcell (or broader facility) production rates, perform statistical analysis of predicted robot location, speed and power usage over time, and monitor the (wear-and-tear) decay of performance in actuation and position sensing through noise characterization. From the workpiece side, the changing volume of a workpiece can be observed as it is processed, for example, in a subtractive application or a palletizer/depalletizer.
Further, in various embodiments, the control system can transmit the POEs and/or keep-in/keep-out zones to a non-safety-rated component in a robot controller via, for example, the robot communication module 1011 and the non-safety-rated channel 1014 for adjusting the state (e.g., speed, position, etc.) of the machinery (step 1614) so that the machinery is brought to a new, safe state. Subsequently, the control system can transmit instructions including, for example, the new state of the machinery to a safety-rated component in the robot controller for ensuring that the machinery is operated in a safe state (step 1616).
The terms and expressions employed herein are used as terms and expressions of description and not of limitation, and there is no intention, in the use of such terms and expressions, of excluding any equivalents of the features shown and described or portions thereof. In addition, having described certain embodiments of the invention, it will be apparent to those of ordinary skill in the art that other embodiments incorporating the concepts disclosed herein may be used without departing from the spirit and scope of the invention. Accordingly, the described embodiments are to be considered in all respects as only illustrative and not restrictive.
This application claims the benefit of and priority to U.S. Ser. No. 16/999,676, filed on Aug. 21, 2020, which itself claims priority to U.S. Provisional Patent Application Nos. 62/890,718 (filed on Aug. 23, 2019) and 63/048,338 (filed on Jul. 6, 2020). The entire disclosures of the foregoing priority documents are hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| 62890718 | Aug 2019 | US | |
| 63048338 | Jul 2020 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16999676 | Aug 2020 | US |
| Child | 17709298 | US |
