Patent Application
20230047013
A variety of security, monitoring, and control systems equipped with a plurality of cameras, audio input devices, and/or sensors have been used to detect certain human presence or a particular human activity at a monitored location, specifically in a home/living environment. In many cases, a snapshot of an event or activity happened at the monitored location is collected by the devices/sensors to piece together if this occurrence is part of the normal trend or is an abnormal event. However, there is often no context or feedback loop on whether a sequence of activities has occurred in a certain zone or location of interest by a person. As such, it is often impossible for the current approaches to intelligently determine if a certain protocol or procedure has been complied with or violated. Such protocol compliance is especially important for workplace environment like factories and offices where many safety protocols or operational procedure have to be followed to ensure efficiency operations in compliance with required regulations. Almost always, these safety or operational protocols encompass a collection of events/activities that need to occur in a specific order in a particular zone of interest as well as to be executed by a specific person, which is a non-trivial task when insufficient input parameters/context are available.
The foregoing examples of the related art and limitations related therewith are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent upon a reading of the specification and a study of the drawings.
Aspects of the present disclosure are best understood from the following detailed description when read with the accompanying figures. It is noted that, in accordance with the standard practice in the industry, various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.
The following disclosure provides many different embodiments, or examples, for implementing different features of the subject matter. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. In addition, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
A new approach is proposed that contemplates systems and methods to support protocol compliance by a person in various workplace applications and environments. Specifically, the proposed approach determines if a person is following a set of protocols/procedures created and defined to ensure safety and efficiency of the workers/employees in his/her workplace environment. This proposed approach focuses on specifying one or more zones of interest, identifying presence of the person and/or an object associated with the person in the one or more zones of interest, classifying a sequence of activities and/or postures of the person, and determining the durations of the activities. A user (e.g., an employer) is notified if it is determined that the person is not in compliance with the set of protocols in the workplace environment. In addition, data collected from the one or more zones of interest is stored securely in a local site to protect confidentiality of production processes as well as privacy of the person at the workplace.
The proposed approach has many applications in workplace environments (e.g., factories, offices, and retail stores) where strict compliance with a set of pre-determined or regulated set of protocols/procedures is critical for health/safety of the workers, operational efficiency of the businesses, and/or warehouse inventory control and theft prevention. For example, for the safety and health of the workers, the following activities of a person need to be monitored, documented or reported for protocol compliance in certain workplace environments: use of personal protection equipment (PPE), falls and dangerous activities, non-compliance in dangerous zones or equipment areas, non-compliance with maintenance protocols such as lock-out tag-out (LOTO) or chemical/gases change, lone worker monitoring, etc. Similarly, the following may also be tracked, documented, or reported for protocol compliance to ensure that a worker's operation efficiency is optimized in conjunction with equipment processes: if a worker performs required procedures at various stages of production, the time or durations when the worker is idle vs productive, if the worker is performing a set of procedures optimally, or if retraining is needed, the worker's response time/action to an equipment. For inventory theft control, the proposed approach can detect if an unauthorized person is in a designated area during off-hours or violating protocols by removing certain inventory.
In the example of
In the example of
In the example of
In some embodiments, the human activity tracking engine 102 is configured to maintain the collected information (e.g., images, video, and/or audio) of the person in a secured local storage 103, which can be a data cache associated with the human activity tracking engine 102, to ensure data privacy and security of the person. In some embodiments, the data locally maintained in the secured local storage 103 can be accessed by the human activity tracking engine 102 and/or protocol compliance engine 106 via an Application Programming Interface (API) only under strict data access control policies (e.g., only accessible for authorized personnel or devices only) to protect the person's privacy. In some embodiments, information retrieved from the secured local storage 103 is encrypted before such information is transmitted over a network for processing or being accessed by an authorized application or a web-based service. In some embodiments, the secured local storage 103 resides onsite behind a user's firewall. Note that none of the sensitive video/audio of the person leaves the secured local storage 103 hence guaranteeing the person being monitored at the location/zone of interest has full control of his/her data, which is particularly important in highly confidential manufacturing or work areas as well as sensitive/private hospital or healthcare environment.
In some embodiments, the human activity tracking engine 102 is configured to track and identify the sequence of activities executed by the person or an object at the one or more zones/locations of interest executed for a certain amount of time. Here, the zones of interest are a pre-defined/prescribed space or area where the set of protocols must be complied with. In some embodiments, the human activity tracking engine 102 is configured to systematically define/mark out the zones of interest such that if an activity, a person, or an object is detected in the zones of interest by the human activity tracking engine 102, a series of actions will be triggered to ascertain whether the set of protocols for the zones of interest is enforced. In some embodiments, the human activity tracking engine 102 is configured to detect the presence of a person or an object on, associated with, or around the person at the zone of interest subject to the set of protocol in order to determine if compliance of the set of protocols is maintained. In some embodiments, the human activity tracking engine 102 is configured to track and/or record the amount of time the person spent in the zone of interest or doing certain activities in order to ascertain the person's compliance with the set of protocols.
Once the sequence of activities of the person at the one or more zones of interest has been detected, the sequence of activities of the person is provided to the protocol compliance engine 106, which is configured to determine if the sequence of activities of the person at the zone of interest follows the set of protocols or not. In some embodiments, the protocol compliance engine 106 is configured to create/define/describe the set of protocols to be enforced by a user (e.g., an employer, a production/safety manager, etc.) that needs to be complied by the person (e.g., an employee, a factory operator, etc.). Here, the set of protocols cover one or more of the one or more zones of interest, the presence of the person in the one or more zones of interest, the sequence of activities by the person or the object in the one or more zones of interest, and the duration of the sequence of activities. In some embodiments, the protocol compliance engine 106 is configured to create and enforce a set of protocols that are mandatory in a workplace environment to ensure worker safety, wherein such set of protocols includes one or more of the following:
In some embodiments, the protocol compliance engine 106 is configured to create and enforce a set of protocols for operational efficiency at a workplace, wherein such set of protocols includes one or more of the following:
When a non-compliance event has been identified, the protocol compliance engine 106 is configured to trigger one or more notification/escalation procedures to document and report the non-compliance event to a user per agreement with the user. For example, the protocol compliance engine 106 is configured to provide one or more of an email/text message notification or a phone call to the manager-in-charge, a notification in the user portal monitored by a designated employee, and an alert to a smart mobile device. In some embodiments, the protocol compliance engine 106 is configured to report the violation to the user in the form of a dashboard in order for the user to analyze and correct/recover the situation.
In some embodiments, the protocol compliance engine 106 is configured to accept input from an existing alarm system (e.g., Andon lights, Sound alarms etc.) to identify/classify an escalation event when a safety compliance protocol or an operation procedure is being violated. This will reinforce any escalation protocols (emails, phone calls etc.) to the designated person-in-charge to correct/recover the situation. In some embodiments, the protocol compliance engine 106 is configured to utilize/trigger any existing alarm system (e.g., sound or light) in the affected work environment to notify the person of a violation event in order to minimize the risk to the person and/or other affected/surrounding person(s), e.g., a forklift out of control in a work zone or a chemical spill due to non-compliance of maintenance protocols.
In the example of
One embodiment may be implemented using a conventional general purpose or a specialized digital computer or microprocessor(s) programmed according to the teachings of the present disclosure, as will be apparent to those skilled in the computer art. Appropriate software coding can readily be prepared by skilled programmers based on the teachings of the present disclosure, as will be apparent to those skilled in the software art. The invention may also be implemented by the preparation of integrated circuits or by interconnecting an appropriate network of conventional component circuits, as will be readily apparent to those skilled in the art.
The methods and system described herein may be at least partially embodied in the form of computer-implemented processes and apparatus for practicing those processes. The disclosed methods may also be at least partially embodied in the form of tangible, non-transitory machine readable storage media encoded with computer program code. The media may include, for example, RAMs, ROMs, CD-ROMs, DVD-ROMs, BD-ROMs, hard disk drives, flash memories, or any other non-transitory machine-readable storage medium, wherein, when the computer program code is loaded into and executed by a computer, the computer becomes an apparatus for practicing the method. The methods may also be at least partially embodied in the form of a computer into which computer program code is loaded and/or executed, such that, the computer becomes a special purpose computer for practicing the methods. When implemented on a general-purpose processor, the computer program code segments configure the processor to create specific logic circuits. The methods may alternatively be at least partially embodied in a digital signal processor formed of application specific integrated circuits for performing the methods.
This application claims the benefit of U.S. Provisional Patent Application No. 63/232,894, filed Aug. 13, 2021, which is incorporated herein in its entirety by reference. This application is related to co-pending United States patent application No. ______, filed ______, and entitled “SYSTEM AND METHOD FOR ARTIFICIAL INTELLIGENCE (AI)-BASED ACTIVITY TRACKING FOR PROTOCOL COMPLIANCE,” which is incorporated herein in its entirety by reference.
| Number | Date | Country | |
|---|---|---|---|
| 63232894 | Aug 2021 | US |
