Patent Application
20210258783
The present invention relates to terminals, systems and method to control a state of a terminal automatically upon obtaining an signal via an external apparatus.
The present invention is directed to a system and corresponding method for respectively controlling a state of the terminal when a individual in possession of the terminal has arrived or departed a environment. For the purpose of summarizing, certain aspects, advantages, and features of the present invention has been described herein. In accordance with one or more embodiments, a method for adjusting the state of a terminal in relationship to when the individual has arrived or departed the environment is provided. The method comprises one or more wireless transceivers configured to obtain wireless transmission of the terminal to indicate the arrival or departure of the individual in possession of the terminal, one or more A/V recording and communication apparatus within the environment, obtaining biometric data of the individual to determine an match of identity with prior stored biometric data; and one or more VSIM servers of the service provider configured to distribute at lease one behavior state signal (e.g., volume control signal or power-down control signal) to the terminal causing the terminal to operate at one behavior at which the distributed behavior state signal pertains to. The behavior state signal represents adjusting the terminal ringtone/notification volume level or powering-down of the terminal.
In response to the terminal obtaining an power-down control signal, the terminal goes into a partial sleep mode for a discrete interval of time. The power-down control signal can consist of an power-down control signal and an behavior state duration control signal, further the behavior state duration signal determines the predetermine duration of the sleep mode. The terminal can comprise at lease one application that allocates the terminal to adjust the ringtone/notification volume levels via an volume adjusting device via obtaining an behavior state signal. The system comprises an behavior state processing unit which can obtain and process data and distribute data to an server and other components of the system. These and other embodiments of the present invention will also become readily apparent to those skilled in the arts from the following detailed description of the embodiments having reference to the attached figures, the invention not being limited to any particular embodiments disclosed.
A more complete understanding of the present invention is derived by referring to the detailed description and claims when considered in connection with the Figures, wherein like reference numbers refer to similar items throughout the Figures.
In accordance with the present invention, when the term terminal is used in the current disclosure, the terminal can refer to a mobile terminal, wearable terminal (e.g., smart-watch, smart-ring, smart-bracelet, smart-glasses, a belt, a necklace, a earring, headband, helmet or a device embedded in the cloths), a server, a personal computer (PC), a laptop, a notebook, a subnotebook, a netbook, an ultra-mobile PC (UMPC), a tablet personal computer (tablet), a phablet, a mobile internet device (MID), a personal digital assistant (PDA), an enterprise digital assistant (EDA), a digital camera, a digital video camera, a portable game console, an MP3 player, a portable/personal multimedia player (PMP), a handheld e-book, an ultra mobile personal computer (UMPC), a portable lab—top PC, a global positioning system (GPS) navigation, a personal navigation device, portable navigation device (PND), a handheld game console, an e-book, a high definition television (HDTV), a smart appliance, communication systems, image processing systems, graphics processing systems, various Internet of Things (IoT) devices that are controlled through a network, other consumer electronics/information technology (CE/IT) device, automated teller machine (ATM) or any other device capable of wireless communication or network communication.
According to
Terminal 1 include processor 27 which controls a multitude of hardware or software components that may be connected thereon and may also perform various data processing and operations by executing an operating system, application program, or operating system and application instructions. Processor 27 may be implemented with a system on chip (SoC). Processor 27 may further include a graphic processing unit (GPU) and/or an image signal processor. Processor 27 can execute one or more programs stored within memory 7 and control the general operation of the program.
Interface 50 includes a universal serial bus (USB), or an optical interface. Additionally or alternatively, the interface 50 can include a mobile high definition link (MHL) interface, a secure Digital (SD) card/multi-media card (MMC) interface, or an infrared data association (IrDA) standard interface. Interface 50 can act as a passage for supplying terminal 1 with power from a cradle or delivering various command signals input from the cradle if terminal 1 is connected to an external cradle. Each of the various command signals input from the cradle or the power may operate as a signal enabling terminal 1 to recognize that it is correctly loaded in the cradle. Interface 50 may be coupled to terminal 1 with external devices, such as wired/wireless head phones, external chargers, power supplies, storage devices configured to store data (e.g., audio, video, pictures, etc.), earphones, and microphones, etc. In addition, interface 50 may use a wired/wireless data port, a card socket (e.g., for coupling to a memory card, a Subscriber identity module (SIM) card, a user identity module (UIM) card, a removable user identity module (RUIM) card, etc.), audio input/output ports and/or video input/output ports, for example. Input/output module 75 comprises speaker 74 and microphone 17. Speaker 74 may receive call mode, voice recognition, voice recording, and broadcast reception mode from wireless communication module 10 and or output audio sound or sound data that may be stored inside of memory 7, external storage, or transmitted from an external device. For example, terminal 1 can comprise of multiple ring-tone/notification volume levels output from one component of input/output module 75 such as speaker 74. For example, input/output module 75 can comprise of, but is not limited to sixteen ringtone/notification volume levels designated as “0”, “1”, “2”, “3”, “4”, “5”, “6”, “7”, “8”, “9”, “10”, “11”, “12”, “13”, “14”, “15”, “16”. Further, each respective volume level with an higher value than an preceding lower value comprises an higher volume level output than the volume level below. For instance, volume level “1” represents an behavior state at which terminal 1 is in silent mode (SM) behavior state 1 (BS1), whereas upon obtain an incoming phone-call, notification or message(s), not limited to, SMS messages, (e.g., text-messages, news alert messages, financial information messages, logos, ring-tones and the like) e-mail messages, multimedia messaging service messages (MMS) (e.g., graphics, animations, pictures, video clips, etc.) the ring-tone/notification volume level output from input/output module 75 is completely silent. For instance, volume level “2” represents an behavior state at which terminal 1 is in vibrate mode (VM) behavior state 2 (BS2), whereas upon obtain an incoming phone-call, notification or message(s) processor 27 causes the battery or an vibrating component to preform an vibrating motion as a notification to the user alerting the user of an incoming phone-call and/or message(s). More of, volume levels “3-16” represents an behavior state at which input/output module 75 speaker outputs an ring-tone/notification volume level in response to obtaining an incoming phone-call, notification or message(s), whereas the volume levels of the ring-tone can vary low or high depending on the volume levels at which the ring-tone/notification is set at in conjunction with volume adjusting device 49 with volume level “3” the lowest, volume level “8” the medium and volume level “16” the highest according to output levels. More of, terminal 1 can comprise of “16” ringtone/notification volume adjustment tones (R/NVAT) which respectively corresponds with the “16” ring-tone/notification volume levels. The ringtone/notification volume adjustment tone (R/NVAT) can be an beeping sound or the likes output via one component of input/output module 75 (e.g., speaker 74) in response to the user interacting with an physical button on the subscriber terminal 1 to adjust the volume levels of the ringtone/notification volume levels. More of, each respective ringtone/notification volume adjustment tone (R/NVAT) volume level with an higher value than an preceding ringtone/notification volume adjustment tone (R/NVAT) comprises an higher volume level output than the preceding ringtone/notification volume adjustment tone (R/NVAT) volume level. For instance, in response to an user manually interacting with an input/output module 75 such as an button on terminal 1 to adjust the ringtone/notification volume levels input/output module 75 (e.g., speaker 74) can output an ringtone/notification volume adjustment tone (R/NVAT) (e.g., an beeping sound or the likes) for each respective ring-tone/notification volume adjustment tone (R/NVAT), the volume level of each ringtone/notification volume adjustment tone (R/NVAT) can be higher or lower than an corresponding volume adjustment tone (VAT) depending on if the user adjust the ring-tone/notification volume level up or down.
Input/output module 75 can include microphone 17 configured to obtain an external or internal noise such as the ringtone/notification volume adjustment tone (R/NVAT). Terminal 1 composes sound measuring device (SMD) 31 (e.g., such as an volume sensor or the likes) configured to obtain and measure (e.g., in dB) external and internal sounds such as the ringtone/notification volume adjustment tone (R/NVAT) etc. Sound measuring device (SMD) 31 may obtain external and/or internal sounds from microphone 17 associated with terminal 1 or from another component of input/output module 75 (e.g., such as speaker 74) within terminal 1.
Terminal 1 further comprises at lease one volume adjusting device 49, which allocates the subscriber to increase or decrease input/output module 75 volume via instructions provided via a user manually pressing an input button(s) interacting with the user interface or navigating at lease one menu to select an desired volume level via terminal 1 display, or in agreement with instructions provided via behavior state adjustment application (BSAA) 9, behavior state adjustment application 9 allocates volume adjusting device 49 to be controlled via terminal 1 obtaining at lease one behavior state signal (power-down control signal or volume control signal) via VSIM server 2.
Furthermore, input/output module 75 speaker 74, microphone 17, sound measuring device 31 and volume adjustment device 49 can be embedded in the same electrical module. Alternatively, each of said devices either individually or in combination may comprise one or more electrical modules or components that operate to send or receive control signals to processor 27 in accordance with instructions dictated by behavior state adjustment application 9 and/or control software.
Terminal 1 further includes battery 25, such as a vibrating battery pack, for powering various circuits and components that is required to operate terminal 1, as well as optionally providing mechanical vibration as a detectable output.
For instance, when terminal 1 obtains an behavior state signal (e.g., behavior state 2) via VSIM server 99 the ringtone/notification adjustment tone position is set or adjusted to “2” and the battery pack is capable of vibrating terminal 1. In this regard, volume level “2” correspond to vibrate mode (VM) behavior state 2 (BS2).
Accelerometer 107 can sense accelerations with respect to one or more axes of the accelerometer and generated acceleration data corresponding to the sensed accelerations.
For example, accelerometer 107 can be a multi—axis accelerometer including x, y, and z axes and can be configured to sense accelerations with respect to the x, y, and z axes of accelerometer 107. The acceleration data generated by accelerometer 107 can be used to determine one or more metrics associated with the subscriber in possession of terminal 1.
For example, the acceleration data generated by accelerometer 107 can be used to determine a quantity of steps the subscriber has taken over time.
Accelerometer 107 can output acceleration data corresponding to each axes of measurement and/or can output one or more signals corresponding to an aggregate or combination of the three axes of measurement. For example, in some embodiments, accelerometer 107 can be a three—axis or three—dimensional accelerometer that includes three outputs (e.g., accelerometer can output x, y, and z component data). Accelerometer 107 can detect and monitor a magnitude and direction of acceleration (e.g., as a vector quantity, and/or can sense an orientation, vibration, and/or shock. In some embodiments, gyroscope 108 can be used instead or in addition to accelerometer 107, to determine an orientation of terminal 1. In some embodiments, the orientation of terminal 1 can be used to aid in determining whether the acceleration data corresponds to a step taken by subscriber.
Terminal 1 includes memory 7, an internal memory that may comprise a SSD (Solid State Drive), NAS (Network Attached Storage), Dual-Channel RAM (Random Access Memory), Multi-ROM (Read-Only Memory), Flash Memory (Flash Memory Type), Hard Disk (Hard Disk Type), Multimedia Card Micro (Multimedia Card Micro Type), SRAM (Static Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), may further include a card type memory Compact Flash (CF), Secure Digital (SD), Micro Micro-SD, Mini-SD, Extreme Digital (xD), Multimedia Card (MMC) or a memory stick.
The external memory may be functionally and/or physically connected to terminal 1, these components may preform processing cores and dedicated graphics, alternatively some components of memory 7 may store terminal 1 operating system components, application data, and critical system files, many of the previously mentioned files and systems may be separated to different storage chips throughout terminal 1 printed circuit-board. Memory 7 can further store the subscriber-related information, the subscriber's volume control data, and associated software supporting behavior state adjustment application (BSAA) 9. Memory 7 can store instructions and or codes to power-off, power-up and adjust the volume levels of terminal 1.
Memory 7 can store firmware and data for use by terminal 1. In exemplary embodiments, data can include the acceleration data or any other suitable data associated with the subscriber in possession of terminal 1 or the output of sensors (e.g., accelerometer 104) included in terminal 1. Memory 7 can also store a unique identifier 126 that can be used to distinguish transmissions from terminal 1 to terminal or external apparatus.
Memory 7 also includes VSIM memory 2 which is used to store the provisioning information of one or more enabled VSIM subscriptions. VSIM memory 2 may be a partition within memory 7 or may be a separate internal memory unit. In addition, VSIM memory unit 2 may store personal data downloaded from one or more VSIM servers 99 for use with applications being executed on processor 6.
For instance; when creating an account or subscription the subscriber can accomplish this over an cellular communication network or by using an external computer that is connected to the Internet. Such an account can be created by the user entering personal information into a webpage or into terminal 1.
During the process, the user can create an account name (or user name) which is an arbitrary but unique account name that will be associated with terminal 1 being registered to the network. The account activation can also require the user to enter a password to be associated with the user account for accessing the account in the event of changing personal information and obtain an new terminal 1, the user's biographical information and user account name are stored as a file in Virtual SIM Database (VSIMD) 41 via the VSIM server 99 via terminal 1 over network 21.
Further, setting up the user account, the user can be prompted to enter authentication credentials prior to transferring data at the time the account is being created that will be used in subsequent sessions to authenticate each user prior to granting access to the sensitive information. Any of a number of authentication methods can be employed, including password verification, biometric recognition, and a combination thereof. The authentication credentials can be obtained by VSIM server 99 via terminal 1 over network 21 or through the an external computer via an Internet link and distributed to authentication database 52 via authentication server 32 and is stored as an authentication file associated with the user account name. The authentication credential can be a simple alphanumeric password. Next, the user is prompted to create an virtual identification card (VIC) to be used for authenticating an subscriber via facial recognition when the subscriber is within an predetermine region of environment 100.
The virtual identification card (VIC) may be the like(s) of an digital or virtual drivers license, identification card, school identification card or employment identification card. During the process of creating the virtual identification card (VIC) the user is prompted to enter personal information such as an first and last name into an personal information field within the virtual identification card (VIC). Following, the user is prompted to capture an acceptable face-shot of themselves via the camera module arranged on their terminal or upload an acceptable face-shot image that is stored in memory 7 if the face image is accepted the face image is then attached to an image field within virtual identification card (VIC), this step allocates the behavior state processing unit 46 to preform an matching/comparing task with contextual biometric data obtained by one or more A/V recording and communication apparatus 14 within environment 100 with the photo attached to the virtual identification card (VIC). Upon completion of attaching an acceptable image, the application or web page random generates an authentication key and associates the authentication key within an field on the virtual identification card, this may be done by way of the user being prompted to click on an button labeled generate authentication key. Specifically, the authentication key comprises of the first initial of the subscriber first name, followed by the subscriber complete last name and an randomly generated seven digit alphanumeric.
For example, an subscriber name John Sims authentication key may be JSim9U07P19. Further, biological information and subscriber key is stored in service provider database 60 of VSIM server 99.
Once the user account and authentication credentials is established and stored in one or more databases, and the authentication key is generated the virtual identification card (VIC) is transferred to the virtual identification card store (VICS) 12 via VSIM server 99 via behavior state processing unit 46.
VSIM memory 2 comprises behavior state adjustment application (BSAA) 9 which may analyze an obtain an behavior state signal (BSS) (e.g., volume-control signal) via VSIM server 99 and distribute an volume-control signal request to volume adjusting device 49 to adjust the ringtone/notification volume level. For instance, behavior state adjustment application 9 may obtain an behavior state signal (BSS) via VSIM server 99, in response behavior state adjustment application 9 may analyze the data associated with the obtained behavior state signal (BSS) and compare the obtained signal data with behavior state data associated within subscriber behavior state database 43 to adjust terminal 1 behavior state (e.g., ringtone/notification volume level) from one to another by way of volume adjustment device 49.
VSIM memory 2 comprises an subscriber behavior state database (SBSD) 43 that comprises data such as the ring-tone/notification volume adjustment tone (R/NVAT) volume level position on sound bar/meter 67 and volume levels in the form of “output action threshold”; for instance, behavior state 1 (BS1) may be equivalent to position “0” on sound bar/meter 67 and the volume level threshold may be “output action threshold” T0 within subscriber behavior state database 43 which would be silent mode/do not disturb mode (SM/DNDM), and behavior state 2 (BS2) may be equivalent to position “1” on sound bar/meter 67 and the volume level threshold may be “output action threshold” T1 within subscriber behavior state database 43 which would be vibrating mode (VM). Subscriber behavior state database (SBSD) 43 may also be updated with terminal 1 original behavior state (OBS) prior to obtaining an predetermine behavior state signal.
VSIM memory 2 comprises behavior state duration application (BSDA) 26 that, when processed by processor 27 enables processor 27 to: obtain and analyze data associated with an behavior state duration signal and generate one or more timers and associate the one or more timers with an predetermine schedule portion (behavior state duration time), wherein upon the one or more timers reaching an value of 0:00:00 processor 6 generates an original behavior state signal instructing behavior state adjustment application 9 to adjust terminal 1 back to its original behavior state (OBS) via behavior state duration application 26 sending one or more control signal request to volume adjustment device 49. In addition the one or more timers may correspond with the one or more timers associated with behavior state processing unit(s) 46 behavior state duration application 36. Further, the timer associated with behavior state duration application 26 is the likes of an count-down timer. In addition the timer may be associated with an identifier that may distinguish an opposing timer from another.
For instance, upon receipt of behavior state duration application 26 obtaining an behavior state duration signal and generating one or more timers and associating the one or more timers with one or more predetermine schedule portions total times (e.g., behavior state duration time), behavior state duration application 36 associated with behavior state processing unit 46 may also comprise one or more timers associated with the same predetermine schedule portion total time as subscriber terminal 1 one or more timers, so that when the one or more timers of behavior state duration application 26 elapse terminal 1 is adjusted back to its original behavior state (OBS) (e.g., ringtone/notification volume level) and when the one or more timers of behavior state duration application 36 timer elapse behavior state processing unit 46 distributes an view-point signal to one or more A/V recording and communication apparatus 14.
Further, behavior state adjustment application 9, behavior state duration application 26 and subscriber behavior state database 43 may be uploaded to terminal 1 VSIM memory 12 along with the provisioning data during the activation of the service provided by the service provider.
The above arrangements of the applications may be implemented in a computer-readable medium using, for example, computer software, hardware, or some combination thereof. For a hardware implementation, the above described arrangements may be implemented within one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, and/or a selective combination thereof arrangements may also be implemented by processor 27. For a software implementation, the above described arrangements may be implemented with separate software modules, such as procedures and functions, each of which may perform one or more of the functions and operations described herein. The software codes may be implemented with a software application written in any suitable programming language and can be stored in a memory (e.g., the memory 7), and executed by processor 27. The applications may be an web application, a native application, and or an mobile application (e.g., an app) downloaded from an digital distribution application platform that allows users to browse and download applications developed with mobile software development kits (SDKs).
System 5 comprises behavior state processing unit (BSPU) 46 that respectively obtains and distribute data and/or instruction to A/V recording and communication apparatus, one or more wireless transceivers, one or more mobile terminals 1, VSIM servers 99, virtual identification card database (VICD) 12, user authentication database (UAD) 59, employee classifier database (ECD) 28, student classifier database (SCD) 24, miscellaneous database (MD) 93 and biometric data classifier database 34.
The virtual identification card database (VICD) 12 respectively store an virtual identification card(s) (VIC) for each subscriber operating on the system, further the virtual identification card comprises biographical information such as the subscriber first and last name, an digital photo of the subscriber and an respective subscribers authentication key, whereas the subscriber authentication key comprises the first initial of the subscriber first name, followed by the subscriber complete last name and an randomly generated seven digit alphanumeric.
User authentication database 59 respectively store authentication credentials for each subscriber that has been authenticated via the facial recognition tasker application 30. For example, an subscriber biological and face data can be obtained, analyzed and authenticated under the control of facial recognition processor 63, one or more databases 15 and applications 30, in response the accepted authentication credentials are stored within the user authentication database 59 as an user authentication file.
Employee classifier database 28 respectively store an employee classifier file for each subscriber that may be an employee of the environment 100. Student classifier database 24 respectively store an student classifier file for each subscriber that may be an student of the environment 100. Miscellaneous database 45 respectively store an classifier file for each subscriber that may be an visitor of the environment 100. Biometric data database 34 respectively store an classifier file for an subscriber comprising biometric data.
Referring to
VSIM server 99 may be configured to distribute one or more behavior state signals (e.g., volume control signal, power-down signal and original behavior state signal) to one or more subscriber terminals 1 over cellular network 37 upon obtaining instructions provided by behavior state processing unit 46.
VSIM database 41 may store the personal data information for each subscriber terminal 1 operating on the system.
Subscriber functionality database 19 can store information for one or more individuals comprising an VSIM subscription for Terminal Behavior System Virtual SIM System. Specifically, subscriber functionality database 19 comprises data such as each subscriber biological information, identifier for each terminal (e.g., terminal make and model), data pertaining to the hardware and software capabilities of each subscriber terminal 1 (e.g., such hardware and software capabilities may be the amount of ringtone/notification volume adjustment tones the terminal may comprise, the position of the ringtone/notification volume adjustment tone arranged on a sound bar/meter 67, the mode (e.g., silent mode or vibrate mode) the terminal 1 may be in when the ringtone/notification volume adjustment tone is at an predetermine position on sound bar/meter 67 and thresholds (e.g., “output action threshold”) of each respective ringtone/notification volume adjustment tone of the subscriber terminal 1).
For instance, upon one or more subscribers obtaining one or more subscription provided by the service provider, VSIM server 99 may search one or more manufactures and/or software and hardware developer sites and/or database by the subscriber terminal identifier (e.g., make and model), operating system, system software and control software to obtain subscriber terminal 1 hardware and software capability's, the amount of ringtone/notification volume adjustment tones an terminal comprises, the position of the ringtone/notification volume adjustment tone arranged on sound bar/meter 67, the mode (e.g., silent mode or vibrate mode) an subscriber terminal 1 may be in when the ringtone/notification volume adjustment tone is at an predetermine position on sound bar/meter 67 and thresholds (e.g., “output action threshold”) of each respective ringtone/notification volume adjustment tone. Upon determining the ringtone/notification volume adjustment tone position on sound bar/meter 67, threshold (e.g., “threshold output action”) and mode the terminal may be in upon an ringtone/notification volume adjustment tone arranging at an predetermine position on sound bar/meter 67. VSIM server 99 may generate an subscriber behavior state database 43 and associate subscriber behavior state database 43 with an value for the ringtone/notification volume adjustment tone “position” on sound bar/meter 67 and threshold (e.g., “threshold action output”) and may associate the one or more ringtone/notifications volume adjustment tones with an behavior state (e.g., behavior state 1 or behavior state 2) upon completion VSIM server may distribute subscriber behavior state database 43 to subscriber terminal 1.
Authentication server 62 may be in connection with authentication database 52, to store the authentication credentials for each subscriber terminal 1 operating on system 5.
Network 21 may be any type of network, such as Ethernet, To Firewire, USB, Blue Tooth, Fibre Channel, WiFi, IEEE 802.11g, 802.11n, 802.11ac, WiMAX or other any other network type know to one skilled in the art(s). Network 37 may be any type of cellular network such as LTE, UMTS, 5G, 6G, or any other cellular network type known to one skilled in the art(s).
System 5 comprises one or more A/V recording and communication apparatus 14 arranged at predetermine regions of the environment 100. A/V recording and communication apparatus 14 may be the likes of an wireless-enabled digital camera module capable of capturing digital video and still images in its field of view. AN recording and communication apparatus 14 can be configured to record images periodically, (e.g. a fixed rate), or in response to one or more movement activities within a zone in front of A/V recording and communication apparatus 14, (e.g., in response to a subscriber moving into position in view of A/V recording and communication apparatus 14.
In one implementation, A/V recording and communication apparatus 14 can be configured to record images at a low rate when activity is not detected within a zone in front of A/V recording and communication apparatus 14 and to record images at an higher rate when activity is detected within the zone. In the preferred embodiment A/V recording and communication apparatus 14, are configured to collect biometric data (e.g., facial data) from an subscriber to determine an match of the obtained contextual biometric data and with historical biometric data associated with the subscriber virtual identification card (VIC) stored in virtual identification card database 12 in order to authenticate the subscriber via facial recognition tasker application 30. As used herein, the term “biometric data” refers to data that can uniquely identify an subscriber among other humans (at an high degree of accuracy) based on the subscriber physical or behavioral characteristics.
In some implementations, the obtained biometric data can comprise an unique identifier which can be used to characteristically distinguish one biometric data profile from another.
For example, A/V recording and communication apparatus 14 role is to obtain biometric data within the environment 100 in order to determine a present of a subscriber.
Additionally, A/V recording and communication apparatus 14 may comprise communication module 47 required to establish connections and wirelessly communicate with behavior state processing unit 46 via network 21. A/V recording and communication apparatus 14 can communicate via communication systems such as PAN (Personal Area Network), Zigbee, LAN (Local Area Network), WIFI, MAN (Metropolitan Area Network) WiMAX (World Interoperability for Microwave Access), WAN (Wide Area Network), Wibro (Wireless Broadband), UWB (Ultra-Wideband), and IPV6 (Internet Protocol Version 6) communication systems. For example, A/V recording and communication apparatus 14 can obtain biometric data and distribute biometric data to one or more data stores 15 via behavior state processing unit 46 via network 21.
A/V recording and communication apparatus 14 includes one or more processor 44 in connection with memory 22 which controls a multitude of hardware or software components that may be connected thereon and may also perform various data processing and operations by executing an operating system, application program, or operating system and application instructions. Processor(s) may include a Central Processing Unit (CPU), a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), an Application Specific Integrated Circuit (ASIC), a System-on-Chip (SOC), a programmable logic unit, a microprocessor, or any other device capable of performing operations in a defined manner.
Processor 44 may be configured, through a execution of computer readable instructions stored in memory(s) program(s).
A/V recoding and communication apparatus 14 include memory 22 an internal memory that may comprise a SSD (Solid State Drive), NAS (Network Attached Storage), Dual-Channel RAM (Random Access Memory), Multi-ROM (Read-Only Memory), Flash Memory (Flash Memory Type), Hard Disk (Hard Disk Type), Multimedia Card Micro (Multimedia Card Micro Type), SRAM (Static Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), may further include a card type memory Compact Flash (CF), Secure Digital (SD), Micro Micro-SD, Mini-SD, Extreme Digital (xD), Multimedia Card (MMC) or a memory stick.
In some examples, memory(s) may be a temporary memory, meaning that a primary purpose of memory may not be long-term storage.
Additionally, memory 22 comprises one or more modules such modules are face detector module (FDM) 69, characteristic Module (CM) 11, characteristic algorithm module (CAM) 48, Augmented Reality (ARM) module 77, face frame module (FFM) 40 and an application 81.
Each modules (including any sub-modules) may be implemented in hardware, firmware, software (e.g., program modules comprising computer-executable instructions), or any combination thereof. Each module may be implemented on/by one device, such as a computing device, or across multiple such devices. For example, one module may be implemented in a distributed fashion on/by multiple devices such as servers or elements of a network service or the like. Further, each module (including any sub-modules) may encompass one or more sub-modules or the like, and the modules may be implemented as separate modules, or any two or more may be combined in whole or in part. The division of modules (including any sub-modules) described herein in non-limiting and intended primarily to aid in describing aspects of the invention.
Face detector module 69 respectively obtain image data and depth data from an image sensor and depth-sensor in time, in frames; for instance, the image data can be obtained at 60 frames/second (fps), and depth data can be obtained at 15 fps. In response to detecting an face of an subscriber in the field of view and acquiring an respective position and size of an individual facial via face detector module 69, processor 44 can cause the recording or image capturing process to display at lease one face detection frame when an individual image and depth data is detected via an image and depth sensor, the face detection frame can surround the acquired face of the individual in the field of view of A/V recording and communication apparatus 14. Upon detecting the individual face and image and depth data face detector module 69 can generate an face base mesh metadata of the individual head and face and distributed base mesh metadata to memory 22 and behavior state processing unit 46 via network 21 under the control of processor 44. Any suitable techniques may be used by face detector module 69 to detect an face of an individual. Characteristic module 11 may obtain the face mesh data structure from AR module 77, in response to AR module 77 generating an face mesh data structure upon receipt of A/V recording and communication apparatus 14 obtaining an view-point signal via behavior state processing unit 46 and associate the face mesh data structure with characteristic points at areas of interest.
For instance, characteristic module 11 can detect facial features at arears of interest; such as the eyes, noses, ears, mouths and eyebrows and associate one or more facial features with facial characteristic points, characteristic module 69 can also detect detailed facial features such as the size of individual eyes, the distance between the individual eyes, the shape and size of the individual nose, the size of the individual lips and the relative position of the individual eyes, nose and lips respectively or in combination and associate the detailed facial features with characteristic points. Characteristic module 11 can associate each set or each respective characteristic points of the face profile mesh and face mesh data structure with an characteristic identifier which may distinguish one set of characteristic points from another. Characteristic module 11 can further distribute the face mesh data structure to characteristic algorithm module 48.
Any suitable techniques may be used by characteristic module 11 to detect facial features and generate characteristic points with the detected facial features of the face mesh data structure.
Characteristic algorithm module 48 may obtain and analyze the face mesh data structure comprising characteristic points to determine an respective value for each respective set of characteristic point(s) associated with the respective face mesh data structure. For instance, characteristic algorithm module 48 may generate an block grid on the face mesh data structure to determine an respective number value for each set of characteristic points.
Characteristic algorithm module 48 may generate an axis on specific regions of the face mesh data structure to determine an respective angle value for each set of characteristic points.
Characteristic algorithm module 48 may generate an circumference table on the face mesh data structure to determine an respective degree value for each set of characteristic points. In response, to characteristic algorithm module 48 processing one or more characteristic point characteristic algorithm module 48 can associate each respective set of characteristic points with an respective value such an number, angle or degree, in conjunction the face mesh data structure comprising an value associated with its characteristic points may be stored in memory 22 temporary and/or distributed to face frame module 40 under the control of processor 44.
Any suitable algorithm techniques may be used by characteristic module 11 to determine and associate an value with the characteristic points of the face mesh data structure.
Augmented reality (AR) module 77 respectively obtains image and depth data (e.g., face base mesh metadata) from face detector module 69 or memory 22 to generate an face mesh data structure that represents an 3D depth profile of the face and head of the individual being used, the AR module 77 may then distributed the face mesh data (e.g., 3D depth profile) structure to face frame module 40, application 81 or memory 22 under the control of processor 44.
Face frame module 40 may obtain the face mesh data structure via AR module 77 and generate an face profile match frame (FPMF) comprising the obtained face mesh data structure upon A/V recording and communication apparatus 14 obtaining an view-point signal via behavior state processing unit 46 via network 21.
For instance, upon receipt of A/V recording and communication apparatus 14 obtaining an view-point signal (VPS) via behavior state processing unit 46 face frame module 40 respectively obtains the face mesh data structure and generates an face profile match frame (FPMF) comprising the face mesh data structure of the respective individual associated with the view-point signal.
Comparing module 82 may obtain an contextual face mesh data structure and an face mesh data structure associated with an face profile match frame generated in response to A/V recording and communication apparatus 14 obtaining an view-point signal via behavior state processing unit 46. Upon obtaining the contextual face mesh data structure and face mesh data structure associated with an face profile match frame generated in response to A/V recording and communication apparatus 14 obtaining an view-point signal comparing module 82 can preform an annalistic task to compare and determine equivalent values with the characteristic points associated with the contextual face mesh data structure generated in response to A/V recording and communication apparatus 14 obtaining biometric data (e.g., image and depth data) and face mesh data structure associated with an face profile match frame generated in response to A/V recording and communication apparatus 14 obtaining the view-point signal.
For instance, the contextual face mesh data structure may have characteristic points that determines the distance between the subscriber eyes with an value of 3.5, while face mesh data structure associated with an face profile match frame generated in response to A/V recording and communication apparatus 14 obtaining an view-point signal may also have characteristic points that determines the distance between the individual eyes but with an value of 3.1 during the annalistic task comparing module 82 may determine that the two values are not equivalent with each other, in response the face match profile frame is configured to alternate under the control of processor 44. Any suitable matching/comparing task may be to determine an equivalent of values associated with the characteristic points of the face mesh data structure.
Memory 22 further stores an application 81 that, when processed by processor 44, enables processor 44 to: obtain an face match profile frame (FMPF) from memory 22 and display the face match profile frame during the recording process. Further, during the recording process the face match profile frame is configured to alternate from an subscriber face to another subscriber face at an predetermine time of 1 to 2 seconds until A/V recording and communication apparatus 14 obtains an equivalent value associated within the contextual face profile match frame and face profile match frame generated in response to A/V recording and communication apparatus 14 obtaining the view-point signal.
For instance, during the recording process an plurality of subscriber may be within the field of view of the recording, further upon deployment of the face match profile frame A/V recording and communication apparatus 14 may be configured to determine the present of an subscriber when the subscriber is in the field of view by way of an image or depth sensor associated to A/V recording and communication apparatus 14 or any other sensing mean known to one skilled in the art(s).
Additionally, upon the A/V recording and communication apparatus 14 detecting the present of individual the face match profile frame may assemble on the subscriber face region for an predetermine time until image and depth data is obtained the image and depth data may be obtained in time in frames, the obtained image and depth data may be processed via face detector module (FDM) 69, characteristic module (CM) 11, characteristic algorithm module (CAM) 48 and augmented reality (ARM) module 77.
Upon processing the contextual obtained image and depth data, where the augmented reality module 77 generating an mesh data structure of the image and depth data obtained by face detector module 69, characteristic module 11 associate the mesh data structure with characteristic points at areas of interest and characteristic algorithm module 48 determining an respective value for each respective or set of characteristic points, in conclusion comparing module 82 is configured to obtain the contextual mesh data structure and the mesh data structure associated with the face match profile frame and preform an annalistic task as to compare/match the characteristic points values associated with the contextual mesh data structure and mesh data structure associated with the face match profile frame, where if each value of each respective or set of characteristic points do not corresponds or match the face match profile frame is configured to alternate to the next individual face within the field of view of the recording.
Memory 22 can further (and optionally) store data 13 relating to image data and depth data obtained via an image sensor and depth sensor associated with A/V recording and communication apparatus 14; for example, in some implementations, the one or more modules can distribute face profile mesh data and an image of the individual face to memory 22 for later purposes. Memory(s) 22 can further store data 13 relating to characteristic points, axis's, profile base mesh's and identity base mesh's of an subscriber to be recalled during one or more called upon tasks. Behavior state processing unit 46 in environment 100 is in communication with one or more A/V recording and communication apparatus 14, VSIM server 99, one or more wireless transceivers 109 and databases 12, 59, 28, 24, 93 & 34. Behavior state processing unit 46 may communicate directly or indirectly with wireless transceivers 109, A/V recording and communication apparatus 14 and databases 12, 59, 28, 24, 93 & 34 by a wired or wireless connection via network 21.
Behavior state processing unit 46 may communicate with one or more A/V recording and communication apparatus 14 to obtain and distribute biometrics of a individual and one or more signals via network 21. Behavior state processing unit 46 may provide instructions to VSIM server 99 to distribute one or more behavior state signals to terminal 1 via network 37. Behavior state processing unit 46 may communicate with one or more wireless transceivers 109 to obtain positioning data of one or more terminals 1 via one or more wireless transceivers 109 via network 21.
Behavior state processing unit 46 comprises processor 6, facial recognition processor 63, one or more memory(s) (29, 39) and communication interfaces 38.
Processor 6 and facial recognition processor (FRP) 63 comprises software and/or hardware or an combination of both, processor(s) (6, 63) can be configured to control a multitude of hardware or software components that may be connected thereon and may also perform various data processing and operations by executing an operating system, application program, or operating system and application instructions stored within memory (29, 39) described herein. Processor(s) (6, 63) may include one or more application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or digital signal processors (DSPs). Facial recognition processor 63 can include an secure enclave processor (SEP) which stores an protects information used for identifying one terminal devices, biometric information operating system information and more. Processor 6 may be configured to distribute positioning request signals to one or more wireless transceiver 109 to obtain acceleration data in order to determine if individuals is still within environment 100.
Communication interfaces (CI) 38 can be provided as interface cards (sometimes referred to as “line cards”). Generally, they control the sending and receiving of data and data packets over a computing network and sometimes support other peripherals used with the behavior state processing unit 46. Among the interfaces that may be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various types of interfaces may be provided Such as, for example, universal serial bus (USB), Serial, Ethernet, Firewire, PCI, parallel, radio frequency (RF), cellular network interfaces, Bluetooth™, near-field communications (e.g., using near-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fast Ethernet interfaces, Gigabit Ethernet interfaces, asynchronous transfer mode (ATM) interfaces, high speed serial interface (HSSI) interfaces, Point of Sale (POS) interfaces, fiber data distributed interfaces (FDDIs), and the like. Generally, such communication interfaces 38 may include ports appropriate for communication with the appropriate media. In some cases, they may also include an independent processor and, in Some instances, volatile and/or nonvolatile memory (e.g., RAM).
Behavior state processing unit 46 include one or more memory(s) (29, 39) coupled to processor(s) (6, 63), memory(s) (29, 39) can be such as an internal memory that may comprise a SSD (Solid State Drive), NAS (Network Attached Storage), Dual-Channel RAM (Random Access Memory), Multi-ROM (Read-Only Memory), Flash Memory (Flash Memory Type), Hard Disk (Hard Disk Type), Multimedia Card Micro (Multimedia Card Micro Type), SRAM (Static Random Access Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), PROM (Programmable Read-Only Memory), may further include a card type memory Compact Flash (CF), Secure Digital (SD), Extreme Digital (xD), Multimedia Card (MMC) or a memory stick. For example, memory (29, 39) can store data information such as applications, programs, hardware or software, and instructions for corresponding components of terminal behavior system (TBS) 5, such described data information will be later explained herein.
Memory 29 can also store data such as positioning location of individuals upon behavior state processing unit 46 obtaining acceleration data and unique identifier from one or more wireless transceivers 109. For instance, positioning location data of each individual may be input into a positioning location log having a time stamp and a indication indicating if individuals is or isn't within environment 100.
For example, memory 29 may store computer-readable and computer-executable instructions and/or software (e.g., such as user tracking and communication engines) for implementing exemplary operations and performing one or more processes as described down below with wireless transceivers 109.
Memory 39 further stores facial recognition tasker application (FRTA) 30 that, when processed by facial recognition processor (FRP) 63, enables facial recognition processor 63 to: analyze and compare the obtained contextual biometric data via AN recording and communication apparatus 14 with historical biometric data (e.g., digital image associated on virtual identification card (VIC)) stored in VICD 12. If facial recognition processor 63 determines an match is found between the obtained contextual biometric data and stored historical biometric data subscriber authentication credentials are stored in user authentication store 59 as an user authentication file.
Additionally, upon receipt of determining an respective match of contextual biometric data with historical biometric data associated with an respective virtual identification card, facial recognition tasker application 30 is also configured to crop and extract specific data associated with an respective virtual identification card such as the subscriber name and subscriber authentication key 83 and associate this information with an respective user authentication file within user authentication database 59. Facial recognition tasker application 30 may also associate each respective user authentication file with an identifier that may distinguish one file from another.
Many facial recognition techniques can be used in operation with facial recognition tasker application 30. For instance, techniques can be used that distinguish an face from other features and measure the various features of the face. Every face has numerous, distinguishable landmarks, and different peaks and valleys that make up respective facial features. The landmarks can be used to define a plurality of nodal points on a face, which can include information about the distance between an individual eyes, the width of the individual nose, the depth of the individual eye sockets, the shape of the user's cheekbones, the length of the individual jaw line. The nodal points of the individual face can be determined from one or more images of an individual face to create a numerical code, known as a faceprint, representing the individual face. The facial recognition can also be performed based on three-dimensional images of the individual face or based on a plurality of two-dimensional images which together can provide three-dimensional information about the individual face. Three-dimensional facial recognition uses distinctive features of the face (e.g., where rigid tissue and bone is most apparent, such as the curves of the eye socket, nose and chin, to identify the individual and to generate a faceprint of the individual. The faceprint of a user can include quantifiable data such as a set of numbers that represent the features on a individual face.
Memory 29 further stores behavior state algorithm application (BSAA) 105 that, when processed by processor 6, enables processor 6 to: analyze and obtain individuals schedule data from employee classifier database 28, student database 24 or miscellaneous database 45, in response behavior state adjustment application 105 respectively bifurcate the individuals schedule data into respective portions to preform one or more equation task in order determine an predetermine behavior state duration time and behavior state duration grace period time, upon receipt of facial recognition tasker application 30 authenticating the respective individual. For instance, if the individual is an employee of environment 100 and individual day work schedule is 8 a.m. (start time) and 5 p.m. (end time) and an 1-hour lunch-brake at 12:00 p.m. Behavior state duration algorithm application 105 obtains an report of the individual schedule data from employee classifier database 28. Further, the schedule data is bifurcated into an first schedule portion and second schedule portion; wherein the first schedule portion comprises the time range of 8 a.m. (start-time) to 12:00 p.m. (end-time), and the second schedule portion comprises the time range of 1 p.m. (start-time) to 5 p.m. (end-time). Upon bifurcating the schedule data into respective portions behavior state duration application 105 preform an first equation process; wherein the first schedule portion end-time is subtracted by an 10-minutes (grace-period) respectively reducing the first schedule portion end-time by 10-minutes and the first schedule portion overall time; wherein the 12:00 p.m. (end-time) is subtracted by an 10-mins (grace-period), 12:00 (end-time)-: 10=11:50 (end-time), in response to subtracting the first schedule portion end-time by 10-minutes the first schedule portion overall time is reduced to 8 a.m. (start-time) to 11:50 a.m. (end-time).
Further, behavior state duration algorithm application 105 preforms an second equation process as to determining the total amount of time of the reduced first schedule portion and second schedule portion respectively; wherein the reduced first schedule portion start-time is subtracted by the reduced first schedule portion end-time to respectively determine the total amount of time of the reduced first schedule portion, 8:00 a.m. (start-time)-11:50 a.m. (end-time)=3.50 hr., and the second schedule portion start-time is subtracted by the second schedule portion end-time, 1 p.m. (start-time)-5 p.m. (end-time)=4 hr. Behavior state duration algorithm application 105 can be configured to format the reduced first schedule portion and second schedule portion into an format better understood by behavior state duration application 26 timer. Further, behavior state duration algorithm application 105 can distribute the above mentioned schedule portion(s) to behavior state duration application 26 under the control of processor 44.
For instance, if individuals is an student of environment 100 and the individual first class is 9 a.m. (start time) to 10 a.m. (end time), second class 10:30 a.m. (start-time) to 11:30 a.m. (end-time) and the third class 11:35 a.m. (start-time) to 12:35 p.m. (end-time). Behavior state duration algorithm application 105 obtains an report of the individual schedule data from student classifier database 24, bifurcating the class schedule into respective portion(s) and preforms an equation process on the portion(s); wherein the first schedule data portion comprises the first class time range of 9 a.m. (start-time) to 10:00 a.m. (end-time), second schedule portion comprises the second class time range of 10:30 a.m. (start-time) to 11:30 a.m. (end-time) and the third schedule portion comprises the third class time range of 11:35 a.m. (start-time) to 12:35 p.m. (end-time).
Secondly, behavior state duration algorithm application 105 preforms an first equation process as to determining the total amount of time of the first schedule portion, second schedule portion and third schedule portion respectively; wherein the first schedule portion start-time is subtracted by the first schedule portion end-time, 9:00 a.m. (start-time)-10:00 a.m. (end-time)=1 hr., the second schedule portion start-time is subtracted by the second schedule portion end-time, 10:30 a.m. (start-time)-11:30 a.m. (end-time)=1 hr and the third schedule portion start-time is subtracted by the third schedule portion end-time, 11:35 a.m. (start-time)-12:35 p.m. (end-time)=1 hr.
Next, behavior state algorithm application 105 respectively preforms an second equation process with an predetermine time associated with the first schedule portion and second schedule portion to determine an respective grace period of the first and second schedule portion(s); wherein the first schedule portion end-time is subtracted by the second schedule portion start-time, 10:00 (fsp end-time)-10:30 (sspstart-time)=: 30 grace period, and the second schedule portion end-time is subtracted by the third schedule portion start-time, 11:30 (ssp end-time)-11:35 (tsp start-time)=: 05 grace period. Behavior state duration algorithm application 105 can be configured to format the above mentioned schedule portion(s) and grace periods into an format better understood by behavior state duration application 26 timer. Additionally, behavior state duration algorithm application 105 can distribute the first schedule portion along with its grace period and second schedule portion along with its grace period and third schedule portion to behavior state duration application 26 under the control of processor 44. Any suitable algorithm techniques may be used by behavior state algorithm application 105 to determine an total time of an respective grace period.
Memory 29 further stores behavior state duration application (BSDA) 26 that, when processed by processor 6, enables processor 6 to: respectively generate an behavior state duration timer file comprising an timer upon receipt of obtaining at lease one schedule portion or grace period via behavior state duration algorithm application 105, further associating the behavior state duration timer file timer with an total time of at lease one or more predetermine schedule portions or grace periods and generating and distributing an view-point signal to A/V recording and communication apparatus 14 upon the one or more timers reaching an predetermine value of 00:00:00.
The behavior state duration timer file comprise biological information (e.g., such as an the individual name), an respective identifier that distinguish one respective behavior state duration timer file from another. Specifically, the timer associated with behavior state duration application 26 is an virtual countdown timer that count downs from an predetermine value, in addition the timer may comprise an input format such as HH:MM:SS for hours (HH), minutes (MM) and seconds (SS).
Further, upon receipt of behavior state duration application 26 obtaining at lease one predetermine schedule portion or grace period via behavior state duration algorithm application 105, behavior state duration application 26 generates an behavior state duration timer file and associates the file with an respective identifier, name of the individual and generates at lease one timer associated with an total time of an predetermine schedule portion or grace period.
For example, if behavior state duration application 26 obtains the reduced first schedule portion total time 8:00 a.m. (start-time)-11:50 a.m. (end-time)=3.50 hr., and the second schedule portion total time 1 p.m. (start-time)-5 p.m. (end-time)=4 hr. mention above in reference to behavior state duration algorithm application 26 discussion, behavior state duration application 26 generates an first timer and second timer. The first timer is set for the reduced first schedule portion total time in the format of 3:50:00 hr., and second timer is set for the second schedule portion total time in the format of 4:00:00 hr. The first timer is set for the reduced first schedule portion total time and is configured to start counting down at the reduced first schedule portion start-time or upon associating the first timer with the reduced first schedule portion total time. Whereas, the second timer is set for the second schedule portion total time and is configured to start counting down at the second schedule portion start-time or upon associating the second timer with the second schedule portion total time. Behavior state processing unit 46 may comprise an internal clock which may allocate processor 44 to determine an current time and date to start the timers.
Further, upon one or more timers values reaching 0:00:00 behavior state processing unit 46 is set to distribute an view-point signal to A/V recording and communication apparatus 14 via network 21, under the control of processor 6.
View-point signal(s) instruct A/V recording and communication apparatus 14 to reobtain biometric data of individuals, in cause of one or more behavior state duration time elapsing.
Memory 29 further stores environment state database (ESD) 73 that comprises data such as a predetermine behavior state at which one or more terminals 1 may obtain and operate at upon individuals entering environment 100 and whereby obtaining the behavior state signal via one or more VSIM servers 99.
For instance, an administrator of environment 100 may access an web page or application via the internet from an external terminal such as an laptop or computer, the application or web page may be in association with behavior state processing unit 46 via an external server. The application or web page may require the administrator to enter authentication credentials such as an password and user name for security purposes. Further, the application or web page may comprise of an drop-down menu or side menu panel labeled environment behavior state, that comprise of three behavior state options (e.g., “keywords”) labeled “behavior state 1”, “behavior state 2”, and “behavior state 3”. Furthermore, when the administrator chooses and selects the desired behavior state (e.g., “keywords”) the “keyword” is distributed to behavior state processing unit 46 where environment state database 73 may be updated with the contextual obtained behavior state (“keyword”) and the administrator may logout of the application or web page. In addition, the environment state database may be updated at an given period with an predetermine behavior state “keyword”.
Referring to database 15 associated with behavior state processing unit 46, database 15 can be configured to hold an substantial amount of data for analytical and comparison purposes.
Further, database 15 can exist within behavior state processing unit 46 as additional memory banks, a server or set of servers, one or more clients, or be distributed between one or more servers and a client. Database 15 includes an biometric data classifier database (BDCD) 34, employee classifier database (ECD) 28, virtual identification card database (VICD) 12, student classifier database (SCD) 24, miscellaneous database (MD) 45 and an user authentication database (UAD) 59.
Behavior state processing unit 46 may access virtual identification card database (VICD) 12 to obtain biometric data, biological information and other information associated with subscriber virtual identification card (VIC), in the event of authenticating an subscriber via facial recognition tasker application 30; for instance, facial recognition tasker application 30 may access the virtual identification card database 12 to preform an comprising task of the contextual biometric data with historical biometric data (e.g., digital image arranged on the virtual identification card), upon receipt of obtaining an suitable match the facial recognition tasker application 30 may also extract/collect other data associated with the virtual identification card such as the subscribers name and subscriber authentication key.
Behavior state processing unit 46 may distribute contextual biometric data to biometric data classifier database (BDCD) 34 via obtaining contextual biometric data from one or more A/V recording and communication apparatus 14 and upon receipt of facial recognition tasker application 30 determining an respective match of the contextual biometric data with an photo associated with an respective virtual identification card (VIC) stored in (VICD) 12. Upon, receipt of obtaining biometric data via A/V recording and communication apparatus 14 behavior state processing unit 46 can generate an respective biometric data classifier file an associate the contextual biometric data within the biometric data classifier file and store it into biometric data classifier database (BDCD) 34, additionally each respective biometric data classifier file may comprise an identifier.
Behavior state processing unit 46 may access student classifier database (SCD) 24 to obtain data pertaining to an subscriber (e.g., student) predetermine schedule. Student classifier database 24 respectively stores biological information, and information relating to class scheduling times and locations of each respective class-room as an respective student classier file. For instance, student scheduling data can be stored within student classifier database 24 via an administrator of environment 100 or other personnel's that handles the scheduling task, this data may also be input into student classifier database 24 via an external terminal via an network.
Behavior state processing unit 46 may access miscellaneous classifier database (MCD) 45 to obtain data pertaining to an subscriber (e.g., guest) predetermine schedule information. Miscellaneous classifier database 45 may also store the subscriber biological information, and information relating to an predetermine reason for visiting environment 100. For instance, the visitor scheduling data can be stored within miscellaneous classifier database 45 via an administrator of environment 100 or other personnel's that handles the scheduling task, this data may also be input into miscellaneous classifier database 45 via an external terminal via network 21.
Behavior state processing unit 46 may access employee classifier database (ECD) 28 to obtain data pertaining to an subscriber (e.g., employee) work schedule, the work schedule may be presented as daily or weekly. Employee classifier database (ECD) 28 respectively store biological information, contextual and historical data relating to event(s) of employee(s) such as clock-in and clock-out times, destination route(s) taking by employee(s) within environment 100 and the employee(s) office/work location(s) within environment 100. The employee data can be stored in employee classifier database 28 as an employee classifier file. The employee data can be collected in real time from one or more image module(s) 14, time clock(s) or any other data collection component(s) configured to obtain and distribute data within environment 100. Additionally, the employee scheduling data may be stored within employee classifier database 28 via an administrator of the environment 100 or other personnel's that handles the scheduling task, this data may also be input into employee classifier database 28 via an external terminal via an network.
Behavior state processing unit 46 may access user authentication database 59 to obtain and verify authentication credentials of an subscriber that has been authenticated via facial recognition tasker application 30. The authentication credentials may comprise of an respective identifier and other data such as biological information (e.g., name and photo of the individual) and the authentication key. For instance, behavior state processing unit 46 may distribute an subscriber authentication signal to VSIM server 99 via network 21, upon receipt VSIM processor 53 may access the service provider database 60 and determine if the obtained data associated with the subscriber authentication signal (e.g., biological information and subscriber authentication key) matches with data stored in service provider database 60 upon distributing an behavior state signal to terminal 1 via cellular network 37.
Wireless transceivers 109 may comprise a wireless transmitter and wireless receiver configured to obtain and distribute wireless transmissions. As one example, wireless transceivers 109 can be configured to distribute and obtain data, directly or indirectly, to and from one or more terminals 1 and/or behavior state processing unit 46 in response to individual entering and/or exiting environment 100. In some embodiments, wireless transceiver 109 can be configured to receive radio transmissions in the frequency range of approximately 2.4 gigahertz (GHz) to approximately 5.6 GHz.
Wireless transceivers 109 may be distributed throughout environment 100 to form a network of wireless transceivers 109 to facilitate communication with terminal 1 when individual is within a proximity range of environment 100, and to facilitate uninterrupted communication with terminal 1 as individual moves throughout environment 100. For example, one or more terminal 1 can transmit acceleration data and unique identifier, and at least one wireless transceivers 109 can be configured to receive the acceleration data and the unique identifier in response to one or more wireless transceivers 109 being within a proximity range of terminal 1. A more precise location of terminal 1 can be determined based upon which of the one or more wireless transceivers 109 receive the transmission from terminal 1, and/or based on signal strength of the transmission when wireless transceivers 109 receive the transmission from terminal 1.
For instance, in response to behavior state processing unit 46 obtaining biometric of individual, generating and distributing positioning request signal to wireless transceivers 109, behavior state processing unit 46 obtains acceleration data and unique identifier from one or more terminals 1 (e.g., via transmission from terminal 1 to behavior state processing unit 46 through one or more wireless transceiver 109), behavior state processing unit 46 can determine individual of terminal 1 is or isn't within environment 100 and can set a memory location in memory to indicate the individual is or isn't within environment 100. As one example, behavior state processing unit 46 can generate a first indicator or parameter in a physical memory location indicating the individual in possession of terminal 1 has arrived within environment 100 in response to obtaining receipt of acceleration data and unique identifier via one or more wireless transceivers 190 disposed in proximity to a entrance of environment 100.
In response to behavior state processing unit 46 generating first indicator in the memory location, behavior state processing unit 46 can generate and distribute authentication signal to VSIM server 99. In another instance, behavior state processing unit 46 can generate second indicator or parameter in a second physical memory location or can reset first indicator in first physical memory location, in response to the receipt of the acceleration data and unique identifier by one or more wireless transceivers 109 disposed in proximity to a exit through the individual in possession of terminal 1 departs to indicate a departure of environment 100.
In response to behavior state processing unit 46 generating second indicator or resetting first indicator, behavior state processing unit 46 can instruct VSIM server 99 to distribute original behavior state signal (OBSS) to terminal 1.
Upon behavior state processing unit 46 generating indicators or parameters set to indicate the presence of individuals in possession terminal 1, behavior state processing unit 46 can be configured to obtain positions of terminals 1 in environment 100 to determine locations of individuals. For example, if a individual is positioned at a predetermine location one or more wireless transceivers 109 can be within range of transmissions from terminal 1 such that some wireless transceivers 109 obtain transmission while others wireless transceivers 109 may be out of range to obtain transmission. In cause, based upon locations of wireless transceivers 109 that obtain transmissions (e.g., acceleration data and unique identifier) in environment 100, behavior state processing unit 46 can estimate a location at which the terminal 1 that sent transmission is located. For instance, different sets of wireless transceivers 109 obtaining acceleration data and unique identifier subsequent to behavior state processing unit 46 generating first indicator, behavior state processing unit 46 can be configured to determine a second location of individuals based on the subset of wireless transceivers 109 that obtains transmissions and signal strength of received wireless transmissions from terminal 1. Wireless transceivers 109 that obtains transmissions from terminal 1 can determine signal strengths at which transmissions was obtained and behavior start processing unit 46 can use the signal strengths to triangulate the estimated location of terminals 1.
In other instances, receipt of acceleration data can also be used to pinpoint a relative location of individuals of terminal 1 and the physical steps taken by individuals. In other cases, upon behavior state processing unit 46 generating first indicator, behavior state processing unit 46 can determine the individuals in possession of terminal 1 is located near a entrance of environment 100. For instance, behavior state processing unit 46 can estimate the individual is at the second location based on its relative location to the first location and the accumulated x, y, and z acceleration data between the first location and second location.
In other instances, when individuals in possession of terminal 1 is at a distance away from wireless transceiver 109 such that it no longer obtains wireless transmission from terminal 1, wireless transceiver 109 can distribute signal to behavior state processing unit 46 indicating that the individual has departed environment 100, and in response behavior state processing unit 46 can instruct VSIM sever 99 to distribute original behavior state signal to terminal 1.
Behavior state processing unit 46 can be communicable coupled to wireless transceivers 109 and can be configured to obtain transmission signal strength data from wireless transceivers 109, and/or to transmit data/information to wireless transceivers 109 for propagation to one or more terminals 1. Behavior state processing unit 46 can be configured to execute user tracking and communication engines to perform one or more processes described herein. Wireless transceivers 109 may comprise one or more processors coupled to one or more memorys having executable instructions configure to carry out the instruction describer herein.
Behavior state processing unit 46, A/V recording and communication apparatus 14 wireless transceivers 109, terminals 1 and VSIM server 99 may communicate via one or more network 21 or cellular network 37. Communication networks may involve the internet, a cellular communication network, a WI-FI network, a packet network, a short-range wireless network or another wired and/or wireless communication network or a combination of any of the foregoing. Behavior state processing unit 46 may communicate with A/V recording and communication apparatus 14 and VSIM server 99 in data packets, messages, or other communications using a common protocol, (e.g., Hypertext Transfer Protocol (HTTP) and/or Hypertext Transfer Protocol Secure (HTTPS).
A/V recording and communication apparatus 14, time clock(s) and data collection component(s) may be configured to translate radio signals and video signals into formats better understood by database 15. In conclusion, behavior state processing unit 46 may include any appropriate combination of hardware and/or software suitable to provide the described above functionality's.
Furthermore, memory(s) (29, 39) storing application(s) (30, 26, 105) is an example of a computer program product, comprising a non-transitory computer usable medium having a computer readable program code adapted to be executed to implement a method, for example a method stored in application(s) (30, 26, 105).
VSIM server 99 can include one or more communication interfaces (70, 8) that can be provided as an interface card (sometimes referred to as “line cards”), that control the sending and receiving of data, data packets and behavior state signals over network 21 and cellular network 37 to and from one or more terminals 1 via cellular tower 65, or another wireless communication network (e.g., Internet).
Among the interfaces that may be provided are Ethernet interfaces, frame relay interfaces, cable interfaces, DSL interfaces, token ring interfaces, and the like. In addition, various types of interfaces may be provided such as, for example, universal serial bus (USB), Serial, Ethernet, Firewire, PCI, parallel, radio frequency (RF), cellular networks, Bluetooth™, near-field communications (e.g., using near-field magnetics), 802.11 (WiFi), frame relay, TCP/IP, ISDN, fast Ethernet interfaces, Gigabit Ethernet interfaces, asynchronous transfer mode (ATM) interfaces and high speed serial interface (HSSI) interfaces. Generally, such communication interfaces (70, 8) may include ports appropriate for communication with the appropriate media. In some cases, they may also include an independent processor and, in some instances, volatile and/or nonvolatile memory (e.g., RAM). More specifically, communication interface 70 is used for communicating with one or more terminals 1 via cellular network 37 via cellular tower 65, while communication interface 8 is used for communicating with one or more terminals 1 and behavior state processing unit 46 via network 21.
VSIM server 99 includes processor 23 comprising software or hardware or an combination of the two, processor 23 may be configured to control a multitude of hardware or software components that may be connected thereon and may also perform various data processing and operations by executing an operating system, application program, or operating system and application instructions stored within RAM 71.
In addition, VSIM server 99 includes service provider database 60 that stores an file for each respective subscriber operating on the system, and data such as the subscriber name, unique identifier (e.g. telephone number), subscribers authentication key 83 and other provisioning information etc. Further, each file stored in service provider database 60 can be labeled (e.g., named) by subscriber authentication key 83. The above mentioned data associated with the subscriber file stored within service provider database 60 may be obtained from terminal 1 during the service account creation/activation set-up of the service offered by the terminal service provider (TSP).
For instance, upon the service provider obtaining an subscriber authentication signal via one or more VSIM servers 99 via behavior state processing unit 46 via network 21, (VSIM) processor 23 may analyze services provider database 60 to find an suitable match with the obtained contextual data associated with the subscriber authentication signal and historical data stored within service provider database 60 in the event of distributing an behavior state signal (volume-control signal or power-down signal) to one or more terminals 1; examples of the comparable matching data may be any of one or an combination of the subscribers name, unique identifier (e.g., telephone number) and subscriber authentication key 83.
RAM 71 can comprises any suitable software or applications known to one skilled in the art(s) configured to respectively preform an comparing, matching and extracting task of data information as mentioned above.
RAM 71 further store instructions and/or codes configured to determine an predetermine behavior state signal (volume-control signal or power-down signal) to distribute to one or more terminals 1 upon obtaining an subscriber authentication signal having a respective “keyword” (e.g., “behavior state 1”, “behavior state 2” or “behavior state 3”) via behavior state processing unit 46 under the control of VSIM processor 23. For instance, behavior state processing unit 46 may distribute an subscriber authentication signal to VSIM server 99 via network 21 comprising an “keyword” such as “behavior state 1”, “behavior state 2”, or “behavior state 3”, further each keyword signifies an respective behavior state, keyword “behavior state 1” and “behavior state 2” can instruct VSIM server 99 to distribute an behavior state signal (e.g., volume control signal) adjusting one or more terminals 1 to silent mode or vibrate mode via cellular network 37 under the control of VSIM processor 23. Upon terminal 1 obtaining the behavior state signal (e.g., volume control signal) behavior state adjustment application 9 may determine an respective behavior state (e.g., determining an ringtone/notification volume adjustment tone/volume level position and output action threshold) terminal 1 is to be adjusted to via subscriber database 43. Alternatively, if VSIM server 99 obtains an subscriber authentication signal comprising the “keyword” “behavior state 3” VSIM server 99 can distribute an behavior state signal (e.g., power-down control signal) to one or more terminals 1 causing terminals 1 to go into an sleep mode under the control of VSIM processor 23.
While
Further,
Referring to
Furthermore,
In process 87, in response to one or more individual arranging in the field of view of at lease one A/V recording and communication apparatus 14 and an recording session is initiated face detector module 69 can obtain one or more frames of depth and/or image data via the image sensor and depth sensor associated with A/V recording and communication apparatus 14, and may also be configured to determine if image and depth data has been obtained. Additionally, upon obtaining the image and depth data associated with one or more individuals, A/V recording and communication apparatus 14 is also configured to obtain an 3D or 2D image of the individual face and/or head and temporary store it in memory 22 as data 13.
Further, in process 16, the one or more image or depth sensors associated with A/V recording and communication apparatus 14 may be configured to determined whether the individual face or head has been detected. If image and depth data is detected, method 66 continues at process 51, otherwise method 66 continues at process 87 if image and depth data is not obtained.
In process 51, upon receipt of face detector module 69 obtaining image and depth data, face detector module 69 generates an face base mesh metadata of the individual face and/or head using one or more frames of the obtained image and depth data, and distributes the face base mesh metadata to memory 22 storing it within the file with the 3D or 2D image of the subscribers face and/or head, and also distributing the face base mesh metadata and 2D or 3D image to biometric data classifier database 34 via behavior state processing unit 46 via network 21 under the control of processor 44.
In process 61, upon receipt of behavior state processing unit 46 distributing an view-point signal to A/V recording and communication apparatus 14, AR module 77 obtains the data associated with the view-point signal such as the face base mesh metadata and AR module 77 generates an face mesh data structure that represents an 3D depth profile of the face base mesh metadata. Further, AR module 77 distributes the face mesh data structure to characteristic module 11 where characteristic module 11 generates characteristic point at areas of interest of the face mesh data structure upon completion the face mesh data structure is distributed to characteristic algorithm module 48. Upon characteristic algorithm module 48 obtaining the face mesh data structure characteristic algorithm module 48 associates each respective or set of characteristic points with an value. Further, characteristic algorithm module 48 distributes the face mesh data structure to face frame module 40, in response to face frame module 40 obtaining the face mesh data structure face frame module 40 generates an face profile match frame (FPMF) comprising the obtained face mesh data structure. Upon face frame module 40 generating the face profile match frame (FPMF) face frame module 40 distributes the face profile match frame to memory 22 and application 81.
In process 55, application 81 is configured to obtain the face profile match frame (FPMF) and deploy the face profile match frame (FPMP) during the recording session. Further, during recording session the face profile match frame (FPMP) is configured to surround an subscribers face region upon A/V recording and communication apparatus 14 detecting image and depth data. In addition, the face profile match frame (FPMP) is furthermore configured to alternate from one subscribers face to another until AN recording and communication apparatus 14 detects equivalent values of all characteristic points associated with the face profile match frame (FPMP) generated upon obtaining an view-point signal and the face profile match frame (FPMF) generated upon AN recording and communication apparatus 14 obtaining contextual biometric data (e.g., image and depth data).
Subscriber behavior state database 43 can comprise data which associates an respective behavior state with an respective ring-tone/notification volume adjustment tone (R/NVAT) volume level, additionally the ringtone/notification volume adjustment tone (R/NVAT) can be associated with an respective ring-tone/notification volume adjustment tone position that traditionally represents an position at which the ringtone/notification volume level (R/NVL) is set at indicated by an volume level marker 64 on sound bar/meter 67 volume level indicator 76. As mentioned above with reference to
For instance, when behavior state adjustment application 9 request terminal 1 original behavior state (OBS) (e.g., prior ringtone/notification volume level) microphone 21 can obtain the contextual ring-tone/notification volume adjustment tone volume level via speaker 74 and sound measuring device 31 can measure the contextual ringtone/notification volume adjustment tone volume level, upon measuring the ring-tone/notification volume adjustment tone volume level the tone can be referenced and matched with an ring-tone/notification volume adjustment “output action threshold” and ringtone/notification adjustment tone position in subscriber behavior state database 43 and stored within subscriber behavior state database 43 as the original behavior state (OBS).
Further, one or more terminals 1 obtains an behavior state signal (e.g., volume control signal) via service provider VSIM server 99 via cellular network 37 (S72). Upon receipt of terminal 1 obtaining an respective behavior state signal (e.g., volume-control signal) behavior state adjustment application 9 obtains the behavior state volume control data associated with the obtained behavior state signal (e.g., behavior state 1 or 2) from subscriber behavior state database 43, the behavior state volume control data is the ringtone/notification adjustment tone/volume level position on sound bar/meter 67 and output action threshold (S54).
For instance, if behavior state 1 is obtained behavior state adjustment application 9 instructs one or more components of terminal 1 to adjust terminal 1 into silent mode/do not disturb mode which is equivalent to ringtone/notification volume adjustment tone/volume level position “0” on sound bar/meter 67 and “output action threshold” T0 (e.g., ringtone/notification volume adjustment tone volume level), if behavior state 2 is obtained behavior state adjustment application 9 instructs one or more components of terminal 1 to adjust terminal 1 to vibrate mode which is equivalent to ringtone/notification volume adjustment tone/volume level position “I” on sound bar/meter 67 and output action threshold T1 (e.g., ringtone/notification volume adjustment tone volume level).
Furthermore, upon behavior state adjustment application 9 determining the predetermine data associated with the obtained behavior state signal (e.g., volume-control signal) from subscriber behavior state database 43, which is equivalent to the predetermine ringtone/notification volume adjustment tone/volume level position associated with the predetermine behavior state signal and “output action threshold” behavior state adjustment application 9 sends an first control signal request to activate microphone 17 for an predetermine time (e.g., 0.5 to 1 seconds) to obtain an sample of the ringtone/notification volume adjustment tone (R/NVAT).
During activation of microphone 17 for the predetermine time of 0.5 to 1 seconds upon microphone 17 obtaining the first control signal request, microphone 17 is configured to obtain an sample of the ringtone/notification volume adjustment tone volume level to determine terminal 1 original behavior state (OBS).
More of, behavior state adjustment application 9 sends an first control signal request to volume adjustment device 49 in conjunction with microphone 17 first control signal request. During, adjustment of volume adjustment device 49 upon volume adjustment device 49 obtaining the first control signal request, volume adjustment device 49 is instructed to adjust the ringtone/notification volume adjustment tone up by one volume level (e.g., one notch). In response to activation of microphone 17 for an predetermine time and volume adjustment device 49 adjusting the ringtone/notification volume up by one level sound measuring device 31 measures volume levels of the ringtone/notification volume adjustment tone (R/NVAT), in response behavior state adjustment application 9 obtains an respective measurement report of the adjusted ringtone/notification volume adjustment tone via sound measuring device 31 (S35).
Further, in response to behavior state adjustment application 9 obtaining the measurement report of the adjusted ringtone/notification volume adjustment tone (AR/NVAT) volume level and an measurement report of the obtained behavior state signal data, wherein the measurement report comprises data indicated to the ringtone/notification volume adjustment tone/volume level (“volume adjustment tone position” and “output action threshold”). Behavior state adjustment application 9 first preforms one or more equation process to determine terminal 1 original behavior state (OBS), wherein the adjusted ring-tone/notification volume adjustment tone “output action threshold” (A/RNVATOAT) is subtracted by 1, which represents the number at which volume adjustment device 49 adjusted the ringtone/notification volume adjustment tone up by upon obtaining the first control signal request; AR/NVATOAT−1=(OBS)), upon determining the original behavior state (OBS) the behavior state adjustment application 9 distributes and stores terminal 1 original behavior state (OBS) within subscriber behavior state database 43.
For, instance if behavior state adjustment application 9 determines behavior state 2 (e.g., vibrate mode) is obtained which is equivalent to “ringtone/notification volume adjustment tone position” 1 and “output action threshold” T1, and terminal 1 ringtone/notification volume adjustment tone current position is 8 on sound bar/meter 67 and “output action threshold” is T8, in response to the volume adjustment device adjusting the ringtone/notification volume adjustment tone up by 1 volume level, in process to determine the original behavior state (OBS) in equation format the equation would be; adjusted ringtone/notification volume adjustment tone “output action threshold” (AR/NVATOAT) T8−1=T7 (OBS) (S42).
In response to behavior state adjustment application 9 determining terminal 1 predetermine original behavior state (OBS), behavior state adjustment application 9 sends an second control signal request to volume adjustment device 49.
During, adjustment of volume adjustment device 49 upon volume adjustment device 49 obtaining the second control signal request, volume adjustment device 49 is instructed to adjust the ringtone/notification volume adjustment tone down by one volume level (e.g., one notch), respectively adjusting terminal 1 back to its original ringtone/notification volume adjustment tone position on sound bar/meter 67 prior to volume adjustment device 49 obtaining the first control signal request (S86).
In response to behavior state adjustment application 9 adjusting terminal 1 back to its original ringtone/notification volume level (R/NVL) position on sound bar/meter 67, behavior state adjustment application 9 preforms an one or more equation process to determine the obtained predetermine behavior state signal “ringtone/notification volume adjustment tone position” and “output action threshold” and adjust terminal 1 to the behavior state associated with the obtained behavior state signal, wherein the adjusted ring-tone/notification volume adjustment tone “output action threshold” (AR/NVATOAT) is subtracted by the obtained behavior state “output action threshold” (BSOAT) associated with the obtained behavior state signal, wherein the equaled value represent the amount of control signal request behavior state adjustment application 9 sends to volume adjustment device 49 in order to adjust terminal 1 to an behavior state associated with the obtained behavior state signal. In equation format the equation would be; (AR/NVATOAT)−(BSOAT)=(VACSR).
For, instance if behavior state adjustment application 9 determines behavior state 2 (BS2) (e.g., vibrate mode) is obtained which is equivalent to “ringtone/notification tone position” 2 on sound bar/meter 67 and “output action threshold” T2 within subscriber database 43, and terminal 1 ringtone/notification volume level currently positioned at 7 on sound bar/meter 67 in response to the volume adjustment device adjusting the ringtone/notification volume level down one level to its original position upon obtaining the second control signal request, which is equivalent to “ringtone/notification tone position” 7, in process to determine the amount of control signal request behavior state adjustment application 9 would need to distribute to volume adjustment device 49 in order to adjust the subscriber terminal 1 to the behavior state associated with the obtained behavior state signal. In equation format the equation would be; adjusted ringtone/notification volume adjustment tone “output action threshold” (AR/NVATOAT) T8−(BSOAT) T2=6 (VACSR).
Upon receipt of behavior state adjustment application 9 determining the amount of control signal request to send to volume adjustment device 49, behavior state adjustment application 9 distributes an first control signal request to volume adjustment device 49, further if behavior state adjustment application 9 determines the value at which the amount of control signal request (CSR) required to adjust terminal 1 to an behavior state associated with the obtained behavior state signal (BSS) is greater than 1 (e.g., (CSR)>1), behavior state adjustment application 9 distributes the control signal request at intervals of an predetermine time wherein the first control signal is distributed to volume control device 49 followed by an corresponding control signal request at the predetermine interval of 05. to 1 seconds.
For example, if behavior state adjustment application 9 determines that it would take an total amount of 6 control signal request (CSR) which is greater than one (e.g., 6>1) to adjust terminal 1 to the behavior state associated with obtained behavior state signal 2 (BSS2), which is equivalent to ringtone/notification volume adjustment tone position” 1 on sound bar/meter 67 and “output action threshold” T1, and terminal 1 ringtone/notification volume level positioned at 7, in response to behavior state adjustment application 9 distributing the six respective control signal request to the volume adjustment device 49, volume adjustment device 49 decreases the ringtone/notification volume level position to 1 on sound bar/meter 67, wherein terminal 1 is now at behavior state 2 (BS2) vibrate mode (VM).
More of, if terminal 1 obtains behavior state signal 2 (BS2) (e.g., vibrate mode) which is equivalent to “ringtone/notification volume level position” 1 on sound bar/meter 67 and “output action threshold” T1 within subscriber database 43, and terminal 1 ringtone/notification volume level position at 1 on sound bar/meter 67. Contrarily, terminal 1 is already in vibrate mode prior to obtaining the behavior state signal (BSS) and volume adjustment device 49 increasing the ringtone/notification volume adjustment tone by 1 volume level. Further, when behavior state adjustment application 9 preforms the second equation process to determine behavior state signal 2 (BSS2) “ringtone/notification volume level position” and “output action threshold”, wherein the adjusted ring-tone/notification volume adjustment tone (AR/NVAT) “output action threshold” is subtracted by behavior state 2 (BS2) “output action threshold” to determine the amount of volume levels volume adjustment device 49 decreases the adjusted ring-tone/notification adjustment tone (e.g., (AR/NVAT) T2−(BS2) T2=00 (decibels), further behavior state adjustment application 9 determines terminal 1 is already at an state associated with behavior state 2 according to the equation value at 0 control signal request required to adjust terminal 1 to the behavior state associated with the obtained behavior state signal (S62).
Referring to
Method 20 is further configured to obtain and process biometric data of one or more individuals when one or more individuals is within an predetermine region of environment 100, upon receipt of behavior state processing unit 46 processing the biometric data one or more VSIM server 99 distributes at lease one behavior state signal (e.g., volume-control signal or power-down control signal) to one or more terminals 1 via cellular network 37, in conjunction adjusting terminal 1 behavior to an behavior associated with the distributed behavior state signal (BSS).
At block B3, when one or more individuals is within an predetermine region of environment 100 one or more A/V recording and communication apparatus 14 obtains biometric data of one or more individuals. The predetermine region can be the main entrance or main lobby of environment 100. Additionally, the biometric data is image an depth data obtained via an image and depth sensor associated with one or more A/V recording and communication apparatus 14.
Upon receipt of A/V recording and communication apparatus 14 obtaining the image and depth data face detector module 69 processes the biometric data in time in frames (e.g., such that the image data can be obtained by face detector module 69 can be at 60 frames/second (fps), and depth data can be obtained at 15 fps).
Specifically, face detector module 69 obtains and process the biometric data (e.g., image and depth data) and generates an face base mesh metadata of individual head and face. Further, upon face detector module 69 generating the face base mesh metadata, the face base mesh metadata and 2D or 3D image of individual face is distributed to behavior state processing unit 46 via network 21 under the control of processor 6, wherein behavior state processing unit 46 stores the face base mesh metadata and 2D or 3D image of individual face within biometric data classifier database 34. Alternatively, the face base mesh metadata is stored within A/V recording and communication apparatus 14 memory 22.
At block B18, upon receipt of behavior state processing unit 46 obtaining the face base mesh metadata and 2D or 3D image of individual face via one or more A/V recording and communication apparatus 14 over network 21, generating an biometric data classifier file, associating the obtained face base mesh metadata and 2D or 3D image of individual face within the biometric data classifier file and storing the biometric data classifier file within biometric data classifier database 34.
Additionally, upon behavior state processing unit 46 obtaining the face base mesh metadata and 2D or 3D image of individual face, facial recognition tasker application 30 obtains the 2D or 3D image of individuals face from the biometric data classifier file stored in biometric data classifier database 34 under the control of facial recognition processor 63.
In conjunction, facial recognition tasker application 63 analyzes virtual identification card database 12 to obtain an suitable match of identity with the 2D or 3D image of individual face and an photo image associated with an respective virtual identification card stored in virtual identification database 12. If facial recognition tasker application 30 determines an suitable match of identity is found within virtual identification card database 12 facial recognition tasker application 30 distributes the facial recognition authentication credentials to user authentication database 59 and stores the credentials as an user authentication file under the control of facial recognition processor 63.
If facial recognition tasker application 30 determines no match was found within virtual identification card database 12 facial recognition processor 63 can instruct facial recognition tasker application 30 to execute an second facial recognition authentication session. If facial recognition tasker application 30 determines no match was found during second facial recognition authentication session method 20 ends and individual is incapable of obtaining behavior state signal (BSS) (68).
In addition to authenticating individual via the facial recognition operation, facial recognition tasker application 30 respectively analyzes biometric data classifier database 34 and updates the biometric data classifier file associated with the 2D or 3D image of individual face that was used to authenticate individual with biological information (e.g., the name of subscriber). For instance, upon facial recognition tasker application 30 analyzing virtual identification card database 12 in search of an suitable match of identity of the 2D or 3D image of individual face, in response to determining an suitable match facial recognition tasker application 30 also collect (e.g., extract) biological data such as individual name and data such as subscriber authentication key from the respective virtual identification card that the suitable match was found in and associate the data with one or more files such as the user authentication file and biometric data classifier file under the control of facial recognition processor 63.
At block B577, upon receipt of behavior state processing unit 46 obtaining the face base mesh metadata and 2D or 3D image of individual face, and preforming one or more comparison task with a 2D or 3D image of individuals face from virtual identification card database 12 under the control of facial recognition processor 63, processor 6 is configured to distribute a positioning request signal to one or more wireless transceivers 109 via network 21. Upon one or more wireless transceiver 109 obtaining positioning signal via behavior state processing unit 46, wireless transceiver 109 is configured to obtain wireless transmission data from one or more terminals 1 in order to determine a presents of individual in possession of terminal 1. Further, if wireless transmission from terminal 1 is obtained by wireless transceiver 109, wireless transceiver 109 distributes positioning detected signal to behavior state processing unit 46 indicating that the individual present is acknowledged within environment 100. On the other hand, if wireless transmission from terminal 1 range is out of reach of wireless transceiver 109 (e.g., individual leaving environment 100 after and/or while biometric task is executed, or individual in possession of terminal wireless transmission is out of reach of wireless transceiver 100 obtaining wireless transmission), wireless transceiver 109 distributes positioning non-detected signal to behavior state processing unit 46 indicating that the individual isn't within environment 100 and method 20 ends (555).
Also, in response to processor 6 obtaining positioning detected signal, processor 6 obtains acceleration data and unique identifier in positioning detected signal. Behavior state processing unit 46 generates a first indicator or parameter in a physical memory location indicating the individual in possession of terminal 1 is within environment 100 in response to obtaining receipt of acceleration data and unique identifier via one or more wireless transceivers 190 disposed in proximity to a entrance of environment 100. In response to behavior state processing unit 46 generating first indicator in the memory location, behavior state processing unit 46 can generate and distribute subscriber authentication signal to VSIM server 99.
At block B55, upon receipt of obtaining positioning detected signal via one or more wireless transceivers 109 and authenticating individual via facial recognition tasker application 30, behavior state processing unit 46 respectively access employee classifier database 28, student classifier database 24, or miscellaneous database 45 to obtain scheduling information regarding individual under the control of processor 6. Upon receipt of behavior state processing unit 46 obtaining scheduling data associated with individual under the control of processor 6, behavior state algorithm application 105 obtains an report of the scheduling data and preforms one or more equation process to bifurcate the schedule data into at lease two portion and to determine an predetermine total amount of time of an predetermine schedule portion and grace period, as described above, wherein one predetermine schedule portion is subtracted by an opposing predetermine schedule portion to determine an total amount of time of an predetermine schedule portion (e.g., behavior state duration). Alternatively, one predetermine schedule portion start or end time is subtracted by an opposing predetermine schedule portion start or end time to determine an total amount of time of an grace period (e.g., behavior state duration).
Upon, behavior state duration application 36 obtaining one or more predetermine schedule portions and/or grace periods total times via behavior state algorithm application 105 under the control of processor 6. Behavior state duration application 36 generates an behavior state duration timer file, in conjunction behavior state duration application 36 generates one or more timers and associates the one or more timers with an predetermine total time of an predetermine scheduling portion or predetermine grace period (e.g., behavior state duration) under the control of processor 6.
At block B83, upon receipt of authenticating one or more subscriber via facial recognition tasker application 30, obtaining scheduling data and associating the one or more total times of an predetermine scheduling portion with the one or more timers, behavior state processing unit 46 respectively generates and distributes an subscriber authentication signal to VSIM server 99 under the control of processor 6 via network 21. In process to distributing the subscriber authentication signal behavior state duration processing unit 46 access user authentication database 59 to obtain data indicating the subscriber biological information such as the subscriber name and subscriber authentication key, in conjunction the behavior state processing unit 46 access behavior state duration application 36 to obtain data (e.g., total time of an predetermine schedule portion and predetermine grace period) associated with an respective timer associated with an behavior state duration timer file. Behavior state processing unit 46 also analyzes environment state database 73 within memory 29 to obtain an “keyword” (e.g., “behavior state 1”, “behavior state 2”, or “behavior state 3”) that refers to an command that instructs VSIM server 99 to distribute an respective behavior state signal (BSS) to one or more terminals 1 via cellular network 37.
It is noted, the subscriber authentication signal comprises data indicated to the individual name subscriber authentication key, one or more total time of an predetermine schedule portion or predetermine grace period (e.g., the behavior state duration time) and an respective “keyword”.
At block B79, upon receipt of the VSIM server 99 obtaining the subscriber authentication signal via behavior state processing unit 46 via network 21, VSIM server 99 access the service provider database 60 to determine if the obtained contextual data associated with the subscriber authentication signal corresponds with the subscriber historical data stored within service provider database 60 under the control of VSIM processor 53. Specifically, the data used to determine an match by VSIM processor 53 is the subscribers name and subscriber authentication key. Upon VSIM processor 53 preforming one or more matching task of the obtained contextual data and historical data stored in service provider database 60, VSIM server 99 respective determines the “keyword” associated with the obtained subscriber authentication signal distributes an respective behavior state signal to terminal 1 via cellular network 37 under the control of VSIM processor 53. More of, if VSIM server 99 obtains an subscriber authentication signal comprising the “keyword” “behavior state 1” VSIM server 99 distributes an behavior state signal (e.g., volume-control signal) that instruct terminal 1 to adjust to silent mode. If VSIM server 99 obtains an subscriber authentication signal comprising the “keyword” “behavior state 2” VSIM server 99 distributes an behavior state signal (e.g., volume-control signal that instruct terminal 1 to adjust to vibrate mode. And if VSIM server 99 obtains an subscriber authentication signal comprising the “keyword” “behavior state 3” VSIM server 99 distributes an behavior state signal (e.g., power-down control signal) that instruct terminal 1 to adjust to an sleep mode.
The behavior state signal comprises two portions an behavior state control signal (e.g., volume-control or power-down signal) which is an behavior at which terminal 1 is to operate at, and an behavior state duration signal which is an predetermine time frame at which terminal 1 is to operate at upon obtaining the behavior state signal. Specifically, the behavior state duration signal comprises data indicated to total time of an predetermine schedule portion or predetermine grace period (e.g., the behavior state duration time) obtained via behavior state processing unit 46 determined by the behavior state algorithm application 105 and behavior state duration application 36. The behavior state signal (e.g., volume-control signal) can instruct behavior state adjustment application 9 to adjust terminal 1 to either silent mode which is equivalent to behavior state 1 or vibrate mode behavior state 2 depending on the respective “keyword” determined by VSIM server 99 upon receipt of obtaining an subscriber authentication signal via behavior state processing unit 46.
At block B97, upon receipt of terminal 1 obtaining an respective behavior state signal via VSIM server 99 via cellular network 37, terminal 1 obtains the behavior state signal and preform at an behavior state associated with the obtained behavior state signal. If terminal 1 obtains an behavior state signal indicating behavior state 3 which is equivalent to an power-down control signal terminal 1 goes into an sleep mode for an predetermine time frame associated with the behavior state duration signal. During sleep mode terminal 1 timing circuitry may be aware of the time, date, and elapsed time this allocates the timing circuitry to reference the behavior state duration time with the time associated with the subscriber terminal 1 clock in order to repower-up terminal 1 when the behavior state duration time elapse. Further, upon receipt of terminal 1 obtaining the behavior state signal and behavior state duration signal, the behavior state duration signal notifies terminal 1 timing circuitry the duration time at which terminal 1 is to behave at an sleep state before repowering-up terminal 1. The data associated with the behavior state duration signal is the total time of an predetermine schedule portion or predetermine grace period (e.g., the behavior state duration time).
Alternatively, if terminal 1 obtains an behavior state signal indicating behavior state 1 or 2 which is equivalent to an volume-control signal (silent mode or vibrate mode) terminal 1 ringtone/notification volume level (R/NVL) is adjusted to an predetermine position on sound bar/meter 67, as described above. More of, the predetermine position on sound bar/marker 67 for behavior state 1 is ringtone/notification volume level (R/NVL) 0, and the predetermine position on sound bar/marker 67 for behavior state 2 is ringtone/notification volume level (R/NVL) 1. Further, upon receipt of terminal 1 obtaining the behavior state signal and behavior state duration signal, behavior state duration application 26 obtains the data associated with the behavior state duration signal and generates an timer and associate the timer with an predetermine time, wherein when the timer reaches an predetermine value of 0:00:00 processor 27 is configured to instruct the behavior state adjustment application 9 to adjust terminal 1 back to its original behavior state (OBS) via behavior state adjustment application 9 sending one or more control signal request to terminal 1 volume adjustment device 49, as described above.
The data associated with the behavior state duration signal is the total time of an predetermine schedule portion or predetermine grace period (e.g., the behavior state duration time).
At 78, in reference to the timer associated with behavior state duration application 26, timing circuitry of terminal 1 and the timer associated with behavior state duration application 36, the timer associated with behavior state duration application 26 and timing circuitry of terminal 1 and the timer associated with behavior state duration application 36 is configured to operate and count equivalent to each other, so that when the timer associated with terminal 1 behavior state duration application 26 reaches an value of 0:00:00 the timer associated with behavior state duration application 36 of behavior state processing unit 46 also reaches an value of 0:00:00. Or when terminal 1 timing circuitry determines that the behavior state duration time associated with the behavior state duration signal elapses the timer associated with behavior state duration application 36 of behavior state processing unit 46 also reaches an value of 0:00:00 and elapses.
Upon receipt of one or more timers associated with behavior state duration application 36 reaching an predetermine value of 0:00:00 behavior state processing unit 46 generates and distributes an view-point signal to A/V recording and communication apparatus 14. In process to distributing the view-point signal behavior state processing unit 46 access biometric data classifier database 34 and obtains the face base mesh metadata of the individual associated with the respective behavior state duration timer file timer that reached the predetermine value of 0:00:00, associating the face base mesh metadata with the view-point signal under the control of processor 6.
Data associated with the view-point signal is the face base mesh metadata of the subscriber.
Upon receipt of A/V recording and communication apparatus 14 obtaining the view-point signal via behavior state processing unit 46 via network 21, augmented reality module 77 obtains the face base mesh metadata and generates an face mesh data structure (e.g., an 3D depth profile of the face and head), upon generating the face mesh data structure augmented reality module 77 distributes the face mesh data structure to characteristic module 11 under the control of processor 44. Upon characteristic module 11 obtaining the face mesh data structure via augmented reality module 77, characteristic module 11 detects facial features of the face mesh data structure and associate facial features of the face mesh data structure with characteristic points at areas of interest so that an value can be associated with the characteristic points and distributes the face mesh data structure to the characteristic algorithm module 48 under the control of processor 44.
Further, upon characteristic algorithm module 48 obtaining the face mesh data structure comprising characteristic points, characteristic algorithm module 48 preforms an one or more equation task to determine an respective value for each respective or set of characteristic point(s) associated with the respective face mesh data structure, in response, to characteristic algorithm module 48 associating an value with one or more characteristic points of the face mesh data structure characteristic algorithm module 48 distributes the face mesh data structure to face frame module 40 under the control of processor 44.
Furthermore, upon face frame module 40 obtaining the face mesh data structure via characteristic algorithm module 48 face frame module 40 generates an face profile match frame (FPMF) comprising the obtained face mesh data structure and distributes the face profile match frame (FPMF) to memory 22 and application 81 under the control of processor 44. For example, the face profile match frame (FPMF) may be the likes of an face or object detection frame displayed during an recording session upon one or more components of an A/V recording and communication apparatus 14 such as an image or depth sensor detecting image or depth data when an individual is within an predetermine field of view.
Upon, receipt of face frame module 40 generating the respective face profile match frame (FPMF) and distributing the face profile match frame to application 81, application 81 respectively obtains the face profile match frame (FMPF) and displays the face profile match frame (FPMF) during the recording session. Further, during the recording session the face match profile frame (FMPF) is configured to alternate from an individual face to another individual face at an predetermine time of 1 to 2 seconds until A/V recording and communication apparatus 14 obtains equivalent values of each set or respective characteristic points associated with the contextual face profile match frame generated upon A/V recording and communication apparatus 14 obtaining biometric data (e.g., image and depth data) and the face profile match frame generated in response to A/V recording and communication apparatus 14 obtaining the view-point signal under the control of processor 44.
More of, upon A/V recording and communication apparatus 14 detecting one or more faces in the field of view of during the recording session via the image or depth sensor associated with A/V recording and communication apparatus 14, the face profile match frame (FPMF) surrounds the individual face for an predetermine time until image and depth data is obtained. Upon A/V recording and communication apparatus 14 detecting image and depth data of the individual face match profile frame surrounds the individual face for an predetermine time until image and depth data is obtained, the obtained image and depth data is then obtained and processed via face detector module (FDM) 69 to generate an face base mesh data of the individual face, augmented reality (ARM) module 77 to generate an face mesh data structure of the face base mesh data, characteristic module (CM) 11 to detect facial features at areas of interest and associate the face mesh data structure with characteristic point at the areas of interest and characteristic algorithm module (CAM) 48 to determine an number value for each respective or set of characteristic points of the face mesh data structure.
Upon receipt of characteristic algorithm module (CAM) 48 determining an value for each respective and set of characteristic points, characteristic algorithm module 48 distributes the face mesh data structure to comparing module 82. Upon comparing module 82 obtaining the contextual face mesh data structure generated in response to A/V recording and communication apparatus 14 obtaining biometric data (e.g., image and depth data), comparing module 82 also obtains the face mesh data structure generated in response to A/V recording and communication apparatus 14 obtaining the view-point signal from within memory 22 and preform an annalistic task to compare and determine equivalent values associated with the characteristic points of the contextual face mesh data structure and face mesh data structure associated with an face profile match frame (FPMF) generated in response to A/V recording and communication apparatus 14 obtaining the view-point signal, where if equivalent values are not determined by comparing module 82 the face match profile frame (FPMF) alternates to another subscriber face in the field of view of the recording until equivalent values are detected, and if equivalent values are determined A/V recording and communication apparatus 14 distributes an view-point detected signal to behavior state processing unit 46 via network 21 under the control of processor 44. Further, the view-point detected signal is configured to translate to behavior state processing unit 46 that one or more A/V recording and communication apparatus 14 has detected individual associated with the view-point signal during the recording session.
For instance, during the detecting of an suitable match of biometric data with the face profile match frame (FPMF) the process may be for only an temporary time frame during the recording session such as 10 to 15 minutes before an view-point non detected signal is generated and distributed to behavior state processing unit 46 under the control of processor 44, wherein this signal indicates one or more A/V recording and communication apparatus 14 could not find an suitable match of biometric data within the predetermine time frame.
Upon, behavior state processing unit 46 obtaining the view-point detected signal the processor 6 is configured to first analyze the one or more timers of behavior state duration application 26 associated with the individual to determine if the time associated with the one or more timers has elapsed, if so processor 6 is configured to instruct behavior state duration application 26 to generate an respective timer and set the timer for ten minutes 00:10:00 (e.g., behavior state duration time) and from there the method 20 starts at B83. This process repeats itself until A/V recording and communication apparatus 14 generates and distributes an view-point non-detected signal or one or more wireless transceiver 109 distributes a positioning non-detected signal to behavior state processing unit 46.
At 557, during operational task of system 5 wireless transceivers 109 is configured to perpetually distribute positioning signals to behavior state processing unit 46 as the individual move throughout environment 100 to obtain wireless transmission in order to indicate a present of individual in response to behavior state processing unit 46 obtaining scheduling data and associating the one or more total times of an predetermine scheduling portion with the one or more timers. If at any giving instance, whether upon behavior state processing unit 46 obtaining scheduling data and associating the one or more total times of an predetermine scheduling portion with the one or more timers, the individual in possession of terminal 1 obtaining behavior state signal (BSS), one or more timers time elapsing or if the individual departs environment 100 before the one or more timers time elapses one or more wireless transceivers 109 is unable to obtain wireless transmission from terminal 1 in action because terminal 1 transmission signals range is out of reach of one or more wireless transceivers 109, wireless transceivers 109 is configure to generate and distribute positioning non-detected signal to behavior state processing unit 46 via network 21. Upon behavior state processing unit 46 obtaining positioning non-detected signal behavior state processing unit 46 is configured instruct VSIM server 99 to generate and distribute a original behavior state signal (OBS) to terminal 1 via network 37.
In addition, in response to behavior state processing unit 46 obtaining positioning non-detected signal, behavior state processing unit 46 is configured to generate a second indicator and replacing the first indicator indicating the individual in possession of the terminal has departed the environment 100, behavior state processing unit 46 is further configured to instruct VSIM server 99 to distribute original behavior state signal (OBSS) to terminal 1.
It should also be understood that the programs, modules, processes, methods, and the likes, described herein are but exemplary implementations and are not related, or limited, to any particular computer, apparatus, or computer programming language. Rather, various types of general-purpose computing machines or customized devices may be used with logic code implemented in accordance with the teachings provided, herein. Further, the order in which the methods of the present invention are performed is purely illustrative in nature. These methods can be performed in any order or in parallel, unless indicated otherwise in the present disclosure. The methods of the present invention may be performed in either hardware, software, or any combination thereof. In particular, some methods may be carried out by software, firmware, or macrocode operating on a single computer a plurality of computers. Furthermore, such software may be transmitted in the form of a computer signal embodied in a carrier wave, and through communication networks by way of Internet portals or websites, for example. Accordingly, the present invention is not limited to any particular platform, unless specifically stated otherwise in the present disclosure. The present invention has been described above with reference to preferred embodiments. However, those skilled in the art will recognize that changes and modifications may be made in these preferred embodiments without departing from the scope of the present invention. Other system architectures, platforms, and implementations that can support various aspects of the invention may be utilized without departing from the essential characteristics as described herein. These and various other adaptations and combinations of features of the embodiments disclosed are within the scope of the invention. The invention is defined by the claims and their full scope of equivalents.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 16274241 | Jun 2019 | US |
| Child | 17156022 | US |
