Patent Grant
11997445
This disclosure relates generally to headset devices, and in particular relates to wireless communication of headset devices.
Wireless headsets are increasingly being used by people. These wireless headsets use, for example, Bluetooth connectivity to transmit the audio signal by radio waves from source devices like cellphones and digital players. Many wireless headsets include both headphones and microphone to enable telecommunication. The wireless headset may enable a user to perform other tasks without restrictions of a wired connection to a device.
Mobile Client System Overview
In particular embodiments, the one or more processor(s) 104 may be operably coupled with the memory 106 to perform various algorithms, processes, or functions. Such programs or instructions executed by the processor(s) 104 may be stored in any suitable article of manufacture that includes one or more tangible, computer-readable media at least collectively storing the instructions or routines, such as the memory 106. The memory 106 may include any suitable articles of manufacture for storing data and executable instructions, such as random-access memory (RAM), read-only memory (ROM), rewritable flash memory, hard drives, and so forth. Also, programs (e.g., an operating system) encoded on such a computer program product may also include instructions that may be executed by the processor(s) 104 to enable the electronic device 100 to provide various functionalities.
In particular embodiments, the sensors 108 may include, for example, one or more cameras (e.g., depth cameras), touch sensors, microphones, motion detection sensors, thermal detection sensors, light detection sensors, time of flight (ToF) sensors, ultrasonic sensors, infrared sensors, or other similar sensors that may be utilized to detect various user inputs (e.g., user voice inputs, user gesture inputs, user touch inputs, user instrument inputs, user motion inputs, and so forth). The cameras 110 may include any number of cameras (e.g., wide cameras, narrow cameras, telephoto cameras, ultra-wide cameras, depth cameras, and so forth) that may be utilized to capture various 2D and 3D images. The display 112 may include any display architecture (e.g., AMLCD, AMOLED, micro-LED, and so forth), which may provide further means by which users may interact and engage with the electronic device 100. In particular embodiments, as further illustrated by
In particular embodiments, the input structures 114 may include any physical structures utilized to control one or more global functions of the electronic device 100 (e.g., pressing a button to power “ON” or power “OFF” the electronic device 100). The network interface 116 may include, for example, any number of network interfaces suitable for allowing the electronic device 100 to access and receive data over one or more cloud-based networks (e.g., a cloud-based service that may service hundreds or thousands of the electronic device 100 and the associated users corresponding thereto) and/or distributed networks. The power source 118 may include any suitable source of power, such as a rechargeable lithium polymer (Li-poly) battery and/or an alternating current (AC) power converter that may be utilized to power and/or charge the electronic device 100 for operation. Similarly, the I/O interface 120 may be provided to allow the electronic device 100 to interface with various other electronic or computing devices, such as one or more auxiliary electronic devices.
In particular embodiments, the accelerometer 316 may be configured to sense motions of the user. As an example and not by way of limitation, the accelerometer 316 may be a 3D accelerometer. While this disclosure describes using an accelerometer 316, the disclosure contemplates using an inertial measurement unit (IMU) in addition to or in replacement of the accelerometer 316. The accelerometer 316 may instantly detect a change in the motion pattern of a user, as well as monitor trends in motion. As an example and not by way of limitation, a user may at a certain time be walking, and at a time later start jogging. The accelerometer 316 may detect accelerations, i.e. changes in movements. For instance, the acceleration or movement pattern would be different between when the user walks, jogs, or sprints. As the accelerometer 316 detects such changes in motion, the example MCU 310 may receive the corresponding sensor data from the accelerometer 316, determine the state of the user, and control the one or more components of the architecture 300 according to a predetermined setting. As an example and not by way of limitation, one or more motion sensors may be located in one or both of the earpieces 302 of the architecture 300. Thereby, through measurements of the accelerometer 316, motions of the head of the user may be registered.
In particular embodiments, in addition to using sensor data from an accelerometer 318 for determining a state of the user, the data from the IR sensor 318 may be filtered so as to avoid disturbance sensor data by filtering out motion artifacts that correspond to the detected relative motions between the earpiece 302 and the ear of the user as detected by the accelerometer 316. In particular embodiments, a second accelerometer may be implemented in the architecture 300.
In particular embodiments, the MCU 310 may be communicatively connected to one or more earpieces 302 As an example and not by way of limitation, the MCU 310 may be an included component of the earpiece 302. A control of the adjustment or the selection of audio output by the MCU 310 may be automatically or autonomously performed by the MCU 310. In particular embodiments, one or more of the data from the accelerometer 316 or IR LED 314 and IR sensor 318 may be offloaded to an electronic device 102. One or more operations to determine the state of the user and the providing, adjusting, and/or selecting playing of audio output through speaker 320 may be performed in a distributed manner. The MCU 310 may include Bluetooth low energy (BLE) 322 and I2C circuit 324. The MCU 310 may include other standardized wireless communication protocols.
In particular embodiments, the power control unit 310 may be included in the architecture for controlling when the different components are to be activated, deactivated, and inactive.
In particular embodiments, the architecture 300 may include microphone 306. The microphone 306 may be included in the earpiece 302. As an example and not by way of limitation, a microphone 306 may be included in a pair of earpieces 302. The microphone 306 may be used to record a sound from the user, such as for handling a phone call or voice command for a connected electronic device 102. In particular embodiments, the MCU 310 may control the microphone 306 to capture ambient sounds and based on the captured ambient sounds, the MCU 310 may control the volume of the audio output to be increased or decreased.
In particular embodiments, the control buttons 312 may be located on the earpiece 302. The control buttons 312 may be physical buttons, a capacitive sensor, or another type of sensor. The control buttons 312 may receive input to control the audio output of the earpiece 302. As an example and not by way of limitation, the control buttons 312 may enable a play/pause/stop function, a fast-forward function, a rewind function, and the like. In particular embodiments, the control buttons 312 may include one or more buttons to perform one or more functions. In particular embodiments, the control buttons 312 may be configured to receive different inputs to perform one or more functions. As an example and not by way of limitation, the control buttons 312 may be configured to receive a double-tap input to perform a different function than a function corresponding to a single-tap input.
Live Conversation Using Hearing Devices
In particular embodiments, the disclose technology includes a system for live audio communication. The live audio communication may be enabled between two wireless hearing devices. As an example and not by way of limitation, the system may enable a direct live audio communication between two Bluetooth headsets. In particular embodiments, a control device may initially connect to two different hearing devices. The control device may be embodied as a mobile phone electronic device, a tablet computer electronic device, a laptop computer electronic device, and so forth. The control device may establish a connection between the two different hearing devices to enable live audio communication between the two hearing devices. As an example and not by way of limitation, a control device may initially connect to two different Bluetooth headsets. The control device may then establish a live audio communication between the two different Bluetooth headsets by redirecting live audio communication between a first Bluetooth headset and the control device to a second Bluetooth headset. In particular embodiments, subsequently to the control device establishing the connection, the two different hearing devices may directly connect to each other for a live audio communication without the control device. In particular embodiments, a communications profile may be used to verify whether a live audio communication should be established between a first hearing device and a second hearing device or to verify whether a live audio communication should be redirected between a first hearing device and a control device to a second hearing device. The communications profile may be used to securely enable the live audio communication such that no unwanted user may establish live audio communication with a hearing device of another user. In particular embodiments, the communications profile may be associated with each hearing device. The communications profile may include one or more parameters for verifying to connect a hearing device to another hearing device. In particular embodiments, the communications profile may be embodied as a Bluetooth profile. The communications profile may indicate a predetermined input that must be received in order to initiate a process to establish a live audio communication between two different hearing devices. As an example and not by way of limitation, two different hearing devices may include a communications profile that indicates a received motion (e.g., a shake) must be received to initiate a process to connect to another hearing device. Additionally, the communications profile may indicate that the received motion must be received within a time period of another received motion. As an example and not by way of limitation, the communications profile may require that two users shake their hearing devices at a time that is within a threshold time period (e.g., 1 second). The hearing devices may establish a live audio communication if the conditions/prerequisites of the communications profile are satisfied. Although this disclosure describes live audio communication in a particular manner, this disclosure contemplates live audio communication in any suitable manner.
Certain technical challenges exist for live audio communication. Without the use of devices, users may have difficulty understanding one another as a result of environmental factors. As an example and not by way of limitation, if a person is wearing a face cover (e.g., a mask) the person may be hard to understand. As another example and not by way of limitation, if two people having a conversation are far apart, then each person may have difficulty talking to one another. Two people may also be having a conversation in a noisy environment. A solution presented by the embodiment disclosed herein to address these challenges may be to implement a live audio communication between two hearing devices. A control device may be used to establish the live audio communication between two or more hearing devices. By enabling live audio communication between two hearing devices, two people may be able to clearly hear each and converse with one another. Certain embodiments disclosed herein may provide one or more technical advantages. A technical advantage of the embodiments may include a secure and fast method of connecting two or more hearing devices to enable a live audio communication between the devices. Certain embodiments disclosed herein may provide none, some, or all of the above technical advantages. One or more other technical advantages may be readily apparent to one skilled in the art in view of the figures, descriptions, and claims of the present disclosure.
In particular embodiments, a first electronic device may receive a predetermined input. In particular embodiments, the first electronic device may be embodied as a hearing device (e.g., hearing device 202 including earpieces 204a, 204b). The first electronic device may be connected to a control device. As an example and not by way of limitation, the first electronic device may be wirelessly connected to a control device, such as via Bluetooth. While this disclosure describes a first electronic device performing certain functions, a control device may be used in conjunction or in replacement of the first electronic device to perform one or more functions. The first electronic device may receive the predetermined input from an input sensor of the first electronic device. As an example and not by way of limitation, the input sensor may be embodied as one or more of a microphone, an inertial measurement unit (IMU), a touch sensor, or the like. In particular embodiments, the predetermined input may be embodied as one or more of an audio input, a received motion, a touch gesture, or the like. In particular embodiments, the predetermined input may correspond to a request to wirelessly connect the first electronic device to one or more second electronic devices. As an example and not by way of limitation, the predetermined input may be a request to connect a first hearing device to a second hearing device to enable a live audio communication. In particular embodiments, the request may be for one or more second electronic devices that are within a communication range of a transceiver of the first electronic device. As an example and not by way of limitation, the transceiver of the first electronic device may be used to establish a communication range of which the first electronic device requests to wirelessly connect to other devices. For instance, if the transceiver has a range of 200 m, then the first electronic device may request to only communicate with second electronic devices within the 200 m range. In particular embodiments, the predetermined input may be associated with an audio redirection function of a control device. In particular embodiments, the control device may be embodied as a mobile phone electronic device, a tablet computer electronic device, a laptop computer electronic device, and so forth. In particular embodiments, the predetermined input may initiate a trigger signal and/or initiate a detection of a trigger signal. As an example and not by way of limitation, the predetermined input may initiate a process to establish a live audio communication. Although this disclosure describes receiving a predetermined input in a particular manner, this disclosure contemplates receiving a predetermined input in any suitable manner.
In particular embodiments, the first electronic device may wirelessly connect to one or more second electronic devices. The first electronic device may use a communications profile associated with each second electronic device to wirelessly connect to each of the second electronic devices. As an example and not by way of limitation, the first electronic device may use a communications profile associated with a second electronic device to determine parameters that need to be verified to connect to the second electronic device. In particular embodiments, the communications profile may comprise one or more parameters for verifying to connect the first electronic device and the one or more second electronic devices. As an example and not by way of limitation, the communications profile may indicate that a text keyword needs to be received to initiate an audio redirection function of a control device. Although this disclosure describes wirelessly connecting a first electronic device to one or more second electronic devices in a particular manner, this disclosure contemplates wirelessly connecting a first electronic device to one or more second electronic devices in any suitable manner.
In particular embodiments, the first electronic device may redirect a live audio communication. The first electronic device may redirect a live audio communication between the first electronic device and a control device to one or more second electronic devices. As an example and not by way of limitation, if a first electronic device is connected to a control device, such as a Bluetooth headset connected to a smartphone, then the Bluetooth headset may redirect the audio sent to the smartphone to another Bluetooth headset. Thereby, redirecting the input received by a microphone of a first Bluetooth headset to a second Bluetooth headset. This allows the second Bluetooth headset hear what a user of the first Bluetooth headset is saying. In particular embodiments, the live audio communication between a first electronic device and a control device may be redirected to one or more second electronic devices in response to a verification of a communications profile associated with each of the one or more second electronic devices. In particular embodiments, the first electronic device may generate a notification confirming redirection of the live audio communication between the first electronic device and the control device to the one or more second electronic devices. As an example and not by way of limitation, the first electronic device may generate a notification to be outputted to the user of the first electronic device to hear a confirmation of redirection of the live audio communication. In particular embodiments, the first electronic device may detect a signal strength between the first electronic device and one or more second electronic devices. The first electronic device may detect that the signal strength between the first electronic device and one or more second electronic devices falls below a threshold signal strength. In particular embodiments, the first electronic device may terminate live audio communication in response to detecting the signal strength falls below the threshold signal strength. As an example and not by way of limitation, a first hearing device may disconnect from a second hearing device in response to detecting the signal strength between the two hearing devices fall below a threshold signal strength. In particular embodiments, a threshold distance may be used based on the threshold signal strength. In particular embodiments, the first electronic device may generate a notification of a termination of the live audio communication. In particular embodiments, the first electronic device may request to connect with a previously connected second electronic device after termination of the live audio communication as a result of the signal strength falling below the threshold signal strength. The first electronic device may periodically send a broadcast message for a specified time period to request to reconnect to a previously connected second device that was disconnected as a result of the signal strength falling below the threshold signal strength. Although this disclosure describes redirecting a live audio communication in a particular, this disclosure contemplates redirecting a live audio communication in any suitable manner.
In particular embodiments, the communications profile may indicate a text keyword may be used to initiate a request for a first electronic device to wirelessly connect to one or more second electronic devices. As an example and not by way of limitation, a user may say a phrase or text keyword to request to connect their hearing device to other hearing devices. A microphone of the first electronic device may be configured to receive an audio input. The first electronic device may detect a text keyword within the audio input. The first electronic device may send the audio input to a control device (e.g., a smartphone) to process the audio input and detect a text keyword. The communications profile used by hearing devices may indicate a text keyword is “Pair”. The user may say “Pair” to initiate a process to enable a live audio communication between his or her hearing device to other hearing devices that have the communications profile. In particular embodiments, the communications profile may indicate a particular device and/or a group of particular devices associated with a text keyword or phrase. As an example and not by way of limitation, if Tom wishes to connect his hearing device to a group of hearing devices associated with his family members, then Tom may need to say, “Pair family”. As another example and not by way of limitation, if Tom wishes to connect his hearing device to Alice's hearing device, Tom may need to say, “Pair with Alice”, “Pair Alice”, or the like to his hearing device. Tom's hearing device may broadcast a message using the communications profile to request to connect to Alice's hearing device. Alice's hearing device may be configured to receive any broadcast message from hearing devices requesting to connect or Alice's hearing device may need to be placed into a receiving mode to receive broadcast messages. For instance, Alice may place her device into a receiving mode by saying “Pair”. Alternatively, Alice may specifically request to pair with Tom's hearing device saying, “Pair Tom”. Users may be able to configure a corresponding text keyword associated with a communications profile of their respective devices. As an example and not by way of limitation, Tom may configure his hearing device to respond to requests comprising “Pair Tom”. Users may use an application to perform these configurations. Users may also generate groups of hearing devices to connect to. As an example and not by way of limitation, a user may add one or more second devices to a group such that if the user requests to connect to the group, their hearing device will request to connect to one or more second devices that are in the group. The first electronic device may redirect a live audio communication between the first electronic device and a control device or direct a live audio communication to one or more second electronic devices. In particular embodiments, the first electronic device may identify one or more second electronic devices based on an audio redirection function of a control device. As an example and not by way of limitation, the first electronic device may identify several second hearing devices within a communication range in response to the user saying an audio input comprising the text keyword. The identified several second hearing devices may be configured based on a communications profile to connect to a hearing device that requests to connect to the respective second hearing device. In particular embodiments, the first electronic device may initiate an audio redirection function of the control device in response to the audio input comprising the text keyword. In particular embodiments, the first electronic device may establish a live audio communication in response to the audio input comprising the text keyword (i.e., without a control device). The first electronic device may verify the communications profile of one or more second electronic devices based on the identified one or more second electronic devices. Although this disclosure describes wirelessly connecting a first electronic device to one or more second electronic devices in a particular manner, this disclosure contemplates wirelessly connecting a first electronic device to one or more second electronic devices in any suitable manner.
In particular embodiments, the communications profile may indicate a received motion may be used to initiate a request for a first electronic device to wirelessly connect to one or more second electronic devices. As an example and not by way of limitation, a user may shake his or her hearing device to request to connect their hearing device to other hearing devices. For instance, a user may take one of the earpieces of the hearing device to hold in his or her hand and shake the earpiece. The first hearing device may include an IMU to detect a received motion. The first electronic device may detect a trigger signal in response to the received motion, where the trigger signal is associated with an audio redirection function of a control device or a request to connect to one or more second devices. The first electronic device may send a broadcast message to one or more second electronic devices to request to connect to the one or more second electronic devices. The broadcast message may include a time of the received signal, a type of input received, a device identification, and the like. As an example and not by way of limitation, the broadcast message may include 3:01 PM for the time an input is received, a shake signal is received, and a device identification indicating the device is associated with a first user. The trigger signal may configure the first electronic device to be placed into a listening mode to receive one or more broadcast messages. The trigger signal may configure the first electronic device to send the broadcast message to a control device to process. The control device may be configured to receive one or more broadcast messages from one or more second electronic devices. The first electronic device may use a transceiver to receive one or more broadcast messages indicative of the one or more second electronic devices receiving an input that matches the received motion at a particular time. The control device may process the broadcast messages from one or more second electronic devices, or the first electronic device may process the broadcast messages from the one or more second electronic devices. In particular embodiments, the broadcast messages from the one or more second electronic devices may be processed to determine a same received motion was received at a corresponding second electronic device as a received motion received by the first electronic device. As an example and not by way of limitation, the broadcast message may be processed to determine whether a shake input was received at a second electronic device similarly to a shake input received at a first electronic device. In particular embodiments, the received motion may be customized for particular devices. As an example and not by way of limitation, a communications profile of a first electronic device may be configured to connect (e.g., directly or by redirecting live audio communication as described herein) to a second electronic device if a shake input is received and the first electronic device may be configured to connect to a third electronic device if a double shake input is received. The configurations may be done within an application to specify what received motion indicates a request to connect to a particular hearing device. In particular embodiments, the broadcast messages from the one or more second electronic devices may be processed to determine the input received by the corresponding second electronic device is received within a threshold time period of the input received by the first electronic device. As an example and not by way of limitation, the broadcast message from a second electronic device may be processed to determine that the input was received within 1 second of the input received by the first electronic device. In particular embodiments, the first electronic device may identify one or more second electronic devices that received the same input (i.e., same received motion) within a threshold time period of the input being received by the first electronic device. The first electronic device may verify the communications profile of one or more second electronic devices based on the identified one or more second electronic devices. Although this disclosure describes wirelessly connecting a first electronic device to one or more second electronic devices in a particular manner, this disclosure contemplates wirelessly connecting a first electronic device to one or more second electronic devices in any suitable manner.
In particular embodiments, the communications profile may indicate a touch gesture may be used to initiate a request for a first electronic device to wirelessly connect to one or more second electronic devices. As an example and not by way of limitation, a user may touch a touch sensor of a first electronic device to perform a touch gesture, such as a tap gesture. For instance, a user may tap a touch sensor of an earpiece. The first electronic device may detect a trigger signal in response to the received touch gesture, where the trigger signal is associated with an audio redirection function of a control device or a request to connect to one or more second devices. The first electronic device may send a broadcast message to one or more second electronic devices to request to connect to the one or more second electronic devices. The broadcast message may include a time of the received signal, a type of input received, a device identification, and the like. As an example and not by way of limitation, the broadcast message may include 4:02 PM for the time an input is received, a tap gesture is received, and a device identification indicating the device is associated with a first user. The trigger signal may configure the first electronic device to be placed into a listening mode to receive one or more broadcast messages. The trigger signal may configure the first electronic device to send the broadcast message to a control device to process. The control device may be configured to receive one or more broadcast messages from one or more second electronic devices. The first electronic device may use a transceiver to receive one or more broadcast messages indicative of the one or more second electronic devices receiving an input that matches the touch gesture at a particular time. The control device may process the broadcast messages from one or more second electronic devices, or the first electronic device may process the broadcast messages from the one or more second electronic devices. In particular embodiments, the broadcast messages from the one or more second electronic devices may be processed to determine a same touch gesture was received at a corresponding second electronic device as a touch gesture received by the first electronic device. As an example and not by way of limitation, the broadcast message may be processed to determine whether a touch gesture was received at a second electronic device similarly to a touch gesture received at a first electronic device. In particular embodiments, the touch gesture may be customized for particular devices. As an example and not by way of limitation, a communications profile of a first electronic device may be configured to connect (e.g., directly or by redirecting live audio communication as described herein) to a second electronic device if a tap gesture is received and the first electronic device may be configured to connect to a third electronic device if a double tap gesture is received. The configurations may be done within an application to specify what touch gesture indicates a request to connect to a particular hearing device. In particular embodiments, the broadcast messages from the one or more second electronic devices may be processed to determine the input received by the corresponding second electronic device is received within a threshold time period of the input received by the first electronic device. As an example and not by way of limitation, the broadcast message from a second electronic device may be processed to determine that the input was received within 1 second of the input received by the first electronic device. In particular embodiments, the first electronic device may identify one or more second electronic devices that received the same input (i.e., same touch gesture) within a threshold time period of the input being received by the first electronic device. The first electronic device may verify the communications profile of one or more second electronic devices based on the identified one or more second electronic devices. In particular embodiments, the first electronic device may be configured to receive multiple predetermined inputs to initiate a request for a first electronic device to wirelessly connect to one or more second electronic devices. As an example and not by way of limitation, a user may need to perform a tap gesture on an earpiece of a hearing device to initiate a listening mode where the user may then say an audio input including a text keyword to initiate a request to connect to one or more second electronic devices. Although this disclosure describes wirelessly connecting a first electronic device to one or more second electronic devices in a particular manner, this disclosure contemplates wirelessly connecting a first electronic device to one or more second electronic devices in any suitable manner.
In particular embodiments, the first electronic device may directly connect to one or more second electronic devices. The first electronic device may broadcast a message to one or more second electronic devices to request to connect to the second electronic device. The second electronic device may also broadcast a message to one or more second devices (including the first electronic device) to request to connect to the second electronic device. The second electronic device may similarly process a broadcast message as the first electronic device and/or send the broadcast message to a control device to process. In particular embodiments, the first electronic device may access a device identification of each of the one or more second electronic devices that are within the communication range of the transceiver of the first electronic device. In particular embodiments, the first electronic device may use a control device to access a device identification of each of the one or more second electronic devices within a communication range of the transceiver of the first electronic device. In particular embodiments, the first electronic device may determine whether the first electronic device has been previously connected to each of the second electronic devices within communication range based on the device identification of the one or more second electronic devices. In particular embodiments, the first electronic device may use a control device to determine whether the first electronic device has been previously connected to each of the second electronic devices within communication range based on the device identification of the one or more second electronic devices. In particular embodiments, the verification of the communications profile may comprise determining the first electronic device has been previously connected to the one or more second electronic devices.
Referring to
The method 1000 may begin at step 1010 with the one or more processing devices (e.g., hearing device 202) receiving, from an input sensor of the first electronic device, a predetermined input. For example, in particular embodiments, the predetermined input may correspond to a request to wirelessly connect the first electronic device to one or more second electronic devices within a communication range of a transceiver of the first electronic device. The method 1000 may then continue at step 1020 with the one or more processing devices (e.g., hearing device 202) wirelessly connecting, using the transceiver of the first electronic device responsive to receiving the predetermined input, to the one or more second electronic devices using a communications profile associated with each of the one or more second electronic devices. For example, in particular embodiments, the communications profile may comprise one or more parameters for verifying to connect the first electronic device and the one or more second electronic devices. The method 1000 may then continue at step 1030 with the one or more processing devices (e.g., hearing device 202) redirecting a live audio communication between the first electronic device and a control device to the one or more second electronic devices in response to a verification of the communications profile associated with each of the one or more second electronic devices. Particular embodiments may repeat one or more steps of the method of
AI Architecture
In particular embodiments, as depicted by
In particular embodiments, the deep learning algorithms 1118 may include any artificial neural networks (ANNs) that may be utilized to learn deep levels of representations and abstractions from large amounts of data. For example, the deep learning algorithms 1118 may include ANNs, such as a multilayer perceptron (MLP), an autoencoder (AE), a convolution neural network (CNN), a recurrent neural network (RNN), long short term memory (LSTM), a grated recurrent unit (GRU), a restricted Boltzmann Machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a generative adversarial network (GAN), and deep Q-networks, a neural autoregressive distribution estimation (NADE), an adversarial network (AN), attentional models (AM), deep reinforcement learning, and so forth.
In particular embodiments, the supervised learning algorithms 1120 may include any algorithms that may be utilized to apply, for example, what has been learned in the past to new data using labeled examples for predicting future events. For example, starting from the analysis of a known training dataset, the supervised learning algorithms 1120 may produce an inferred function to make predictions about the output values. The supervised learning algorithms 1120 can also compare its output with the correct and intended output and find errors in order to modify the supervised learning algorithms 1120 accordingly. On the other hand, the unsupervised learning algorithms 1122 may include any algorithms that may applied, for example, when the data used to train the unsupervised learning algorithms 1122 are neither classified or labeled. For example, the unsupervised learning algorithms 1122 may study and analyze how systems may infer a function to describe a hidden structure from unlabeled data.
In particular embodiments, the NLP algorithms and functions 1106 may include any algorithms or functions that may be suitable for automatically manipulating natural language, such as speech and/or text. For example, in particular embodiments, the NLP algorithms and functions 1106 may include content extraction algorithms or functions 1124, classification algorithms or functions 1126, machine translation algorithms or functions 1128, question answering (QA) algorithms or functions 1130, and text generation algorithms or functions 1132. In particular embodiments, the content extraction algorithms or functions 1124 may include a means for extracting text or images from electronic documents (e.g., webpages, text editor documents, and so forth) to be utilized, for example, in other applications.
In particular embodiments, the classification algorithms or functions 1126 may include any algorithms that may utilize a supervised learning model (e.g., logistic regression, naïve Bayes, stochastic gradient descent (SGD), k-nearest neighbors, decision trees, random forests, support vector machine (SVM), and so forth) to learn from the data input to the supervised learning model and to make new observations or classifications based thereon. The machine translation algorithms or functions 1128 may include any algorithms or functions that may be suitable for automatically converting source text in one language, for example, into text in another language. The QA algorithms or functions 1130 may include any algorithms or functions that may be suitable for automatically answering questions posed by humans in, for example, a natural language, such as that performed by voice-controlled personal assistant devices. The text generation algorithms or functions 1132 may include any algorithms or functions that may be suitable for automatically generating natural language texts.
In particular embodiments, the expert systems 1108 may include any algorithms or functions that may be suitable for simulating the judgment and behavior of a human or an organization that has expert knowledge and experience in a particular field (e.g., stock trading, medicine, sports statistics, and so forth). The computer-based vision algorithms and functions 1110 may include any algorithms or functions that may be suitable for automatically extracting information from images (e.g., photo images, video images). For example, the computer-based vision algorithms and functions 1110 may include image recognition algorithms 1134 and machine vision algorithms 1136. The image recognition algorithms 1134 may include any algorithms that may be suitable for automatically identifying and/or classifying objects, places, people, and so forth that may be included in, for example, one or more image frames or other displayed data. The machine vision algorithms 1136 may include any algorithms that may be suitable for allowing computers to “see”, or, for example, to rely on image sensors cameras with specialized optics to acquire images for processing, analyzing, and/or measuring various data characteristics for decision making purposes.
In particular embodiments, the speech recognition algorithms and functions 1112 may include any algorithms or functions that may be suitable for recognizing and translating spoken language into text, such as through automatic speech recognition (ASR), computer speech recognition, speech-to-text (STT), or text-to-speech (TTS) in order for the computing to communicate via speech with one or more users, for example. In particular embodiments, the planning algorithms and functions 1138 may include any algorithms or functions that may be suitable for generating a sequence of actions, in which each action may include its own set of preconditions to be satisfied before performing the action. Examples of AI planning may include classical planning, reduction to other problems, temporal planning, probabilistic planning, preference-based planning, conditional planning, and so forth. Lastly, the robotics algorithms and functions 1140 may include any algorithms, functions, or systems that may enable one or more devices to replicate human behavior through, for example, motions, gestures, performance tasks, decision-making, emotions, and so forth.
Herein, a computer-readable non-transitory storage medium or media may include one or more semiconductor-based or other integrated circuits (ICs) (such, as for example, field-programmable gate arrays (FPGAs) or application-specific ICs (ASICs)), hard disk drives (HDDs), hybrid hard drives (HHDs), optical discs, optical disc drives (ODDs), magneto-optical discs, magneto-optical drives, floppy diskettes, floppy disk drives (FDDs), magnetic tapes, solid-state drives (SSDs), RAM-drives, SECURE DIGITAL cards or drives, any other suitable computer-readable non-transitory storage media, or any suitable combination of two or more of these, where appropriate. A computer-readable non-transitory storage medium may be volatile, non-volatile, or a combination of volatile and non-volatile, where appropriate.
Systems and Methods
This disclosure contemplates any suitable number of computer systems 1200. This disclosure contemplates computer system 1200 taking any suitable physical form. As example and not by way of limitation, computer system 1200 may be an embedded computer system, a system-on-chip (SOC), a single-board computer system (SBC) (e.g., a computer-on-module (COM) or system-on-module (SOM)), a desktop computer system, a laptop or notebook computer system, an interactive kiosk, a mainframe, a mesh of computer systems, a mobile telephone, a personal digital assistant (PDA), a server, a tablet computer system, an augmented/virtual reality device, or a combination of two or more of these. Where appropriate, computer system 1200 may include one or more computer systems 1200; be unitary or distributed; span multiple locations; span multiple machines; span multiple data centers; or reside in a cloud, which may include one or more cloud components in one or more networks.
Where appropriate, one or more computer systems 1200 may perform without substantial spatial or temporal limitation one or more steps of one or more methods described or illustrated herein. As an example, and not by way of limitation, one or more computer systems 1200 may perform in real time or in batch mode one or more steps of one or more methods described or illustrated herein. One or more computer systems 1200 may perform at different times or at different locations one or more steps of one or more methods described or illustrated herein, where appropriate.
In particular embodiments, computer system 1200 includes a processor 1202, memory 1204, storage 1206, an input/output (I/O) interface 1208, a communication interface 1210, and a bus 1212. Although this disclosure describes and illustrates a particular computer system having a particular number of particular components in a particular arrangement, this disclosure contemplates any suitable computer system having any suitable number of any suitable components in any suitable arrangement. In particular embodiments, processor 1202 includes hardware for executing instructions, such as those making up a computer program. As an example, and not by way of limitation, to execute instructions, processor 1202 may retrieve (or fetch) the instructions from an internal register, an internal cache, memory 1204, or storage 1206; decode and execute them; and then write one or more results to an internal register, an internal cache, memory 1204, or storage 1206. In particular embodiments, processor 1202 may include one or more internal caches for data, instructions, or addresses. This disclosure contemplates processor 1202 including any suitable number of any suitable internal caches, where appropriate. As an example, and not by way of limitation, processor 1202 may include one or more instruction caches, one or more data caches, and one or more translation lookaside buffers (TLBs). Instructions in the instruction caches may be copies of instructions in memory 1204 or storage 1206, and the instruction caches may speed up retrieval of those instructions by processor 1202.
Data in the data caches may be copies of data in memory 1204 or storage 1206 for instructions executing at processor 1202 to operate on; the results of previous instructions executed at processor 1202 for access by subsequent instructions executing at processor 1202 or for writing to memory 1204 or storage 1206; or other suitable data. The data caches may speed up read or write operations by processor 1202. The TLBs may speed up virtual-address translation for processor 1202. In particular embodiments, processor 1202 may include one or more internal registers for data, instructions, or addresses. This disclosure contemplates processor 1202 including any suitable number of any suitable internal registers, where appropriate. Where appropriate, processor 1202 may include one or more arithmetic logic units (ALUs); be a multi-core processor; or include one or more processors 1202. Although this disclosure describes and illustrates a particular processor, this disclosure contemplates any suitable processor.
In particular embodiments, memory 1204 includes main memory for storing instructions for processor 1202 to execute or data for processor 1202 to operate on. As an example, and not by way of limitation, computer system 1200 may load instructions from storage 1206 or another source (such as, for example, another computer system 1200) to memory 1204. Processor 1202 may then load the instructions from memory 1204 to an internal register or internal cache. To execute the instructions, processor 1202 may retrieve the instructions from the internal register or internal cache and decode them. During or after execution of the instructions, processor 1202 may write one or more results (which may be intermediate or final results) to the internal register or internal cache. Processor 1202 may then write one or more of those results to memory 1204. In particular embodiments, processor 1202 executes only instructions in one or more internal registers or internal caches or in memory 1204 (as opposed to storage 1206 or elsewhere) and operates only on data in one or more internal registers or internal caches or in memory 1204 (as opposed to storage 1206 or elsewhere).
One or more memory buses (which may each include an address bus and a data bus) may couple processor 1202 to memory 1204. Bus 1212 may include one or more memory buses, as described below. In particular embodiments, one or more memory management units (MMUs) reside between processor 1202 and memory 1204 and facilitate accesses to memory 1204 requested by processor 1202. In particular embodiments, memory 1204 includes random access memory (RAM). This RAM may be volatile memory, where appropriate. Where appropriate, this RAM may be dynamic RAM (DRAM) or static RAM (SRAM). Moreover, where appropriate, this RAM may be single-ported or multi-ported RAM. This disclosure contemplates any suitable RAM. Memory 1204 may include one or more memory devices 1204, where appropriate. Although this disclosure describes and illustrates particular memory, this disclosure contemplates any suitable memory.
In particular embodiments, storage 1206 includes mass storage for data or instructions. As an example, and not by way of limitation, storage 1206 may include a hard disk drive (HDD), a floppy disk drive, flash memory, an optical disc, a magneto-optical disc, magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. Storage 1206 may include removable or non-removable (or fixed) media, where appropriate. Storage 1206 may be internal or external to computer system 1200, where appropriate. In particular embodiments, storage 1206 is non-volatile, solid-state memory. In particular embodiments, storage 1206 includes read-only memory (ROM). Where appropriate, this ROM may be mask-programmed ROM, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), electrically alterable ROM (EAROM), or flash memory or a combination of two or more of these. This disclosure contemplates mass storage 1206 taking any suitable physical form. Storage 1206 may include one or more storage control units facilitating communication between processor 1202 and storage 1206, where appropriate. Where appropriate, storage 1206 may include one or more storages 1206. Although this disclosure describes and illustrates particular storage, this disclosure contemplates any suitable storage.
In particular embodiments, I/O interface 1208 includes hardware, software, or both, providing one or more interfaces for communication between computer system 1200 and one or more I/O devices. Computer system 1200 may include one or more of these I/O devices, where appropriate. One or more of these I/O devices may enable communication between a person and computer system 1200. As an example, and not by way of limitation, an I/O device may include a keyboard, keypad, microphone, monitor, mouse, printer, scanner, speaker, still camera, stylus, tablet, touch screen, trackball, video camera, another suitable I/O device or a combination of two or more of these. An I/O device may include one or more sensors. This disclosure contemplates any suitable I/O devices and any suitable I/O interfaces 1206 for them. Where appropriate, I/O interface 1208 may include one or more device or software drivers enabling processor 1202 to drive one or more of these I/O devices. I/O interface 1208 may include one or more I/O interfaces 1206, where appropriate. Although this disclosure describes and illustrates a particular I/O interface, this disclosure contemplates any suitable I/O interface.
In particular embodiments, communication interface 1210 includes hardware, software, or both providing one or more interfaces for communication (such as, for example, packet-based communication) between computer system 1200 and one or more other computer systems 1200 or one or more networks. As an example, and not by way of limitation, communication interface 1210 may include a network interface controller (NIC) or network adapter for communicating with an Ethernet or other wire-based network or a wireless NIC (WNIC) or wireless adapter for communicating with a wireless network, such as a WI-FI network. This disclosure contemplates any suitable network and any suitable communication interface 1210 for it.
As an example, and not by way of limitation, computer system 1200 may communicate with an ad hoc network, a personal area network (PAN), a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), or one or more portions of the Internet or a combination of two or more of these. One or more portions of one or more of these networks may be wired or wireless. As an example, computer system 1200 may communicate with a wireless PAN (WPAN) (such as, for example, a BLUETOOTH WPAN), a WI-FI network, a WI-MAX network, a cellular telephone network (such as, for example, a Global System for Mobile Communications (GSM) network), or other suitable wireless network or a combination of two or more of these. Computer system 1200 may include any suitable communication interface 1210 for any of these networks, where appropriate. Communication interface 1210 may include one or more communication interfaces 1210, where appropriate. Although this disclosure describes and illustrates a particular communication interface, this disclosure contemplates any suitable communication interface.
In particular embodiments, bus 1212 includes hardware, software, or both coupling components of computer system 1200 to each other. As an example, and not by way of limitation, bus 1212 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a front-side bus (FSB), a HYPERTRANSPORT (HT) interconnect, an Industry Standard Architecture (ISA) bus, an INFINIBAND interconnect, a low-pin-count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCIe) bus, a serial advanced technology attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or a combination of two or more of these. Bus 1212 may include one or more buses 1212, where appropriate. Although this disclosure describes and illustrates a particular bus, this disclosure contemplates any suitable bus or interconnect.
Herein, “or” is inclusive and not exclusive, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A or B” means “A, B, or both,” unless expressly indicated otherwise or indicated otherwise by context. Moreover, “and” is both joint and several, unless expressly indicated otherwise or indicated otherwise by context. Therefore, herein, “A and B” means “A and B, jointly or severally,” unless expressly indicated otherwise or indicated otherwise by context.
Herein, “automatically” and its derivatives means “without human intervention,” unless expressly indicated otherwise or indicated otherwise by context.
The embodiments disclosed herein are only examples, and the scope of this disclosure is not limited to them. Embodiments according to the invention are in particular disclosed in the attached claims directed to a method, a storage medium, a system and a computer program product, wherein any feature mentioned in one claim category, e.g. method, can be claimed in another claim category, e.g. system, as well. The dependencies or references back in the attached claims are chosen for formal reasons only. However, any subject matter resulting from a deliberate reference back to any previous claims (in particular multiple dependencies) can be claimed as well, so that any combination of claims and the features thereof are disclosed and can be claimed regardless of the dependencies chosen in the attached claims. The subject-matter which can be claimed comprises not only the combinations of features as set out in the attached claims but also any other combination of features in the claims, wherein each feature mentioned in the claims can be combined with any other feature or combination of other features in the claims. Furthermore, any of the embodiments and features described or depicted herein can be claimed in a separate claim and/or in any combination with any embodiment or feature described or depicted herein or with any of the features of the attached claims.
The scope of this disclosure encompasses all changes, substitutions, variations, alterations, and modifications to the example embodiments described or illustrated herein that a person having ordinary skill in the art would comprehend. The scope of this disclosure is not limited to the example embodiments described or illustrated herein. Moreover, although this disclosure describes and illustrates respective embodiments herein as including particular components, elements, feature, functions, operations, or steps, any of these embodiments may include any combination or permutation of any of the components, elements, features, functions, operations, or steps described or illustrated anywhere herein that a person having ordinary skill in the art would comprehend. Furthermore, reference in the appended claims to an apparatus or system or a component of an apparatus or system being adapted to, arranged to, capable of, configured to, enabled to, operable to, or operative to perform a particular function encompasses that apparatus, system, component, whether or not it or that particular function is activated, turned on, or unlocked, as long as that apparatus, system, or component is so adapted, arranged, capable, configured, enabled, operable, or operative. Additionally, although this disclosure describes or illustrates particular embodiments as providing particular advantages, particular embodiments may provide none, some, or all of these advantages.
This application claims the benefit under 35 U.S.C. § 119(e) of U.S. Provisional Patent Application No. 63/038,253, filed 12 Jun. 2020, which is incorporated herein by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 9949040 | Bergmann | Apr 2018 | B2 |
| 10205814 | Boesen | Feb 2019 | B2 |
| 10206024 | Fleizach | Feb 2019 | B2 |
| 10291591 | Yu | May 2019 | B2 |
| 10531247 | Kashef | Jan 2020 | B2 |
| 20060258289 | Dua | Nov 2006 | A1 |
| 20190102145 | Wilberding | Apr 2019 | A1 |
| 20190222917 | Perry | Jul 2019 | A1 |
| 20190246196 | Han | Aug 2019 | A1 |
| 20190356980 | Koss | Nov 2019 | A1 |
| 20190379964 | Pergament | Dec 2019 | A1 |
| 20200120727 | Batta | Apr 2020 | A1 |
| Number | Date | Country |
|---|---|---|
| 2016-0036459 | Apr 2016 | KR |
| 2018-0107637 | Oct 2018 | KR |
| 2019-0080305 | Jul 2019 | KR |
| Entry |
|---|
| International Search Report and Written Opinion for International Application No. PCT/KR2021/007303, 9 pages, dated Sep. 16, 2021. |
| Number | Date | Country | |
|---|---|---|---|
| 20210392427 A1 | Dec 2021 | US |
| Number | Date | Country | |
|---|---|---|---|
| 63038253 | Jun 2020 | US |
