A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent file or records, but otherwise reserves all copyright or rights whatsoever. @2023 Eight Count LLC.
One technical field of the present disclosure is computer-implemented data collection and analysis based on human motion tracking and biological indicator tracking. Another technical field is athletics training software, including software that incorporates data gathered by one or more wearable sensors.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.
In recent years, sensor technology has become increasingly prevalent and has found widespread application in various fields, including athletics training. Sensors that track motion and health data can provide useful information to individuals engaged in physical activities. In particular, sensors that track motion data can provide valuable insights into an athlete's performance.
Existing systems for monitoring motion data typically require expensive equipment and professional trainers to interpret the data. Furthermore, these systems often do not provide feedback on the athlete's form or technique, which is crucial for preventing injuries and improving performance.
The present disclosure provides a novel, cost-effective, and user-friendly system that can monitor and provide feedback on various aspects of athletics training. The system utilizes sensors that can track motion count, motion speed, motion power, and motion form, as well as heart rate, distance traveled, calories burned, and workout frequency. The data collected by these sensors is then analyzed by an app that can provide real-time feedback to the user.
The appended claims may serve as a summary of the invention.
In the drawings:
Each of
Each of
Each of
In the following description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, that the present invention may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to avoid unnecessarily obscuring the present invention.
As used herein, the term “coupled” should be understood to include any direct or indirect connection between two things, such as a wired, mechanical, or other physical connection, a wireless or other non-physical connection, a connection allowing for fluid communication, or any combination thereof. Furthermore, the words “comprising” and any form of comprising, such as “comprise” and “comprises,”, “having” and any form of having, such as “has” and “have,”, “including” and any form of including, such as “includes” and “include,” or “containing” and any form of containing, such as “contains” and “contain,” are inclusive and open-ended and do not exclude additional, unrecited elements or method steps.
As used herein, the term “at least one of” is synonymous with “one or more of.” For example, the phrase “at least one of A, B, and C” means any one of A, B, and C, or any combination of any two or more of A, B, and C. For example, “at least one of A, B, and C” includes one or more of A alone; one or more of B alone; one or more of C alone; or one or more of A and one or more of B; or one or more of A and one or more of C; or one or more of B and one or more of C; or one or more of all of A, B, and C. Similarly, as used herein, the term “at least two of” is synonymous with “two or more of.” For example, the phrase “at least two of D, E, and F” means any combination of any two or more of D, E, and F. For example, “at least two of D, E, and F” includes one or more of D and one or more of E; or one or more of D and one or more of F; or one or more of E and one or more of F; or one or more of all of D, E, and F.
The text of this disclosure, in combination with the drawing figures, is intended to state in prose the algorithms that are necessary to program the computer to implement the claimed inventions, at the same level of detail that is used by people of skill in the arts to which this disclosure pertains to communicate with one another concerning functions to be programmed, inputs, transformations, outputs and other aspects of programming. That is, the level of detail set forth in this disclosure is the same level of detail that persons of skill in the art normally use to communicate with one another to express algorithms to be programmed or the structure and function of programs to implement the inventions claimed herein.
Embodiments are described in sections below according to the following outline:
1. General Overview
2. Structural & Functional Overview
3. Implementation Example-Hardware Overview
A set of one or more sensors may interface with a first computer to detect a particular motion and/or health data. For example, the set of one or more sensors may detect one or more data values from among motion counts, motion speed, motion power, motion form, heart rate, calories burned, distance traveled, and workout frequency. Data from the set of one or more sensors may be analyzed by the first computer, which may send display instructions to a second computer. Via the display instructions, the second computer may present information on a user interface to help a user improve their performance of the particular motion and to meet health goals.
Certain embodiments of a human motion detection system are described herein as being programmed to facilitate improvement at boxing. In such embodiments, a particular motion of one or more human arms may be detected in boxing motion, punching, boxing training, and/or competitive boxing. The particular motion detected by the set of one or more sensors may be a punch. Some embodiments incorporate multiple punching techniques into the analysis application. For example, an analysis application could analyze a jab, a cross, a hook, and/or an uppercut. Thus, the user may select a specific punching form to practice, or the analysis engine may judge the user's form based on whichever punching technique most closely resembles the performed motion. However, the techniques disclosed herein are not limited to boxing. Instead, the techniques disclosed herein apply to any domain of athletic motion performance. For example, and without limitation, certain alternative embodiments could be configured to facilitate improvement at one or more motions from among pullups, squats, rowing, yoga, judo, and weightlifting.
In an embodiment, a boxing analysis application can be programmed to analyze punch motions and health data, and corresponding display instructions can be generated to display information on a user interface. This embodiment is primarily described in connection with
In an embodiment, a computer-implemented method executed using a first computer includes receiving first motion data, wherein the first motion data is detected via a first set of one or more sensors. The computer-implemented method further includes initializing one or more motion variables associated with a particular motion, the one or more motion variables including one or more of repetition count, speed, power, and form. The computer-implemented method further includes analyzing the first motion data to identify one or more instances of the particular motion and to determine one or more motion values corresponding to the one or more motion variables. The computer-implemented method further includes generating and transmitting display instructions which, when rendered using a second computer, cause the second computer to display at least one of the one or more motion values.
In certain embodiments, the particular motion of the computer-implemented method may be a boxing punch, and the motion variables may consist of one or more of punch count, punch speed, punch power, and punch form.
In certain embodiments, the one or more motion values of the computer-implemented method may include at least one power value, and the at least one power value may be determined by analyzing the force of acceleration of at least one motion.
In certain embodiments, the computer-implemented method may further include storing two or more particular motions in non-transitory computer-readable media of the first computer, receiving, via a user interface, a selection of at least one motion from among the two or more particular motions, analyzing one or more differences between the one or more instances of the particular motion and the at least one motion, and outputting a form value based at least in part on the analysis of the one or more differences.
In certain embodiments, the computer-implemented method may further include generating and transmitting display instructions which, when rendered using the second computer, cause the second computer to display advice for improving a form of the at least one motion. The advice may be determined based at least in part on the analysis of the one or more differences.
In certain embodiments, the computer-implemented method may further include receiving second motion data from a second set of one or more sensors and initializing the particular motion based at least in part on the second motion data.
In certain embodiments, the computer-implemented method may further include, using the first computer, receiving health data, wherein the health data is detected via the first set of one or more sensors. In certain embodiments, the computer-implemented method may further include initializing one or more health variables, the one or more health variables including one or more of heart rate, calories burned, distance travelled, and workout frequency. In certain embodiments, the computer-implemented method may further include analyzing the health data to determine one or more health values corresponding to the one or more health variables. In certain embodiments, the computer-implemented method may further include generating and transmitting display instructions which, when rendered using the second computer, cause the second computer to display at least one of the one or more health values.
In certain embodiments, the computer-implemented method may further include determining the distance travelled as a first distance that a body travels, or a second distance that a first portion of the body travels while a second portion of the body is at rest.
Each of the above computer-implemented methods may be practiced using one or more non-transitory computer-readable storage media storing one or more sequences of instructions which, when executed using one or more processors, cause the one or more processors to execute the above computer-implemented methods.
In another embodiment, a computer-implemented method using a first computer includes storing two or more particular motions in non-transitory computer readable media. The computer-implemented method further includes determining a particular motion, wherein the particular motion is determined by receiving, via a user interface, a selection of the particular motion; or determining which of the two or more particular motions is most similar to a first motion. The computer-implemented method further includes receiving sensor data, the sensor data including health data and first punch data, wherein the health data and the first punch data are detected via a first set of one or more sensors. The computer-implemented method further includes initializing four or more punch variables corresponding to a particular motion, the four or more punch variables including punch count, punch speed, punch power, and punch form. The computer-implemented method further includes initializing four or more health variables, the four or more health variables including heart rate, calories burned, distance travelled, and workout frequency. The computer-implemented method further includes analyzing the first punch data to identify one or more instances of the particular motion and to determine one or more punch values corresponding to the one or more punch variables, the one or more punch values including a punch form value, wherein the punch form value is determined by analyzing one or more differences between the first motion and the particular motion. The computer-implemented method further includes analyzing the health data to determine one or more health values corresponding to the one or more health variables. The computer-implemented method further includes generating and transmitting display instructions which, when rendered using the second computer, cause the second computer to display at least one of the one or more punch values and at least one of the one or more health values.
In certain embodiments, the one or more punch values of the computer-implemented method may include at least one power value, and the at least one power value may be determined by analyzing the force of acceleration of at least one motion.
In certain embodiments, the computer-implemented method further includes generating and transmitting display instructions which, when rendered using the second computer, may cause the second computer to display advice for improving a form of the particular motion. The advice may be determined based at least in part on the analysis of the one or more differences.
In certain embodiments, the computer-implemented method further includes receiving second motion data from a second set of one or more sensors and initializing the particular motion based at least in part on the second motion data.
In
First computer 124 may include a database 126 for storing data and an analysis application 128 for analyzing the data and generating display instructions 130. The display instructions 130 may be stored in one or more non-transitory computer-readable media and may be sent to one or more of a user computer 132 and an administrator (“admin”) computer 134. User computer 132 and admin computer 134 may be referred to individually or in combination as “second computer.” When executed by the second computer, the display instructions 130 may display information via a user interface.
The second computer may send data to the database 126 of the first computer 124. For example, the second computer may send data corresponding to a user's motion to database 126; database 126 may analyze the user's motion and compare it to a particular motion.
In certain embodiments, the user computer and the admin computer may communicate with one another. For example, in one illustrative embodiment, a user or a student may send exercise data to an individual admin such as a coach, and the admin may send advice back to the user. In another illustrative embodiment, the user and the admin may interface through video an integrated video call in real-time. First computer 124, user computer 132, and admin computer 134 may be any suitable combination of computers; for example, one or more servers, personal computers, smart devices, and/or smartphones may communicate via one or more networks.
The present disclosure describes the analysis of motion data 104 and health data 106 by the analysis application 128 of the first computer 128. Motion data 104 and health data 106 are not limited to the examples of repetition count 108, speed 112, power 110, form 114, heart rate 116, workout frequency 120, distance 118, and calories 122 of
In certain embodiments, repetition count may be measured by sensor 602 and analyzed by first computer 124. As a particular motion is performed by a user, sensor 602 may send count data 108 to the first computer 124, where the count data 108 may be stored in database 126. Count data 126 may then be analyzed by analysis application 128. In certain embodiments, a first baseline measurement of count data 108 may be determined from a user's first measured workout session, which is the first workout session applicable to the particular motion. Based on the first baseline measurement of count data 108, the first computer 124 may perform one or more functions to assist the user in improving its repetition count in future exercises. For example, analysis application 128 may set goals for the user based on the first baseline measurement of count data 108. As another example, analysis application 128 may provide one or more training tips. As yet another example, analysis application 128 may prepare one or more workouts having transferable exercises to improve the repetition count of the particular motion. In each of these examples, analysis application 128 may generate corresponding display instructions 130 which may be transmitted to a second computer.
In the context of boxing, repetition count is important because fatigue is a common deciding factor in a boxing competition. Thus, in the context of boxing, analysis application 128 may be optimized to improve user endurance.
In certain embodiments, motion speed may be measured by sensor 602 and analyzed by first computer 124. In certain embodiments, database 126 may store one or more of a minimum speed, maximum speed, average speed, and progress speed, which is the difference between the minimum speed and maximum speed of the user. In certain embodiments, progress speed may refer to a session progress speed as the difference between the session maximum speed and session minimum speed. In certain other embodiments, progress speed may refer to lifetime progress speed as the difference between the session maximum speed and session minimum speed.
In the context of boxing, speed is important because different punches should be thrown at different speeds. Varying punch speed is advantageous because it allows the boxer to conserve energy and prevent the opponent from predicting the timing of the boxer's punches. Accordingly, the information presented herein with respect to the presentation of speed data may be particularly helpful in the context of boxing; such information may allow a user to be cognizant of punch speed practice to a degree of precision that would be impossible without sensing equipment.
In certain embodiments, motion power may be measured by sensor 602 and analyzed by first computer 124. In certain embodiments, database 126 may store one or more of minimum power, maximum power, average power, threshold power, and power motion count. In certain embodiments, a count of certain motions meeting or exceeding a given power level (“power motions”) may be stored in database 126. A motion may qualify as a power motion if it is within a percentage distance—for example, ten percent—of the user's maximum power. Threshold power may be calculated as the average power of a user's power motions. Power motion count may be the total number of power motions performed. In certain embodiments, a power value may be determined by analyzing the force of acceleration of at least one motion.
In the context of boxing, punch power is important for reasons similar to those applicable to punch speed; that is, varying punch power can help to conserve energy and can prevent the opponent from defending properly.
In certain embodiments, motion form may be measured by sensor 602 and analyzed by first computer 124. In certain embodiments, analysis application 128 may generate a form value corresponding to the difference between the particular motion and the user's attempt at imitating the particular motion. At least one gyroscope may be included within sensor 602. The at least one gyroscope may detect an angle associated with a motion; for example, for certain punching motions, the at least one gyroscope may collect data associated with the angle of a user's elbow with respect to the user's rib cage to determine the user's form value. In certain embodiments, machine learning algorithm may be trained on form data 114 to determine the correct form for a particular motion.
Database 126 may store one or more of average form, perfect count, below average count, and progress. In an embodiment, the average form comprises the average numerical degree of similarity between the user's motion and the particular motion; the perfect count comprises the number of motions in the current session that are within a preset degree of similarity to the particular motion; the below-average count comprises the number of motions in the current session that fall below the average form; and the progress comprises the difference between average form during a user's first session and average form during a user's current session.
Motion form is important in most athletic endeavors, but it is especially important in the context of boxing. The proper form allows a boxer to improve speed, power, endurance, and safety. By automatically providing measurements of form, boxers may be more mindful of punching form and may thereby improve their punching form more quickly.
Turning now to health data 106, in certain embodiments, heart rate may be measured by sensor 602 and analyzed by first computer 124. In certain embodiments, a user may input health data into a user computer 132. For example, in an embodiment, analysis application 128 is programmed to prompt for and receive input specifying values for the user's age, weight, and/or height. User computer 132 may transmit the input health data to database 126. Based on the input health data, analysis application 128 may determine one or more target heart rate ranges. For example, an analysis application may calculate and store one or more of a target resting heart rate, target lower-intensity heart rate, target temperate heart rate, target aerobic heart rate, minimum heart rate, and maximum heart rate. In certain embodiments, analysis application 128 may generate display instructions 130 which, when executed by a second computer, may allow a user to see a current heart rate and a target heart rate range based on the current activity. In certain embodiments, database 126 may store one or more values from among a minimum heart rate, maximum heart rate, average heart rate, and progress heart rate. In an embodiment, the progress heart rate is the difference between the minimum and maximum heart rates of the user over a lifetime. Furthermore, the first computer 124 may continuously monitor heart rate data 116 and alert a user if the heart rate exceeds the calculated maximum heart rate or falls below the calculated minimum heart rate.
In certain embodiments, health data 106 may include two distinct forms of heart rate data 116: resting heart rate data and maximum heart rate data. Resting heart rate data may comprise one or more values associated with a user's heart rate when the user is not exercising and has fully recovered from past exercise. As a user's fitness improves over time, resting heart rate may decrease. In certain embodiments, a progress heart rate showing the difference between a user's initial resting heart rate and best and/or most recent resting heart rate may be displayed to the user. Maximum heart rate data may comprise one or more values associated with a user's heart rate at its highest point during exercise. A user's heart rate may decrease from a session maximum heart rate to a resting heart rate over a period of time (“heart rate settle time”). A user's heart rate settle time may decrease as the user becomes more fit; that is, the user's heart rate may return to its resting level more quickly. In certain embodiments, a progress value showing the difference between a user's initial heart rate settle time and the user's best and/or most recent heart rate settle time may be displayed to the user.
In certain embodiments, calories burned may be measured by sensor 602 and analyzed by first computer 124. In certain embodiments, database 126 may store the number of calories burned by the user over a given time period; this total may be exercise-specific or the total may represent the total calories burned across all exercises. For example, when the total is exercise-specific, only the calories burned in boxing or only the calories burned biking may be included. In certain embodiments, the database 126 may store multiple calories-burned totals across multiple exercises alongside a cumulative total. As used herein, the term “calories” refers to kilocalories, though any measure of expended human energy may be used without departing from the scope of the present disclosure.
In certain embodiments, the distance may be measured by sensor 602 and analyzed by first computer 124. In certain embodiments, the distance may be the distance that the first part of the body travels to perform a particular motion while the second part of the body is at rest; in certain other embodiments, the distance may be the total distance moved by the entire body, such as miles run or walked. As used herein, the term “at rest” means “motionless or nearly motionless.” In certain rowing embodiments, database 126 may store the distance rowed. In certain climbing embodiments, database 126 may store the distance climbed. In biking embodiments, database 126 may store the distance biked. In certain functional training and/or boxing embodiments, database 126 may store the number of steps taken. Database 126 may also store a user's total distance progress over time.
In certain embodiments, workout consistency and/or workout frequency may be measured by sensor 602 and analyzed by first computer 124. Database 126 may store one or more of the total amount of time over which exercises take place. The time recorded in database 126 may be specific to individual exercises or the time recorded in database 126 may be a sum across all exercises performed. For example, a time specific to individual exercises could be a number of minutes for rowing or a number of minutes for boxing. In certain embodiments, both exercise-specific time measurements and total time measurements may be recorded in database 126. The recorded time may be specific to a single session or may be cumulative across multiple sessions. Furthermore, database 126 may store an updating value based on the amount of time that has passed since the last exercise has been performed. In certain embodiments, the updating value may be exercise-specific; in certain other embodiments, the updating value may be exercise-agnostic. In certain embodiments, multiple updating values may be stored. For example, one value may exist for the amount of time since the last boxing exercise, one value may exist for the amount of time since the last rowing exercise, and one value may exist for the amount of time since the last exercise of any kind. First computer 124 may generate and transmit display instructions 130 to a second computer to display an alert on a second computer if an updating value exceeds a particular value. For example, if an individual were to go a week without exercising or a month without practicing boxing, an alert may be generated.
For example, in an embodiment, in step 202, motion data 104 may be received from the sensor 602 and/or other sensors from among a first set of one or more sensors. One or more motion variables corresponding to a particular motion may be initialized in step 204. The motion data 104 may be analyzed to identify one or more instances of the particular motion and to determine one or more motion values corresponding to one or more motion variables in step 206. Display instructions 130 may be generated and transmitted to display at least one of the one or more motion values in step 208.
As discussed above, in certain embodiments, the particular motion may be a punch. In such embodiments, motion data 104 may comprise one or more values from among a punch count, punch speed, punch power, and punch form.
In certain embodiments, multiple particular motions may be stored in database 126. In such embodiments, a user may select one or more particular motions for the analysis application 128 to detect. In certain embodiments, analysis application 128 may determine a form value by first determining which particular motion most closely resembles the user's motion and by second calculating a difference between the particular motion and the user's motion. A particular motion may be initialized by creating and storing digital data representing the particular motion.
In certain embodiments, an admin or a coach may use a set of one or more sensors 602 to detect the admin's motion and to save corresponding motion data 104 to database 126. Subsequently, the admin's motion data may be used as the particular motion against which a user's motion is judged. For example, an admin may use a set of one or more sensors 602 to detect a left jab. The form data 114 corresponding to the admin's jab may then serve as the particular motion by which a user's subsequently attempted jab is measured.
Some embodiments of the present disclosure are directed to remote workouts; that is, certain embodiments involve a user participating in a workout alongside one or more other users and/or one or more admins while being located geographically remote from the one or more other users and/or the one or more admins. In some such embodiments, the second computer may be a smart phone. Furthermore, in such embodiments, each user may have an independent second computer, and the total number of second computers involved in a workout may be equal to the number of users plus the number of admins involved in the workout. Additionally, in embodiments having multiple particular motions, a user or an admin may select which exercise is being performed via a user interface before performing the exercise.
Certain other embodiments of the present disclosure are directed to non-remote workouts. A non-remote workout involves a user participating in a workout alongside at least one other user and/or at least one admin located within line of sight of the user. One example of a non-remote workout is an in-gym workout. In some such embodiments, there may be more users than second computers. For example, a single second computer may display information on a screen that is visible to several users. One example display may include a leaderboard of the users' performance in the current workout. In embodiments having multiple particular motions, the first computer 124 may determine which particular motion a user is performing by determining which stored particular motion form data is most similar to the user's form data. Certain embodiments may include dedicated workout equipment-for example, certain embodiments may include one or more of a treadmill, a rowing machine, a stationary bicycle, and an elliptical machine. This list is purely illustrative and non-limiting; any appropriate workout equipment may be used without departing from the scope of the present disclosure. In such embodiments, the workout equipment may include one or more sensors, and the one or more sensors may communicate workout data to the first computer 124. The workout data may be used to inform one or more variables; for example, and without limitation, a stationary bicycle may communicate distance data 118 to the first computer 124. In such embodiments, at least a portion of the user's body may be at rest; however, the stationary bicycle may estimate a distance that the user would have traveled if the machine were an ordinary bicycle. The same principle of estimating data may be applied to other similar workout machines.
Each of
The home screen graphical user interface 302 of
The workout class data graphical user interface 312 of
While
In certain embodiments, one or more graphical user interfaces may include a leaderboard. The leaderboard may include one or more values corresponding to users' performance and may be sorted in descending order from best performance to worst performance. Users and admins may add other users and admins as friends, and friend data may be stored in database 126. Thus, a leaderboard may include all users stored in database 126 or may include only friends of the user or admin. The values displayed on the leaderboard may correspond to any suitable values stored in the database. For example, and without limitation, any one or more of motion count 108, motion speed 112, motion power 110, motion form 114, heart rate 116, workout frequency 120, distance 118, and calories 122 may be stored and presented in a leaderboard. Multiple leaderboards corresponding to multiple values may be used. Leaderboards may be exercise-dependent or may operate across all exercises; for example, a first leaderboard may only apply to left jabs, a second leaderboard may apply to boxing exercises generally, and a third leaderboard may apply to all exercises.
In certain embodiments, first computer 124 may generate a workout performance value based on motion data 104 and health data 106. For example, first computer may use one or more motion values and/or one or more health values to calculate the workout performance value. One example of such an embodiment may include the eight values depicted in
In certain embodiments, one or more graphical user interfaces may include the option to transact with other users and/or admins. For example, a user may be able to purchase monthly memberships, personal training packages, group classes, or private sessions for in-person or online training.
In certain embodiments, a graphical user interface of a second computer may display a personalized post-workout report. The post-workout report may display one or more values associated with the user's motion data 104 and/or health data 106. Furthermore, the post-workout report may include advice to improve the user's future athletic performance. Recommendations may be based on any value stored in database 126, including, but not limited to, skill level, sleep schedule, recovery needs, workout history, body profile, motion count 108, motion speed 112, motion power 110, motion form 114, heart rate 116, distance 118, calories 122, and workout frequency 120.
In certain embodiments including a camera, a live video of a user and/or an admin may be displayed on a screen. In such embodiments, augmented reality feedback may be employed wherein one or more graphical user interface elements are displayed over the video to instruct the user.
Each of
In the admin dashboard graphical user interface 402 of
The calendar graphical user interface 406 of
The workout session input graphical user interface 408 of
The video call and exercise data user interface 410 of
The motion combination user interface 412 of
In certain embodiments, an admin graphical user interface may be operable to allow an admin to create one or more classes for which users may sign up. The admin may be able to send messages to one or more users to invite them to the classes or to provide workout advice.
While each of
Each of
In certain embodiments, sensor 602 may be enclosed within wristband 600. In certain embodiments, sensor 602 may be coupled to a surface of wristband 600. Wristband 600 may comprise any suitable combination of one or more materials, including any one or more of synthetic fibers, including any one or more of nylon/polyamide, polyester, acrylic, elastane, fleece, and polyvinyl chloride (PVC), wool, cellulosic, cotton, silk, leather, bast, linen, worsted, bamboo, hemp, denim, rayon, modal, Tencel lyocell, cashmere, fur, angora, or any other material(s) suitable for the production of clothing.
According to one embodiment, the techniques described herein are implemented by at least one computing device. The techniques may be implemented in whole or in part using a combination of at least one server computer and/or other computing devices that are coupled using a network, such as a packet data network. The computing devices may be hard-wired to perform the techniques or may include digital electronic devices such as at least one application-specific integrated circuit (ASIC) or field programmable gate array (FPGA) that is persistently programmed to perform the techniques or may include at least one general purpose hardware processor programmed to perform the techniques pursuant to program instructions in firmware, memory, other storage, or a combination. Such computing devices may also combine custom hard-wired logic, ASICs, or FPGAs with custom programming to accomplish the described techniques. The computing devices may be server computers, workstations, personal computers, portable computer systems, handheld devices, mobile computing devices, wearable devices, body mounted or implantable devices, smartphones, smart appliances, internetworking devices, autonomous or semi-autonomous devices such as robots or unmanned ground or aerial vehicles, any other electronic device that incorporates hard-wired and/or program logic to implement the described techniques, one or more virtual computing machines or instances in a data center, and/or a network of server computers and/or personal computers.
Computer system 500 includes an input/output (I/O) subsystem 502 which may include a bus and/or other communication mechanism(s) for communicating information and/or instructions between the components of the computer system 500 over electronic signal paths. The I/O subsystem 502 may include an I/O controller, a memory controller, and at least one I/O port. The electronic signal paths are represented schematically in the drawings, for example as lines, unidirectional arrows, or bidirectional arrows.
At least one hardware processor 504 is coupled to I/O subsystem 502 for processing information and instructions. Hardware processor 504 may include, for example, a general-purpose microprocessor or microcontroller and/or a special-purpose microprocessor such as an embedded system or a graphics processing unit (GPU), or a digital signal processor or ARM processor. Processor 504 may comprise an integrated arithmetic logic unit (ALU) or may be coupled to a separate ALU.
Computer system 500 includes one or more units of memory 506, such as a main memory, which is coupled to I/O subsystem 502 for electronically digitally storing data and instructions to be executed by processor 504. Memory 506 may include volatile memory such as various forms of random-access memory (RAM) or other dynamic storage devices. Memory 506 also may be used for storing temporary variables or other intermediate information during the execution of instructions to be executed by processor 504. Such instructions, when stored in non-transitory computer-readable storage media accessible to processor 504, can render computer system 500 into a special-purpose machine that is customized to perform the operations specified in the instructions.
Computer system 500 further includes non-volatile memory such as read-only memory (ROM) 508 or other static storage devices coupled to I/O subsystem 502 for storing information and instructions for processor 504. The ROM 508 may include various forms of programmable ROM (PROM) such as erasable PROM (EPROM) or electrically erasable PROM (EEPROM). A unit of persistent storage 510 may include various forms of non-volatile RAM (NVRAM), such as FLASH memory, solid-state storage, magnetic disk, or optical disks such as CD-ROM or DVD-ROM and may be coupled to I/O subsystem 502 for storing information and instructions. Storage 510 is an example of a non-transitory computer-readable medium that may be used to store instructions and data which when executed by the processor 504 cause performing computer-implemented methods to execute the techniques herein.
The instructions in memory 506, ROM 508, or storage 510 may comprise one or more sets of instructions that are organized as modules, methods, objects, functions, routines, or calls. The instructions may be organized as one or more computer programs, operating system services, or application programs including mobile apps. The instructions may comprise an operating system and/or system software; one or more libraries to support multimedia, programming or other functions; data protocol instructions or stacks to implement TCP/IP, HTTPS, or other communication protocols; file format processing instructions to parse or render files coded using HTML, XML, JPEG, MPEG, or PNG; user interface instructions to render or interpret commands for a graphical user interface (GUI), command-line interface, or text user interface; application software such as an office suite, internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications. The instructions may implement a web server, web application server, or web client. The instructions may be organized as a presentation layer, application layer, and data storage layer such as a relational database system using a structured query language (SQL) or no SQL, an object store, a graph database, a flat file system, or other data storage.
Computer system 500 may be coupled via I/O subsystem 502 to at least one output device 512. In one embodiment, output device 512 is a digital computer display. Examples of a display that may be used in various embodiments include a touchscreen display or a light-emitting diode (LED) display or a liquid crystal display (LCD), or an e-paper display. Computer system 500 may include other type(s) of output devices 512, alternatively or in addition to a display device. Examples of other output devices 512 include printers, ticket printers, plotters, projectors, sound cards or video cards, speakers, buzzers or piezoelectric devices or other audible devices, lamps or LED or LCD indicators, haptic devices, actuators, or servos.
At least one input device 514 is coupled to I/O subsystem 502 for communicating signals, data, command selections, gestures, or motions to processor 504. Examples of input devices 514 include touch screens, microphones, still and video digital cameras, alphanumeric and other keys, keypads, keyboards, graphics tablets, image scanners, joysticks, clocks, switches, buttons, dials, slides, and/or various types of sensor 602 such as force sensors, motion sensors, heat sensors, accelerometers, gyroscopes, and inertial measurement unit (IMU) sensors and/or various types of transceivers such as wireless, such as cellular or Wi-Fi, radio frequency (RF) or infrared (IR) transceivers and Global Positioning System (GPS) transceivers.
Another type of input device is a control device 516, which may perform cursor control or other automated control functions such as navigation in a graphical interface on a display screen, alternatively or in addition to input functions. Control device 516 may be a touchpad, a mouse, a trackball, or cursor direction keys for communicating direction information and command selections to processor 504 and for controlling cursor movement on display 512. The input device may have at least two degrees of freedom in two axes, a first axis like an X axis and a second axis like a Y axis that allows the device to specify positions in a plane. Another type of input device is a wired, wireless, or optical control device such as a joystick, wand, console, steering wheel, pedal, gearshift mechanism, or another type of control device. An input device 514 may include a combination of multiple different input devices, such as a video camera and a depth sensor.
In another embodiment, computer system 500 may comprise an internet of things (IoT) device in which one or more of the output device 512, input device 514, and control device 516 are omitted. Or, in such an embodiment, the input device 514 may comprise one or more cameras, motion detectors, thermometers, microphones, seismic detectors, other sensors or detectors, measurement devices, or encoders, and the output device 512 may comprise a special-purpose display such as a single-line LED or LCD display, one or more indicators, a display panel, a meter, a valve, a solenoid, an actuator, or a servo.
When computer system 500 is a mobile computing device, input device 514 may comprise a global positioning system (GPS) receiver coupled to a GPS module that is capable of triangulating to a plurality of GPS satellites, determining and generating geo-location or position data such as latitude-longitude values for a geophysical location of the computer system 500. Output device 512 may include hardware, software, firmware, and interfaces for generating position reporting packets, notifications, pulse or heartbeat signals, or other recurring data transmissions that specify a position of the computer system 500, alone or in combination with other application-specific data, directed toward host 524 or server 530.
Computer system 500 may implement the techniques described herein using customized hard-wired logic, at least one ASIC or FPGA, firmware, and/or program instructions or logic which when loaded and used or executed in combination with the computer system causes or programs the computer system to operate as a special-purpose machine. According to one embodiment, the techniques herein are performed by computer system 500 in response to processor 504 executing at least one sequence of at least one instruction contained in main memory 506. Such instructions may be read into main memory 506 from another storage medium, such as storage 510. Execution of the sequences of instructions contained in main memory 506 causes processor 504 to perform the process steps described herein. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions.
The term “storage media” as used herein refers to any non-transitory media that store data and/or instructions that cause a machine to operate in a specific fashion. Such storage media may comprise non-volatile media and/or volatile media. Non-volatile media includes, for example, optical or magnetic disks, such as storage 510. Volatile media includes dynamic memory, such as memory 506. Common forms of storage media include, for example, a hard disk, solid state drive, flash drive, magnetic data storage medium, any optical or physical data storage medium, memory chip, or the like.
Storage media is distinct from but may be used in conjunction with transmission media. Transmission media participates in transferring information between storage media. For example, transmission media includes coaxial cables, copper wire, and fiber optics, including the wires that comprise a bus of I/O subsystem 502. Transmission media can also take the form of acoustic or light waves, such as those generated during radio-wave and infrared data communications.
Various forms of media may be involved in carrying at least one sequence of at least one instruction to processor 504 for execution. For example, the instructions may initially be carried on a magnetic disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a communication link such as a fiber optic or coaxial cable or telephone line using a modem. A modem or router local to computer system 500 can receive the data on the communication link and convert the data to a format that can be read by computer system 500. For instance, a receiver such as a radio frequency antenna or an infrared detector can receive the data carried in a wireless or optical signal and appropriate circuitry can provide the data to I/O subsystem 502 such as place the data on a bus. I/O subsystem 502 carries the data to memory 506, from which processor 504 retrieves and executes the instructions. The instructions received by memory 506 may optionally be stored on storage 510 either before or after execution by processor 504.
Computer system 500 also includes a communication interface 518 coupled to bus 502. Communication interface 518 provides a two-way data communication coupling to network link(s) 520 that are directly or indirectly connected to at least one communication networks, such as a network 522 or a public or private cloud on the Internet. For example, communication interface 518 may be an Ethernet networking interface, integrated-services digital network (ISDN) card, cable modem, satellite modem, or a modem to provide a data communication connection to a corresponding type of communications line, for example, an Ethernet cable or a metal cable of any kind or a fiber-optic line or a telephone line. Network 522 broadly represents a local area network (LAN), wide-area network (WAN), campus network, internetwork, or any combination thereof. Communication interface 518 may comprise a LAN card to provide a data communication connection to a compatible LAN or a cellular radiotelephone interface that is wired to send or receive cellular data according to cellular radiotelephone wireless networking standards, or a satellite radio interface that is wired to send or receive digital data according to satellite wireless networking standards. In any such implementation, communication interface 518 sends and receives electrical, electromagnetic, or optical signals over signal paths that carry digital data streams representing various types of information.
Network link 520 typically provides electrical, electromagnetic, or optical data communication directly or through at least one network to other data devices, using, for example, satellite, cellular, Wi-Fi, or BLUETOOTH technology. For example, network link 520 may provide a connection through network 522 to a host computer 524.
Furthermore, network link 520 may provide a connection through network 522 or to other computing devices via internetworking devices and/or computers that are operated by an Internet Service Provider (ISP) 526. ISP 526 provides data communication services through a worldwide packet data communication network represented as internet 528. A server computer 530 may be coupled to internet 528. Server 530 broadly represents any computer, data center, virtual machine, or virtual computing instance with or without a hypervisor, or computer executing a containerized program system such as DOCKER or KUBERNETES. Server 530 may represent an electronic digital service that is implemented using more than one computer or instance and that is accessed and used by transmitting web services requests, uniform resource locator (URL) strings with parameters in HTTPS payloads, API calls, app services calls, or other service calls. Computer system 500 and server 530 may form elements of a distributed computing system that includes other computers, a processing cluster, server farm, or other organization of computers that cooperate to perform tasks or execute applications or services. Server 530 may comprise one or more sets of instructions that are organized as modules, methods, objects, functions, routines, or calls. The instructions may be organized as one or more computer programs, operating system services, or application programs including mobile apps. The instructions may comprise an operating system and/or system software; one or more libraries to support multimedia, programming, or other functions; data protocol instructions or stacks to implement TCP/IP, HTTPS, or other communication protocols; file format processing instructions to parse or render files coded using HTML, XML, JPEG, MPEG, or PNG; user interface instructions to render or interpret commands for a graphical user interface (GUI), command-line interface, or text user interface; application software such as an office suite, internet access applications, design and manufacturing applications, graphics applications, audio applications, software engineering applications, educational applications, games, or miscellaneous applications. Server 530 may comprise a web application server that hosts a presentation layer, application layer, and data storage layer such as a relational database system using a structured query language (SQL) or no SQL, an object store, a graph database, a flat file system, or other data storage.
Computer system 500 can send messages and receive data and instructions, including program code, through the network(s), network link 520, and communication interface 518. In the Internet example, server 530 might transmit a requested code for an application program through Internet 528, ISP 526, local network 522, and communication interface 518. The received code may be executed by processor 504 as it is received, and/or stored in storage 510, or other non-volatile storage for later execution.
The execution of instructions as described in this section may implement a process in the form of an instance of a computer program that is being executed and consisting of program code and its current activity. Depending on the operating system (OS), a process may be made up of multiple threads of execution that execute instructions concurrently. In this context, a computer program is a passive collection of instructions, while a process may be the actual execution of those instructions. Several processes may be associated with the same program; for example, opening up several instances of the same program often means more than one process is being executed. Multitasking may be implemented to allow multiple processes to share processor 504. While each processor 504 or core of the processor executes a single task at a time, computer system 500 may be programmed to implement multitasking to allow each processor to switch between tasks that are being executed without having to wait for each task to finish. In an embodiment, switches may be performed when tasks perform input/output operations when a task indicates that it can be switched, or on hardware interrupts. Time-sharing may be implemented to allow fast response for interactive user applications by rapidly performing context switches to provide the appearance of concurrent execution of multiple processes simultaneously. In an embodiment, for security and reliability, an operating system may prevent direct communication between independent processes, providing strictly mediated and controlled inter-process communication functionality.
In the foregoing specification, embodiments of the invention have been described with reference to numerous specific details that may vary from implementation to implementation. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense. The sole and exclusive indicator of the scope of the invention, and what is intended by the applicants to be the scope of the invention, is the literal and equivalent scope of the set of claims that issue from this application, in the specific form in which such claims issue, including any subsequent correction.
The present application claims priority to and the benefit of U.S. Provisional Application No. 63/495,215, filed on Apr. 10, 2023, the entire content of which is being incorporated herein by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
63495215 | Apr 2023 | US |