Patent Application
20140330132
Embodiments relate generally to the health field, and more specifically to a new and useful method for measuring detecting physiological characteristics, such as heart rate, of an organism.
The heart rate of an individual can be associated with a wide variety of characteristics of the individual, such as health, fitness, interests, activity level, awareness, mood, engagement, etc. Simple to highly-sophisticated methods for measuring heart rate currently exist, from finding a pulse and counting beats over a period of time to coupling a subject to an EKG machine. However, each of these methods require contact with the individual, the former providing a significant distraction to the individual and the latter requiring expensive equipment.
Thus, there is a need to create a new and useful method for detecting physiological characteristics, such as heart rate, of an organism.
Various embodiments or examples (“examples”) of the invention are disclosed in the following detailed description and the accompanying drawings:
Various embodiments or examples may be implemented in numerous ways, including as a system, a process, an apparatus, a user interface, or a series of program instructions on a computer readable medium such as a computer readable storage medium or a computer network where the program instructions are sent over optical, electronic, or wireless communication links. In general, operations of disclosed processes may be performed in an arbitrary order, unless otherwise provided in the claims.
A detailed description of one or more examples is provided below along with accompanying figures. The detailed description is provided in connection with such examples, but is not limited to any particular example. The scope is limited only by the claims and numerous alternatives, modifications, and equivalents are encompassed. Numerous specific details are set forth in the following description in order to provide a thorough understanding. These details are provided for the purpose of example and the described techniques may be practiced according to the claims without some or all of these specific details. For clarity, technical material that is known in the technical fields related to the examples has not been described in detail to avoid unnecessarily obscuring the description.
Orientation monitor 152 is configured to monitor orientation 112 of the face of the organism, and to detect a change in orientation in which at least one face portion is absent. For example, the organism may turn its head away, thereby removing a cheek portion from image capture device 104. In response, physiological characteristic determinator 150 can compensate for the absence of check portion, for example, by enlarging the surface areas of the face portions, by amplifying or weighting pixel values and/or light component magnitudes differently, or by increasing the resolution in which to process pixel data, just to name a few examples.
Physiological signal extractor 158 is configured to extract one or more signals including physiological information from subsets of light components captured by light capture device 104. For example, each subset of light components can be associated with one or more frequencies. According to some embodiments, physiological signal extractor 158 identifies a first subset of frequencies (e.g., a range of frequencies, including a single frequency) constituting green visible light, a second subset of frequencies constituting red visible light, and a third subset of frequencies constituting blue visible light. Other frequencies and wavelengths are possible, including those outside visible spectrum. As shown, a signal analyzer 159 of physiological signal extractor 158 is configured to analyze the pixel values or other color-related signal values 117a (e.g., green light), 117b (e.g., red light), and 117c (e.g., green light). For example, signal analyzer 159 can identify a time-domain component associated with a change in blood volume associated with the one or more surfaces of the organism. In some embodiments, physiological signal extractor 158 is configured to aggregate or average one or more AC signals from one or more pixels over one or more sets of pixels. Signal analyzer 159 can be configured to extracting a physiological characteristic based on, for example, a time-domain component based on, for example, using Independent Component Analysis (“ICA”) and/or a Fourier Transform.
Physiological data signal generator 160 can be configured to generate a physiological data signal 115 representing one or more physiological characteristics. Examples of such physiological characteristics include a heart rate pulse wave rate, a heart rate variability (“HRV”), and a respiration rate, among others, in a non-invasive manner.
According to some embodiments, physiological characteristic determinator 150 can be coupled to a motion sensor, 104 such as an accelerometer or any other like device, to use motion data from the motion sensor to determine a subset of pixels in a set of pixels based on a predicted distance calculated from the motion data. For example, consider that pixel or group of pixels 171 are being analyzed in association with a face portion. Upon detecting a motion (of either the organism or the image capture device, or both) in which such motion with move face portion out from pixel or group of pixels 171. Surface detector 154 can be configured to, for example, detect motion of a portions of the face in a set of pixels 117c, which affects a subset of pixels 171 including a face portion from the one or more portions of the face. Surface detector 154 predicts a distance in which the face portion moves from the subset of pixels 171 and determines a next subset of pixels 173 in the set of pixels 117c based on the predicted distance. Then, reflected light associated with the next subset of pixels 173 can be used for analysis.
In some embodiments, physiological characteristic determinator 150 can be coupled to a light sensor 107. Signal analyzer 159 can be configured to compensate for a value of light received from the light sensor that indicates a non-conforming amount of light. For example, consider that the light source generating the light is a fluorescent light source that, for instance, provides for less than desirable amount of, for example, green light. Signal analyzer 159 can compensate, for example, by weighting values associated with either the green light (e.g., either higher) or other values associated with other subsets of light components, such as red and blue light (e.g., weight the blue and red light to decrease influence of red and blue light). Other compensation techniques are possible.
In some embodiments, physiological characteristic determinator 150, and a device in which it is disposed, can be in communication (e.g., wired or wirelessly) with a mobile device, such as a mobile phone or computing device. In some cases, such a mobile device, or any networked computing device (not shown) in communication with physiological characteristic determinator 150, can provide at least some of the structures and/or functions of any of the features described herein. As depicted in
For example, physiological characteristic determinator 150 and any of its one or more components, such as an orientation monitor 152, a surface detector 154, a feature filter 156, a physiological signal extractor 158, and a physiological signal generator 160, can be implemented in one or more computing devices (i.e., any video-producing device, such as mobile phone, a wearable computing device, such as UP® or a variant thereof), or any other mobile computing device, such as a wearable device or mobile phone (whether worn or carried), that include one or more processors configured to execute one or more algorithms in memory. Thus, at least some of the elements in
As hardware and/or firmware, the above-described structures and techniques can be implemented using various types of programming or integrated circuit design languages, including hardware description languages, such as any register transfer language (“RTL”) configured to design field-programmable gate arrays (“FPGAs”), application-specific integrated circuits (“ASICs”), multi-chip modules, or any other type of integrated circuit. For example, physiological characteristic determinator 150 and any of its one or more components, such as an orientation monitor 152, a surface detector 154, a feature filter 156, a physiological signal extractor 158, and a physiological signal generator 160, can be implemented in one or more circuits. Thus, at least one of the elements in
According to some embodiments, the term “circuit” can refer, for example, to any system including a number of components through which current flows to perform one or more functions, the components including discrete and complex components. Examples of discrete components include transistors, resistors, capacitors, inductors, diodes, and the like, and examples of complex components include memory, processors, analog circuits, digital circuits, and the like, including field-programmable gate arrays (“FPGAs”), application-specific integrated circuits (“ASICs”). Therefore, a circuit can include a system of electronic components and logic components (e.g., logic configured to execute instructions, such that a group of executable instructions of an algorithm, for example, and, thus, is a component of a circuit). According to some embodiments, the term “module” can refer, for example, to an algorithm or a portion thereof, and/or logic implemented in either hardware circuitry or software, or a combination thereof (i.e., a module can be implemented as a circuit). In some embodiments, algorithms and/or the memory in which the algorithms are stored are “components” of a circuit. Thus, the term “circuit” can also refer, for example, to a system of components, including algorithms. These can be varied and are not limited to the examples or descriptions provided.
The flow 400 functions to determine the HR of the subject through non-contact means, specifically by identifying fluctuations in the amount of blood in a portion of the body of the subject, as captured in a video signal, through component analysis of the video signal and isolation of a frequency peak in a Fourier transform of the video signal. The flow 400 can be implemented as an application or applet executing on an electronic device incorporating a camera, such as a cellular phone, smartphone, tablet, laptop computer, or desktop computer, wherein Blocks of the flow 400 are completed by the electronic device. Blocks of the flow 400 can additionally or alternatively be implemented by a remote server or network in communication with the electronic device. Alternatively, the flow 400 can be implemented as a service that is remotely accessible and that serves to determine the HR of a subject in an uploaded, linked, or live-feed video signal, though the flow 400 can be implemented in any other way. In the foregoing or any other variation, the video signal and pixel data and values generated therefrom are preferably a live feed from the camera in the electronic device, though the video signal can be preexisting, such as a video signal recorded previously with the camera, a video signal sent to the electronic device, or a video signal downloaded from a remote server, network, or website. Furthermore, the flow 400 can also include calculating the heart rate variability (HRV) of the subject and/or calculating the respiratory rate (RR) of the subject, or any other physiological characteristic, such as a pulse wave rate, a Meyer wave, etc.
In the example shown in
The camera preferably operates at a known frame rate, such as fifteen or thirty frames per second, such that a time-domain component is associated with the video signal. The camera also preferably incorporates a plurality of color sensors, including distinct red, blue, and green color sensors, each of which generates a distinct red, blue, and green source signal, respectively. The color source signal from each color sensor is preferably in the form of an image for each frame recorded by the camera. Each color source signal from each frame can thus be fed into a postprocessor implementing other Blocks of the flow 400 to determine the HR, HRV, and/or RR of the subject. In some embodiments, a light capture device can be other than a camera, but can include any type of light (of any wavelength) receiving and/or detecting sensor.
As shown in
Block S410 preferably implements machine vision to identify the face in the video signal. In one variation, Block S410 used edge detection and template matching to isolate the face in the video signal. In another variation, Block S410 implements pattern recognition and machine learning to determine the presence and position of the face in video signal. This variation preferably incorporates supervised machine learning, wherein Block S410 accesses a set of training data that includes template images properly labeled as including or not including a face. A learning procedure can then transform the training data into generalized patterns to create a model that can subsequently be used to identify a face in video signals. However, in this variation, Block S410 can alternatively implement unsupervised learning (e.g., clustering) or semi-supervised learning in which at least some of the training data has not been labeled. In this variation, Block S410 can further implement feature extraction, principle component analysis (PCA), feature selection, or any other suitable technique to prune redundant or irrelevant features from the video signal. However, Block S410 can implement edge detection, gauging, clustering, pattern recognition, template matching, feature extraction, principle component analysis (PCA), feature selection, thresholding, positioning, or color analysis in any other way, or use any other type of machine learning or machine vision to identify the face of the subject in the video signal.
In Block S410, each frame of the video feed, and preferably each frame of each color source signal of the video feed, can be cropped of all image data excluding the face or a specific portion of the face of the subject. By removing all information in the video signal that is irrelevant to the plethysmographic signal, the amount of time required to calculate subject HR can be reduced.
As shown in
The plethysmographic signal that is extracted from the video signal in Block S420 is preferably an aggregate or averaged AC signal from a plurality of pixels associated with a portion of the face of the subject identified in the video signal, such as either or both cheeks or the forehead of the subject. By aggregating or averaging an AC signal from a plurality of pixels, errors and outliers in the plethysmographic signal can be minimized. Furthermore, multiple plethysmographic signals can be extracted in Block S420 for each of various regions of the face, such as each cheek and the forehead of the subject, as shown in
As shown in
As shown in
In one variation of the flow 400, isolation of the peak frequency is limited to the anticipated frequency range that corresponds with an anticipated or possible HR range of the subject. In another variation of the flow 400, the frequency-domain waveform of Block S430 is filtered to remove waveform data outside of the range of 0.65 to 4 Hz. For example, in Block 140, the plethysmographic signal can be fed through a bandpass filter configured to remove or attenuate portions of the plethysmographic signal outside of the predefined frequency range. Generally, by filtering the frequency-domain waveform of Block S430, repeated variations in the video signal, such as color, brightness, or motion, falling outside of the range of anticipated HR values of the subject can be stripped from the plethysmographic signal and/or ignored. For example, alternating current (AC) power systems in the United States operate at approximately 60 Hz, which results in oscillations of AC lighting systems on the order of 60 Hz. Though this oscillation can be captured in the video signal and transformed in Block S430, this oscillation falls outside of the bounds of anticipated or possible HR values of the subject and can thus be filtered out or ignored without negatively impacting the calculated subject HR, at least in some embodiments.
In the variation of the flow 400 in which multiple plethysmographic signals are transformed in Block S430, Block S440 can include isolating the peak frequency in each of the transformed (e.g., frequency-domain) plethysmographic signals. The multiple peak frequencies can then be compared in Block S440, such as by removing outliers and averaging the remaining peak frequencies to calculate the HR of the subject. Particular color source signals can be more efficient or more accurate for estimating subject HR via the flow 400, and the particular transformed plethysmographic signals can be given greater weight when averaged with less accurate plethysmographic signal.
Alternatively, in the variation of the flow 400 in which multiple plethysmographic signals are transformed in Block S430, Block S440 can include combining the multiple transformed plethysmographic signals into a composite transformed plethysmographic signal, wherein a peak frequency is isolated in the composite transformed plethysmographic signal to estimate the HR of the subject. However, Block S440 can function in any other way and implement any other mechanisms.
In a variation of the flow 400 and as shown in
In a variation of the flow 400 and as shown in
As shown in
By enabling a mobile device, such as a smartphone or tablet, to implement the flow 400, the subject can access any of the aforementioned calculations and generate other fitness-based metrics substantially on the fly and without sophisticated equipment. The flow 400, as applied to exercise, is preferably provided through a fitness application (“fitness app”) executing on the mobile device, wherein the app stores subject fitness metrics, plots subject progress, recommends activities or exercise routines, and or provides encouragement to the subject, such as through a digital fitness coach. The fitness app can also incorporate other functions, such as monitoring or receiving inputs pertaining to food consumption or determining subject activity based upon GPS or accelerometer data.
The flow 400 can also be implemented in a crowd control setting. In one example, because HR can correlate with anxiety or anticipation of a future action, an officer or agent can engage a smartphone or other device incorporating a camera to monitor the HRs of various subjects in the crowd and thus anticipate a future action of a particular subject. Generally, the officer or agent can single out a subject in the crowd as a potential threat based upon a HR significantly greater than the HRs of proximal subjects. Alternatively, the officer or agent can adjust crowd control efforts (e.g., deployment of fencing, number of security personnel) based upon elevated HRs of subjects in the crowd or the average HR across a portion or all of the crowd. A microphone or other volume sensor can further corroborate the correlation between HR and anxiety for one or more subjects within the crowd.
According to some embodiments, flow 400 can be implemented by physiological characteristic determinator 150 in the service industry. For example, Block S450 can be configured to determine a mood of the subject. The flow 400 can calculate the HR of an employee of a call center (not shown) as an estimation of frustration with a customer, wherein a HR or correlated frustration level exceeding a threshold level can indicate need for a break or initiate transfer of a call to a supervisor or other employee. Alternatively, the HR of the customer can be monitored (e.g., remotely) to determine the same, such that customer dissatisfaction is limited by ensuring that his experience shifts (e.g., by speaking with a manager) before reaching a critical HR or correlated frustration level. In another alternative, a subject can be asked to look into the camera on his smartphone while on hold with a call center such that subjects with elevated HRs are given priority over subjects with less pending issues, as indicated by HR or correlated frustration level. In this variation, the flow 400 can be augmented by subject voice level, as captured by a microphone, wherein an elevated or rising voice volume reinforces estimated frustration level.
The flow 400 can alternatively be used to guide introductions between two or more subjects based upon changes in HRs thereof. For example the flow 400 can be used to isolate two subjects (i.e., person (“A”) 920 and person (“B”) 922) in a crowd of subjects 910, wherein the two subjects experience elevated HRs when in proximity. This can indicate interest between the two subjects 920 and 922, and the flow 400 can further encourage at least one subject to make an introduction to the other subject. Computing device 930 can capture light reflected from subjects 920 and 922 to determine physiological characteristics using physiological characteristic determinator 150. As shown, subjects 920 and 922 have relatively elevated heart rates. Data from computing device 930 (e.g., a mobile phone device) can be transmitted via networks 942 to a remote computing device 940 that can operate as a social networking platform application.
Further to this example, flow 400 can thus be implemented as an ultra-local dating interface, at least in cooperation with remote computing device 940, wherein potential interest among multiple subjects 920 and 922 is corroborated with physical data, including the HRs of the subjects when in proximity. The flow 400 can also interface with a social or dating network, such as Facebook or Match.com, to ascertain the relationship status, interests, and other relevant information of at least one subject, which can better guide an introduction of the subjects. In this variation, the flow 400 (or a portion thereof) is preferably implemented as a “local dating app” executing on a smartphone, such as devices 950 and/or 952, such that a subject 920 or subject 920 can access the flow 400 without additional equipment beyond that available to the subject.
In another example implementation, the flow 400 is applied to subject testing to indicate subject satisfaction or subject frustration. For example, when a subject 1001 first purchases and downloads an app onto his smartphone, a forward-facing camera integral with the smartphone can capture initial subject reaction to the app, including subject HR, wherein an elevated subject HR indicates frustration or buyer's remorse and a steady subject HR indicates that the app meets subject expectations. The smartphone can also implement facial recognition or other machine vision techniques to capture a smile, frown, furrowed brow, etc. to further corroborate subject emotion following a purchase. In another example, a hardware company can study subject HR during assembly of a product, wherein an elevated HR during product assembly can indicate poor or confusing instructions, missing or mislabeled components, poor packaging, and/or a lower-than expected product quality. The flow 400 can therefore be implemented to qualify and quantify a subject experiences with a hardware or software product, particularly in situations in which a subject is unlikely or unable to provide feedback or in which a subject is typically unable to communicate specific problems or issues with a product.
In another example implementation, the flow 400 is implemented in the marketing and advertising space, wherein subject interest in a product, brand, advertisement, or advertising style is indicated by subject HR 1010 (shown in a computer display) as determined via the flow 400. For example, a camera 1002 integrated into a television or gaming console can capture sentiment or interest of one or more subjects while watching television. In the example shown, computing device 1006 is remote from light capture device 1002, but it does not need by in this or other examples. If the average HR of subjects watching a show on the television escalates during a romantic scene, advertisements and/or commercials presented to the subjects during the show can adjust to include ads for an upcoming romantic comedy, a romantic weekend getaway, or celebratory champagne. Alternatively, if the average HR of one or more subjects 1001 watching a show on the television escalates when food is shown, advertisements presented to the subjects during the show can adjust to include ads for fast-food restaurants or supermarkets. Generally, the flow 400 can be used to select ads more likely to resonate with a subject, wherein subject interest is associated with certain products or experiences based upon elevated subject HR.
In yet another example implementation, the flow 400 can be incorporated into polling services. For example, a public opinion poll for presidential candidates can ask voters to indicate a preferred candidate. A simultaneous elevation in HR of a voter can indicate a level of loyalty to or dislike for a particular candidate, which can provide more powerful information for political polling, such as devotion of a subset of voters to a particular candidate or the divisiveness of a particular candidate. Similarly, HRs of attendees of a debate can be monitored to ascertain topics that most resonate with a portion of demographic of the attendees or the nature of reactions to candidate responses.
In still another example implementation, the flow 400 is similarly employed to gauge public interest in new products, services, technologies, movies, etc. without directly polling the audience. For example, HRs of attendees of the Keynote presentation at an Apple® Worldwide Developers Conference (WWDC), in which a new product is revealed, can be aggregated as a means by which to gauge public interest without directly asking individuals for their opinions of the new product. However, the flow 400 can be applied to any other type of poll and in any other way.
Similarly, a camera 1102 integrated into a laptop computer (or a mobile computing device 1104) can capture subject HR while the subject listens to music, and music selection can be adjusted to maintain the HR of the subject above or below a threshold HR based upon the environment or activity of the subject. Alternatively, rather simply than selecting a genre, artist, album, or song, the subject 1102 can additionally or alternatively select a target HR, wherein songs are selected according to a schedule that maintains the HR of the subject substantially near, above, or below the target HR. Furthermore, when the subject indicates a preference or dislike for a particular song or artist, the HR of the subject can suggest the degree to which the subject likes or dislikes the particular song or artist. For example, physiological characteristic determinator 1120 can generate preference data 1128 for storing in a repository 1121. Consider that subject 1101 has listened to a first song associated song file 1124 (“S1”) and to a second song associated song file 1122 (“S2”). Data 1125 representing a first heart beat and data 1123 representing a second heart beat are associated with song files 1124 and 1122, respectively. In some embodiments, physiological characteristic determinator 1120 determines that subject 1101 is more interested in song 2 rather than song 1 based on, at least heartbeat data 1123 and 1125.
In one example implementation, the flow 400 can be applied to a gaming environment, wherein the HR of a subject playing a game correlates with subject interest in the game. For example, a subject playing poker can experience an elevated HR given an above-average hand or before or after making a sizable bet, and the flow 400 can thus be employed to estimate a future action of the subject during such a game. In another example implementation, the flow 400 can be applied to a gaming console, wherein the HR of the subject correlates with the activity level of the subject while playing a game on the gaming console. Through the flow 400, subject HR can be used adjust game play mechanics to maintain, increase, or assuage game activity. However, the flow 400 can be applied to the field of engagement in any other way.
In another example implementation, an older subject can set up cameras or light capture devices 1210 in a mobile device 1212 (or integrated in the interior of car 1202) at key locations within his car or home, wherein each camera checks the HR of the subject when the subject is within range. In this example implementation, a substantially low, high, or otherwise abnormal HR or RR can automatically alert a doctor or emergency staff of a potential health risk to the subject.
In yet another example implementation, the flow 400 provide safety and health warnings to a subject. For example, a subject engaging in yard work on a hot summer day can arrange a camera strategically within the yard, and the flow 400 can monitor subject HR and provide warnings if significant risk for sunstroke is calculated based upon changes in the HR, RR, perspiration rate, and/or activity level of the subject. The flow 400 can similarly be implemented through a camera integrated into a motor vehicle, wherein a lowered HR and/or RR of a driver of the vehicle can indicate that the driver is drowsy. This estimation can be corroborated by lowered eyelids or squinting, as captured by the camera. The subject can therefore by warned of elevated driving risk. Alternatively, the function of the vehicle can be automatically reduced to ameliorate the likelihood or severity of a pending accident.
In a further example implementation, the flow 400 can be implemented in a video game, such as Wii Tennis or Dance Dance Revolution (DDR), wherein the game can encourage subject activity up to a certain HR (i.e. based upon each individual subject) rather than based upon a preset maximum activity level. However, the flow 400 can be applied to safety in any other way.
Referring back to
HR, HRV, and RR, which can correlate with the health, wellness, and/or fitness of the subject, can thus be tracked over time at 606 and substantially in the background, thus increasing the amount of health-related data captured for a particular subject while decreasing the amount of positive action necessary to capture health-related data on the part of the subject, a medical professional, or other individual. Through the flow 400, health-related information can be recorded substantially automatically during normal, everyday actions already performed by a large subset of the population.
With such large amounts of HR, HRV, and/or RR data for the subject, health risks for the subject can be estimated at 622. In particular, trends in HR, HRV, and/or RR, such as at various times or during or after certain activities, can be determined at 612. In this variation, additional data falling outside of an expected value or trend can trigger warnings or recommendations for the subject. In a first example, if the subject is middle-aged and has a HR that remains substantially low and at the same rate throughout the week, but the subject engages occasionally in strenuous physical activity, the subject can be warned of increased risk of heart attack and encouraged to engage is light physical activity more frequently at 624. In a second example, if the HR of the subject is typically 65 bpm within five minutes of getting out of bed, but on a particular morning the HR of the subject does not reach 65 bpm until thirty minutes after rise, the subject can be warned of the likelihood of pending illness, which can automatically trigger confirmation a doctor visit at 626 or generation a list of foods that can boost the immune system of the subject. Trends can also show progress of the subject, such as improved HR recovery throughout the course of a training or exercise regimen.
In this variation, the flow 400 can also be used to correlate the effect of various inputs on the health, mood, emotion, and/or focus of the subject. In a first example, the subject can engage an app on his smartphone (e.g., The Eatery by Massive Health) to record a meal, snack, or drink. While inputting such data, a camera on the smartphone can capture the HR, HRV, and/or RR of the subject such that the meal, snack, or drink can be associated with measured physiological data. Overtime, this data can correlate certain foods correlate with certain feelings, mental or physical states, energy levels, or workflow at 620. In a second example, the subject can input an activity, such as by “checking in” (e.g., through a Foursquare app on a smartphone) to a location associated with a particular product or service. When shopping, watching a sporting event, drinking at a pub with friends, seeing a movie, or engaging in any other activity, the subject can engage his smartphone for any number of tasks, such as making a phone call or reading an email. When engaged by the user, the smartphone can also capture subject HR and then tag the activity, location, and/or individuals proximal the user with measured physiological data. Trend data at 606 can then be used to make recommendations to the subject, such as a recommendation to avoid a bar or certain individuals because physiological data indicates greater anxiety or stress when proximal the bar or the certain individuals. Alternatively, an elevated HR of the subject while performing a certain activity can indicate engagement in and/or enjoyment of the activity, and the subject can subsequently be encouraged to join friends who are currently performing the activity. Generally, at 610, social alerts can be presented to the subject and can be controlled (and scheduled), at least in part, by the health effect of the activity on the subject.
In another example implementation, the flow 400 can measure the HR of the subject who is a fetus. For example, the microphone integral with a smartphone can be held over a woman's abdomen to record the heart beats of the mother and the child. Simultaneously, the camera of the smartphone can be used to determine the HR of the mother via the flow 400, wherein the HR of the woman can then be removed from the combined mother-fetus heart beats to distinguish heart beats and the HR of the fetus alone. This functionality can be provided through software (e.g., a “baby heart beat app”) operating on a standard smartphone rather than through specialized. Furthermore, a mother can use such an application at any time to capture the heartbeat of the fetus, rather than waiting to visit a hospital. This functionality can be useful in monitoring the health of the fetus, wherein quantitative data pertaining to the fetus can be obtained at any time, thus permitting potential complications to be caught early and reducing risk to the fetus and/or the mother. Fetus HR data can also be cumulative and assembled into trends, such as described above.
Generally, the flow 400 can be used to test for certain heart or health conditions without substantial or specialized equipment. For example, a victim of a recent heart attack can use nothing more than a smartphone with integral camera to check for heart arrhythmia. In another example, the subject can test for risk of cardiac arrest based upon HRV. Recommendations can also be made to the subject, such as based upon trend data, to reduce subject risk of heart attack. However, the flow 400 can be used in any other way to achieve any other desired function.
Further, flow 400 can be applied as a daily routine assistant. Block S450 can be configured to include generating a suggestion to improve the physical, mental, or emotional health of the subject substantially in real time. In one example implementation, the flow 400 is applied to food, exercise, and/or caffeine reminders. For example, if the subject HR has fallen below a threshold, the subject can be encouraged to eat. Based upon trends, past subject data, subject location, subject diet, or subject likes and dislikes, the type or content of a meal can also be suggested to the subject. Also, if the subject HR is trending downward, such as following a meal, a recommendation for coffee can be provided to the subject. A coffee shop can also be suggested, such as based upon proximity to the subject or if a friend is currently at the coffee shop. Furthermore, a certain coffee or other consumable can also be suggested, such as based upon subject diet, subject preferences, or third-party recommendations, such as sourced from Yelp. The flow 400 can thus function to provide suggestions to maintain a energy level and/or a caffeine level of the subject. The flow 400 can also provide “deep breath” reminders. For example, if the subject is composing an email during a period of elevated HR, the subject can be reminded to calm down and return to the email after a period of reflection. For example, strong language in an email can corroborate an estimated need for the subject to break from a task. Any of these recommendations can be provided through pop-up notifications on a smartphone, tablet, computer, or other electronic device, through an alarm, by adjusting a digital calendar, or by any other communication means or through any other device.
In another example implementation, the flow 400 is used to track sleep patterns. For example, a smartphone or tablet placed on a nightstand and pointed at the subject can capture subject HR and RR throughout the night. This data can be used to determine sleep state, such as to wake up the subject at an ideal time (e.g., outside of REM sleep). This data can alternatively be used to diagnose sleep apnea or other sleep disorders. Sleep patterns can also be correlated with other factors, such as HR before bed, stress level throughout the day (as indicated by elevated HR over a long period of time), dietary habits (as indicated through a food app or changes in subject HR or RR at key times throughout the day), subject weight or weight loss, daily activities, or any other factor or physiological metric. Recommendations for the subject can thus be made to improve the health, wellness, and fitness of the subject. For example, if the flow 400 determines that the subject sleeps better, such as with fewer interruptions or less snoring, on days in which the subject engages in light to moderate exercise, the flow 400 can include a suggestion that the subject forego an extended bike ride on the weekend (as noted in a calendar) in exchange for shorter rides during the week. However, any other sleep-associated recommendation can be presented to the subject.
The flow 400 can also be implemented through an electronic device configured to communicate with external sensors to provide daily routine assistance. For example, the electronic device can include a camera and a processor integrated into a bathroom vanity, wherein the HR, HRV, and RR of the subject is captured while the subject brushes his teeth, combs his hair, etc. A bathmat in the bathroom can include a pressure sensor configured to capture at 608 the weight of the subject, which can be transmitted to the electronic device. The weight, hygiene, and other action and physiological factors can thus all be captured in the background while a subject prepares for and/or ends a typical day. However, the flow 400 can function independently or in conjunction with any other method, device, or sensor to assist the subject in a daily routine.
Other applications of Block S450 are possible. For example, the flow 400 can be implemented in other applications, wherein Block S450 determines any other state of the subject. In a one example, the flow 400 can be used to calculate the HR of a dog, cat, or other pet. Animal HR can be correlated with a mood, need, or interest of the animal, and a pet owner can thus implement the flow 400 to further interpret animal communications. In this example, the flow 400 is preferably implemented through a “dog translator app” executing on a smartphone or other common electronic device such that the pet owner can access the HR of the animal without additional equipment. In this example, a user can engage the dog translator app to quantitatively gauge the response of a pet to certain words, such as “walk,” “run,” “hungry,” “thirsty,” “park,” or “car,” wherein a change in pet HR greater than a certain threshold can be indicative of a current desire of the pet. The inner ear, nose, lips, or other substantially hairless portions of the body of the animal can be analyzed to determine the HR of the animal in the event that blood volume fluctuations within the cheeks and forehead of the animal are substantially obscured by hair or fur.
In another example, the flow 400 can be used to determine mood, interest chemistry, etc. of one or more actors in a movie or television show. A user can point an electronic device implementing the flow 400 at a television to obtain an estimate of the HR of the actor(s) displayed therein. This can provide further insight into the character of the actor(s) and allow the user to understand the actor on a new, more personal level. However, the flow 400 can be used in any other way to provide any other functionality.
According to some examples, computing platform 1300 performs specific operations by processor 1304 executing one or more sequences of one or more instructions stored in system memory 1306, and computing platform 1300 can be implemented in a client-server arrangement, peer-to-peer arrangement, or as any mobile computing device, including smart phones and the like. Such instructions or data may be read into system memory 1306 from another computer readable medium, such as storage device 1308. In some examples, hard-wired circuitry may be used in place of or in combination with software instructions for implementation. Instructions may be embedded in software or firmware. The term “computer readable medium” refers to any tangible medium that participates in providing instructions to processor 1304 for execution. Such a medium may take many forms, including but not limited to, non-volatile media and volatile media. Non-volatile media includes, for example, optical or magnetic disks and the like. Volatile media includes dynamic memory, such as system memory 1306.
Common forms of computer readable media includes, for example, floppy disk, flexible disk, hard disk, magnetic tape, any other magnetic medium, CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, RAM, PROM, EPROM, FLASH-EPROM, any other memory chip or cartridge, or any other medium from which a computer can read. Instructions may further be transmitted or received using a transmission medium. The term “transmission medium” may include any tangible or intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine, and includes digital or analog communications signals or other intangible medium to facilitate communication of such instructions. Transmission media includes coaxial cables, copper wire, and fiber optics, including wires that comprise bus 1302 for transmitting a computer data signal.
In some examples, execution of the sequences of instructions may be performed by computing platform 1300. According to some examples, computing platform 1300 can be coupled by communication link 1321 (e.g., a wired network, such as LAN, PSTN, or any wireless network) to any other processor to perform the sequence of instructions in coordination with (or asynchronous to) one another. Computing platform 1300 may transmit and receive messages, data, and instructions, including program code (e.g., application code) through communication link 1321 and communication interface 1313. Received program code may be executed by processor 1304 as it is received, and/or stored in memory 1306 or other non-volatile storage for later execution.
In the example shown, system memory 1306 can include various modules that include executable instructions to implement functionalities described herein. In the example shown, system memory 1306 includes a Physiological Characteristic Determinator 1360 configured to implement the above-identified functionalities. Physiological Characteristic Determinator 1360 can include a surface detector 1362, a feature filter, a physiological signal extractor 1366, and a physiological signal generator 1368, each can be configured to provide one or more functions described herein.
The systems and methods of the preferred embodiment can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions are preferably executed by computer-executable components preferably integrated with a remote hospital, insurance, or health server, with hardware/firmware/software elements of a subject computer or mobile device, or any suitable combination thereof. Other systems and methods of the preferred embodiment can be embodied and/or implemented at least in part as a machine configured to receive a computer-readable medium storing computer-readable instructions. The instructions are preferably executed by computer-executable components preferably integrated by computer-executable components preferably integrated with apparatuses and networks of the type described above. The computer-readable medium can be stored on any suitable computer readable media such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives, floppy drives, or any suitable device. The computer-executable component is preferably a processor but any suitable dedicated hardware device can (alternatively or additionally) execute the instructions.
As a person skilled in the art will recognize from the previous detailed description and from the figures and claims, modifications and changes can be made to the preferred embodiments of the invention without departing from the scope of this invention as defined in the following claims.
This U.S. non-provisional patent application claims the benefit and priority to U.S. Provisional Patent Application No. 61/641,672 filed on May 2, 2012, which is incorporated by reference herein for all purposes.
