The present invention relates generally to the field of pervasive networks, and more particularly relates to the use of pervasive devices for mobile environmental sensing.
The last few decades have seen an exponential increase in the worldwide use of pervasive devices, such as mobile phones, personal digital assistants (PDAs), game players, embedded processors, laptop computers and desktop computers. There continue to be great advances in both technological capability and ubiquity which serve to facilitate increasingly universal communications. Furthermore, these pervasive devices increasingly have integrated capabilities beyond mere telecommunications or computing; for example, such devices nowadays are frequently integrated with internal clocks, GPS systems, and image/video capture systems.
Nonetheless, advances in early detection and notification of environmental conditions have not kept pace with the overall advances in communications technology. Indeed, there have been many well-documented instances of ineffective communication. Some of these have occurred during disasters both natural (for example, the Asian Tsunami and Hurricane Katrina) and man-made (for example, the September 11 terrorist attacks) which resulted in literally thousands of deaths which could likely have been avoided had more timely information been provided both to governmental authorities and to those within the zone of danger. Today's limited early detection systems (for example, seismographs or fire lookouts) are limited not only in density and coverage area, but also lack the ability to collect and collate multiple stimuli in order to obtain a more detailed picture of the surrounding environment. Furthermore, current warning systems tend to focus on advisories communicated via mass media, which frequently are both overbroad (causing people to assume these advisories are irrelevant and thus may be safely ignored) and underbroad (failing to reach the populations most directly endangered who may not be actively monitoring the broadcast media, for example, beachgoers during the Asian Tsunami and office workers on September 11). Notification regarding non-disaster conditions also could benefit from improvement; for example, “macro” weather reports used to predict whether it is a good day for the beach or for boating may not accurately reflect local conditions at a beach or boating region of interest.
It would thus be desirable to overcome the limitations in previous approaches.
Principles of the present invention provide techniques for monitoring environmental conditions. In one aspect, an exemplary method includes the steps of acquiring data pertaining to environmental information using a multiplicity of sensors on a multiplicity of networked pervasive devices, and analyzing the data to determine occurrence of at least one environmental event.
In another aspect, an exemplary networked pervasive device associated with a network includes at least one environmental sensor configured to obtain environmental data, and a communications module coupled to the at least one environmental sensor and configured to cause transmission of a representation of the data via the network in a form for subsequent data mining within the network.
In yet another aspect, an exemplary method of providing a service for monitoring environmental conditions includes the steps of facilitating offering the monitoring service on a subscription basis, facilitating receiving subscriptions to the service from a plurality of subscriber's, facilitating acquisition of data pertaining to environmental information using at least one sensor on at least one networked pervasive device, facilitating analysis of the data to determine occurrence of at least one environmental event, and facilitating notification of at least one of the subscribers of the occurrence of the event.
One or more embodiments of the invention can be implemented in the form of a computer product including a computer usable medium with computer usable program code for performing the method steps indicated. Furthermore, one or more embodiments of the invention can be implemented in the from of an apparatus including a memory and at least one processor that is coupled to the memory and operative to perform exemplary method steps.
These and other features and advantages of the present invention will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
The present invention will be described herein in the context of exemplary methods and apparatus for monitoring environmental conditions. It is to be understood, however, that the techniques of the present invention are not limited to the methods and apparatus shown and described herein. Furthermore, as used herein, “environmental” data should be broadly construed to cover any data that can be sensed and should not be limited to hazard warnings or other specific examples that may be given.
The sensors, which can be embedded into the pervasive computing devices using, for example, Micro-Electro-Mechanical Systems (MEMS) and/or System-on-package (SOP) technology, can include, by way of example and not limitation, images, video streams, temperature, shock, barometric pressure, chemical detection, radioactivity, user biometrics, and the like. Mobile devices can provide access to geographic locations not populated by fixed systems such as highways and outdoor areas, and can provide access to tens of millions of geographic locations; essentially, any location where any person is.
A server can collect this information and creates a view of the environment in both space and time and further analyze the data (for example, by using environmental event algorithm models 106 to determine within statistical certainty that an event is occurring in real time). The system can also combine and fuse data from multiple sensors, integrating information about time, location, temperature, sound, image, heat, infrared, and the like, to sense and understand higher-level features.
For example, on a macroscopic scale, an earthquake in California can be visualized by recording the real time “shock” (accelerometer) sensors in computer devices within a geographic area. On the microscopic scale, chemical sensors could be used to identify a gas leak in a particular location, while also detecting information from pressure sensors to find the wind direction. The two variables can be integrated to predict the direction of gas leakage and notify users, or to visualize the movement of the gas cloud over the environment. Combining information from multiple sensor networks (“data fusion”) enables new applications. Such applications are not limited to disaster sensing and prediction. For example, sensors could be used to determine conditions for recreation, such as determining the local environment at, fox example, a beach. Macroscopic weather forecasts often do not provide sufficient detail to determine if an individual would be comfortable at a beach along the ocean. However, sensing from multiple pervasive devices could provide information on local wind direction, wind temperature, humidity, sun intensity at ground level, beach population density and water temperature, which could be used by a machine implementing an intelligent algorithm to determine if conditions at a beach would meet an individual's comfort level. Weather information for sailing can be generated using the pervasive devices of the number of people that are present at a beach or other region near where one might wish to go sailing. The individual wind and light sensors on the pervasive device of each tourist on the beach (or other region) gives us the wind direction at the location of that particular tourist. By fusing and analyzing the data from the large number of such tourists on the beach, a map of the wind direction and the cloud formation on the sea shore can be generated. This can be matched against the pre-existing information of the ideal wind and/or light conditions for sailing near that particular beach (or other region). Thus, using the multitude of sensors on these devices, a score can be generated that provides information about how conducive the shore is for sailing. This information can be relayed to interested sailors who would not have to travel to the beach (or other region) and identify these conditions. Exemplary types of sensors and their local and global uses are summarized in the table below:
Another exemplary application includes surveillance. For example, many people have cellular phones with image capture. A bag might be abandoned at an airport. Gas sensors might determine dangerous substances in the bag. Heat detectors might locate hidden criminals or terrorists who planted the bag. Data could be gathered passively (for example, gas, heat) or actively (for example, using cell phone cameras). In some embodiments, incentives could be provided for voluntary opt-in to address potential invasion-of-privacy concerns. Other types of surveillance could include a “worried patent” service that could track the location of a child and, for example, the presence of illegal drugs or alcohol, or excessive profanity (for example, via automatic speech recognition). Such approaches could also be adapted to tracking elderly parents, lost pets, and the like
The data from one or more networked pervasive devices 1 . . . N are collected in step 200 and then sent to a central data server in step 202. This may operate using either a “push” mechanism (the devices initiates data transfer to the server) or a “pull” mechanism (the server initiates data transfer from the devices). In other embodiments, data may be sent responsive to a user action. Monitoring can be continuous or intermittent. The server logs the time and location of the pervasive device and whether it is mobile or stationary. This provides the server with temporal and spatial data pertaining to the environmental variables which follow.
In step 206 the server (in other embodiments, analysis can be carried out by one or more of the networked pervasive devices in addition to or in lieu of the central server) analyzes the data within geographic areas labeled X and Y and, based upon environmental models, can determine the probability an event is occurring in real time within the X and/or Y geographic area which is referred to as P (A, X) or P (A, Y). In step 208 the probability is compared to a threshold, which in this case is chosen to be, for exemplary purposes and not limitation, 0.7 or 70%, and if it exceeds that threshold within one of the geographic regions the server may send a notification to, for example, the users of the pervasive devices within the affected region and emergency response units. For example, an earthquake event can be detected by recording the real time shock data of devices within a geographic area or a chemical spill can be detected by changes in chemical composition. Analysis can be performed, for example, using artificial intelligence and/or pattern recognition techniques.
In step 210 the server determines if the event in region X will affect users in a neighboring region Y by calculating the probability that the event in one region would affect the other. For example, upon determining by means of chemical sensors that a gaseous discharge has occurred in region X, the server could analyze wind direction (obtained using wind direction sensors) to see whether region Y will be affected as well. Notification of individuals (including but not limited to users of one or more of the pervasive devices) or authorities (such as emergency responders) in a currently affected region, and/or a region likely to be affected in the future, is conducted in step 212. Past or real-time events can be detected, and future events can be predicted. In a subscription model, subscribers can be informed of detected or predicted events.
Turning now to
In one aspect device 300 comprises a cellular telephone. Further, receive circuitry can be provided, coupled to the antenna 302 and the communications module 306. In the example of
Turning now to
It should be understood that the real time identification of events can be used in several ways. Systems can use this information to take autonomic actions; for example, to protect data and/or power down building systems. The data could be used to send instructions to users to take specific appropriate actions, such as to don gas masks and/or evacuate buildings. Emergency systems can be controlled remotely to assist in mitigation or resources in neighboring locations can be notified to take preemptive action.
Many different types of data can be sensed; for example, location (including elevation) via a global positioning system (GPS); camera images, heat, radiation, infra-red radiation (for example, to detect location of humans in environments that are supposed to be empty), soot, abnormal numbers of phone calls, pressure or other indications of thunderstorms, tornadoes, and the like, temperature, and the like. As noted, artificial intelligence or pattern recognition can be employed to determine what is normal and perform appropriate classification and detection. Protection may be afforded against terrorist acts, severe weather, fires, accidents, or even traffic conditions and the like. Data of many disparate types and from many disparate sources can be readily fused, for example, wind conditions, wave heights, air temperature, humidity, traffic or other road or transportation conditions, and in one or more embodiments, can be compared to a baseline. Other possible applications include battlefield management, integration with stolen car locators such as LOJACK® transponder units (LOJACK® is a registered trademark of Lojack Corporation, Westwood, Mass., USA, and the like). Disparate data may be fused and yield a simple, high-level recommendation. For example, suppose there is a problem at a nuclear power plant. Wind and radiation sensors might determine and/or predict the area to be affected by a radiation plume. Traffic sensors might determine a best, least congested route, in the opposite direction from the plume, and yield a simple recommendation, such as “Drive north on State Route 9” or simply “Run north.”
In another aspect, a service, such as a subscription service, can be offered. With reference to
A variety of techniques, utilizing dedicated hardware, general purpose processors, firmware, software, or a combination of the foregoing may be employed to implement the present invention. One or more embodiments of the invention can be implemented in the form of a computer product including a computer usable medium with computer usable program code for performing the method steps indicated. Furthermore, one or more embodiments of the invention can be implemented in the form of an apparatus including a memory and at least one procession that is coupled to the memory and operative to perform exemplary method steps.
At present, it is believed that one or more embodiments will make substantial use of software running on a general purpose computer or workstation. With reference to
Accordingly, computer software including instructions or code for performing the methodologies of the invention, as described herein, may be stored in one or more of the associated memory devices (e.g., ROM, fixed or removable memory) and, when ready to be utilized, loaded in part or in whole (e.g., into RAM) and executed by a CPU. Such software could include, but is not limited to, firmware, resident software, microcode, and the like.
Furthermore, the invention can take the form of a computer program product accessible from a computer-usable or computer-readable medium (e.g., media 618) providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer usable or computer readable medium can be any apparatus for use by or in connection with the instruction execution system, apparatus, or device.
The medium can be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system (or apparatus or device) or a propagation medium. Examples of a computer-readable medium include a semiconductor or solid-state memory (e.g. memory 604), magnetic tape, a removable computer diskette (e.g. media 618), a random access memory (RAM), a read-only memory (ROM), a rigid magnetic disk and an optical disk. Current examples of optical disks include compact disk-read only memory (CD-ROM), compact disk-read/write (CD-R/W) and DVD.
A data processing system suitable for storing and/or executing program code will include at least one processor 602 coupled directly or indirectly to memory elements 604 through a system bus 610. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories which provide temporary storage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution.
Input/output or I/O devices (including but not limited to keyboards 608, displays 606, pointing devices, and the like) can be coupled to the system either directly (such as via bus 610) or through intervening I/O controllers (omitted for clarity).
Network adapters such as network interface 614 may also be coupled to the system to enable the data processing system to become coupled to other data processing systems or remote printers or storage devices through intervening private or public networks. Modems, cable modem and Ethernet cards are just a few of the currently available types of network adapters.
In any case, it should be understood that the components illustrated herein may be implemented in various forms of hardware, software, or combinations thereof e.g., application specific integrated circuit(s) (ASICS), functional circuitry, one or mole appropriately programmed general purpose digital computers with associated memory, and the like. Given the teachings of the invention provided herein, one of ordinary skill in the related art will be able to contemplate other implementations of the components of the invention.
Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it is to be understood that the invention is not limited to those precise embodiments, and that various other changes and modifications may be made by one skilled in the art without departing from the scope or spirit of the invention.
Number | Name | Date | Kind |
---|---|---|---|
5892690 | Boatman et al. | Apr 1999 | A |
6356842 | Intriligator et al. | Mar 2002 | B1 |
6731952 | Schaeffer et al. | May 2004 | B2 |
6978217 | Morozumi et al. | Dec 2005 | B2 |
20020026284 | Brown | Feb 2002 | A1 |
20020029111 | Peek et al. | Mar 2002 | A1 |
20020144537 | Sharp et al. | Oct 2002 | A1 |
20030154055 | Yoshimura | Aug 2003 | A1 |
20040010372 | Schwoegler | Jan 2004 | A1 |
20050197775 | Smith | Sep 2005 | A1 |
20050222802 | Tamura et al. | Oct 2005 | A1 |
20050240378 | Smith et al. | Oct 2005 | A1 |
20060176169 | Doolin et al. | Aug 2006 | A1 |
20070009104 | Renkis | Jan 2007 | A1 |
20070168131 | Root et al. | Jul 2007 | A1 |
Entry |
---|
“A Guide to the Ontario Air Quality Index System.” Mar. 1991. |
Ken Hinckley et al., “Sensing Techniques for Mobile Interaction”; research.microsoft.com/users/kenh/papers/PPC-Sensing—bw.pdf. |
Hans-W. Gellersen et al., “Multi-Sensor Context-Awareness in Mobile Devices and Smart Artefacts”; www.comp.lancs ac.uk/˜hwg/publ/monet.pdf. |
Karin Leichtenstern et al., “Analysis of Built-in Mobile Phone Sensors for Supporting Interactions with the Real World”; www.medien.ifi lmu.de/permid2005/pdf/KarinLeichtenstern—Permid2005 pdf. |
Office Action dated Nov. 6, 2013 for U.S. Appl. No. 12/613,758. |
S. Mangold et al., “Applying Pattern Recognition Techniques Based on Hidden Markov Models for Vehicular Position Location in Cellular Networks,” 50th IEEE Vehicular Technology Conference (VTC1999-Fall), Sep. 1999, 5 pages. |
“CX70 Emoty MMS at an Entirely New Level,” Siemens Mobile—Mobile Phones Portal, available at: https://web.archive.org/web/20040911111811/http://www.siemens-mobile.com/cds/frontdoor/0,2241, hq—en—0—63511—rArNrNrNrN—pageType%253ADescription,00.html, Sep. 11, 2004, 2 pages. |
A. Mihovska et al., “Location-Based VAS: Killer Applications for the Next-Generation Mobile Internet,” 12th IEEE International Symposium on Personal, Indoor and Mobile Radio Communications, Sep. 2001, pp. B-50-B-54, vol. 1. |
A. Marugame et al., “Focused Object Extraction with Multiple Cameras,” IEEE Transactions on Circuits and Systems for Video Technology, Jun. 2000, pp. 530-540, vol. 10, No. 4. |
S. Yaguchi et al., “Arbitrary Viewpoint Video Synthesis from Multiple Uncalibrated Cameras,” IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics, Feb. 2004, pp. 430-439, vol. 34, No. 1. |
Number | Date | Country | |
---|---|---|---|
20080183389 A1 | Jul 2008 | US |