Patent Application
20080040449
The scientific method begins with observation, that is, data acquisition. Sensors extend the reach and refine the precision of a human observer. For centuries all sensors (from Galileo's telescope to Hewlett and Packard's oscilloscope) were analog in nature, and gathered data only in forms that humans could perceive. Then sensors reached beyond the visible (across the electromagnetic spectrum, out into the astronomically distant heavens, down into the infinitesimally small, and eventually reaching within the atom to observe the operations of the nucleonic forces with the cloud chambers, radiation meters and scintillographs of physics). Sensors also have moved from observing static to moving to dynamic to forever-energetic conditions, until now even the flickering wisps of perception, thought, and life in the living brain have become viewable (as shifting colors within the frame of a living skull) through positive emission tomography. Yet observation alone, no matter how extended by sensors, is not all that there is to the scientific method.
Recording data that had been acquired allowed it to be transmitted across both distance and time. Allowed the data to be compared and contrasted, added to, sorted, ordered and re-ordered and classified. Yet important as the art of Taxonomy was and remains to progress today, the ancients such as Aristotle and Galen knew of such—and yet none of the classical scholars, the Greeks of Athens, the Romans of the Augustan Empire, or the Arabs of Cordoba used the scientific method. Observations were recorded and transmitted, but not used. Few made any comparison, or queried whether such comparisons might mean anything beyond what had been stated by the preceding authorities. Recording data alone, no matter how accurate the transcription or honest and skilled the monk, never lit up the dark age ere the Renaissance.
Recording enables comparison, and comparison eventually leads to analysis; and analysis, first to questions, then to theory, then to testing. Flaws in one attempt at data acquisition led to a second, improved attempt. Observations and recordings began to become connected, one set to another. Eventually, a new process evolved, wherein recital of data one had acquired was no longer sufficient. Observation, comparison and review of the process of data acquisition, as well as of the data acquired, became crucial to the new experimenters. To ensure accuracy, reproducibility became the watchword: to ensure reproducibility, observing and commenting upon the process became as much a part of the dynamic of data acquisition as the observations themselves. The scientific method may have arisen from data acquisition, but it depends as much if not more on observation of itself in process, on the dynamic structure and storage and analysis of process and result together. It is the interaction of the observer with the process, the annotation and analysis of the process during the acquisition, that fully enables the scientific method to reach its most rapid and useful stage. The scientist could take his instrument and notebooks into the field with him, make and annotate his observations synchronously and interactively, and thus remain thoroughly and directly in control of the overall process to ensure its most efficient and effective operation.
The final step from abstract reasoning to practical application of the new scientific method was formalized when merely theorizing (on the results of the data acquired or on the process as annotated) no longer proved sufficient: when action based on such observations became possible even during the process. The feedback had to operate on both the action and the process itself, for no one could guarantee ahead of time whether a particular error might arise from the instrument, the process, or an unexpected element in the environment.
A modem scientist, technician, or even methodical businessman, depends on data collection, analysis, and action to continually refine, improve, and further his knowledge and exertions. Armed with analog sensors to seek out data, with means to record the data, and means to compare and analyze the data, he yet would be floundering, drowning in a sea of events and observations beyond human capacities were it not for one further major innovation; the computer. Mastery (or at least control) of details in digital form serves as the tool for the human brain, as the sensor serves as the tool for the human sense. But to bring the two together, the analog must be converted to the digital, the data formatted to that which the computer, rather than the human, may read and recall. Yet it and the results of such analysis must also be presented back to the user in a form he can comprehend and make use of with the minimum of special training, to render the feedback process most effective. Data became digital for faster, more efficient evaluation. The notebook became the digital computer, with number-crunching analytical prowess. Early computers were room-sized; early analog-to-digital converters were suitcase sized. And as a consequence of this transformation, field observations of the human observer and the subsequent analysis became once again separated; the human was displaced in location or time from the interaction between sensor and environment. Only at the very end of this century did the potential for reuniting the observer in the field with the instruments of both observation and analysis once more become feasible.
The faster that data can be acquired, or the more remote it is from purely human capabilities, the more crucial becomes the ability to use human judgment of context and condition during acquisition. Testing, observing, reacting, and testing again speeds up the process, made more accurate the results. The processing between the ears of the observer was and still is as important as the processing between the sensor and the data record. And the demand has been ever-growing for means to bring this method to the hand and eye of the individual observer and actor at the test-site and during a session; this reduces the costs and problems which any removal in time or space can cause to invalidate an experiment. The more human involvement can be assured, the less robust and rigid need the pre-designed or pre-made sensors and acting responses be. The closer the feedback, the more efficient the entire process becomes. Yet the more that information can be duplicated, distributed, and opened to external commentary, suggestion, analysis, or action, the more valuable any given set of observations may become. One needs data that fits within both the hand of the user and can be shared globally with colleagues everywhere, for the best and most powerful application of such data. And users need tools that fit them (and the environment of their use), or that can be changed by the user to do so, to make the best use of the users time and the effort of bringing the tools to the task and into the field.
This has led to an easily-stated problem; how can a human bring tools for his senses and his mind into the field, where the observations will be taken? People can only carry so much (just as they can only sense so much or remember so much or compare so much), at anyone time and in one place. Portability becomes crucial, for data exists where it exists and not (however much the pure scientist may wish it) solely and easily within the confines of a laboratory environment. No mountain will come to Mohammed should he wish to measure its height with a non-portable laser altimeter and hundred-kilogram mainframe with full tape decks (to process the results of each sighting shot)!
The addition of the digital computer and the mechanical sensor, though they brought a salutary discipline and stringency to observations, separated the observer from the instrument, the records, and the process. Unlike a clipboard, a computer could not be taken into the field. (And was much more subject to disaster in any precipitate event—at least after the Bic became common.) In the thirty to forty years between the Eniac and the Palm Pilot, a generation of scientists, doctors, Technicians, and others struggled with a host of problems. A great many technical and scientific observations must be slightly modified from the original scheme on the fly due to unexpected, minimal, environmental and situational circumstances. Reviewing the patents and prior art in the field emphasizes the need for a reunion between the observer, the instrument, the records, and the entire process.
Furthermore, the act of separating the observer, sensor, and digital computer, however much it offered in accuracy, ease of recording, and analysis, did so at the expense of the potential to alter the situation in accordance with instructions, advice, or guidance from the observer, or from anyone engaged in concurrent and ongoing analysis. Such interaction could often correct for flaws which only become perceived in the operating context, as no projection ever entirely meets the complexity of the real world environment. Rejoining the observer with both the display and the process can prevent, among others, a vicious cycle of incorrect observation/incorrect correction (a variation of ‘GIGO’ or ‘garbage in, garbage out’) from arising, which means that runaway feedback can be readily averted rather than being triggered by a single out-of-bounds observation. But the solution offered in this embodiment of the invention has been woefully absent.
For example, the lightweight, self-contained, programmable data-acquisition system described in U.S. Pat. No. 5,638,299 is entirely dependent upon a remote computer, rather than one located in the user's hand, for both display and analysis of the data. As such, it fails to consider the need for a user-interface or unit controls where the user is located by the sensor, in order to render the data acquisition, analysis, and action processes interactive. Moreover, because the display (of the results of the data acquisition) is separated from the observer, there literally is no chance for that observer to be aware of or to make meaningful comments upon the results of the process as they are displayed. Accordingly, there is little potential for conscious correlation between corrective or insightful contextual observations and displayed results. Such potential as there is either requires the observation to take place where the host computer is located, or requires secondary analysis that tries to reconnect the time or context after the fact of any event rather than contemporaneously, which builds in a source of potential error in the feedback process and thereby limits its usefulness. Moreover, as the sole source of commands and configuration arise from the remote computer and cannot be displayed at the site of the observer and sensor, there is no chance for the observer to add meaningful interaction or effect any adaptation by the individual who is present and able to incorporate his or her own judgment or sense as part of the adaptive process.
Another missing element can be seen with U.S. Pat. No. 5,386,360. That invention, though it envisions data acquisition, analysis, and access that depends on a personal computer, sees as the sole corrective measure for a faulty sensor, rendering it inoperative. While this could prevent a vicious cycle from arising, there is little room left for further corrective measures. No distinction is made between transient, correctable, or permanently fatal faults; an out-of-range reading damns a sensor permanently. Also, unlike the current embodiment of this invention, the observer using that invention has little opportunity to annotate the process directly, potentially thereby allowing a later interpretation to accurately adjust for a localized, temporary, or unpredicted effect. Furthermore, that invention contains no mention of re-calibrating the sensor or the analytical process by the operator. And. while a personal computer can be carried with greater ease than a mainframe, it (plus the attendant monitor) lacks a great deal of ease of portability into difficult or hard-to-reach environments by any save those in good physical condition, while its power requirements are prodigious enough to make extended or often-repeated runs impractical. But the greatest weakness in this patent is that there is no need to limit the data input to the subset of possible values that can be forced through the translation of data keystrokes, which constrains potential digital from analog sensor readings to an artificially-imposed limitation of some hundred-odd possible signals. While one can always translate any radio broadcast into text or even into Morse code, doing so always robs the subtleties of tone, pace, pitch, and background noise that can be more important than the linear wording when it comes to conveying proper meaning. (Or greatly increases the ‘data density required to translate identical readings; it takes much less time to say ‘S.O.S.’ than its Morse equivalent.) Moreover, this invention, though it orients a user to the location and function of the equipment, does not orient the user to the capabilities, constraints, and dynamics of either the equipment or the processes invoked in its use or analysis, or action based on either or both of the preceding. It is a significant advance to allow the use of graphical icons, perhaps; but there is no mention of any flexibility in letting the user dictate the interface according to that which the user finds most serviceable or appropriate to the environment.
Environmental data acquisition via remote sensors, as described in U.S. Pat. No. 5,132,968, need not require radio communication means. There are any number of times or conditions where such would be difficult or dangerous to employ, or merely expensive, as that invention requires sensor(s), computer(s), and user(s) to possess or at least have access to radio communication means. A user may want to obtain direct readings via technologies not subject to radio interference, may not want to risk broadcasting a signal that could be misinterpreted or received by another unanticipated recipient, and be quite capable of modest physical effort to bring a hand-held device to the sensors in turn. Should there be any risk of invoking a blast (e.g. near a construction site using radio-triggered explosives), or the field strength of the radio or magnetic signal of the phenomena around the experiment be crucial, such radio communication means could readily provide more hazard and less value that the user would wish. Moreover, this invention fails to include any means for interacting with the user, fails to include him in the feedback cycle and allow for annotation of the ongoing process, and never considers the potential necessity of recalibrating a sensor rather than abandoning or replacing it. He is reduced, once again, to somehow pasting on handwritten notes and keeping them linked to the computer records.
But focusing on handwritten information alone, as U.S. Pat. No. 5,227,614 does, misses the power of interacting with analog-to-digital sensors. That patent is solely and narrowly focused on capturing handwritten information (subparagraphs (d) and (e) of the sole independent claim specifically address ‘means for digitizing handwritten input information’ and ‘application of handwritten information to said digitizer input means’), though it does see some advantage for hand-held data capture within a ‘shirt pocket’ size (subparagraph (c) of claim one). There are far, far more analog sensor readings possible than mere handwritten information can capture in an accurate, timely, and effective fashion. The specification for that invention, when detailing the software command set, focuses even more narrowly on capturing handwritten signatures, with five commands aimed at self-checks (e.g request version or revision number), six aimed specifically at signature capture, and four moderating the mode of capture. It does not address the need for interactivity during data capture to assess the accuracy thereof, or the need for analysis and comparison within the hand-held device for on-the-spot action, or the need for comparison either across multiple data inputs or of a particular data input against a library and/or remote storage for decisive reaction to the current circumstances. And the specification focuses on analog, rather than digital, and commercial or human-oriented interactivity rather than scientific or analytical processes as the core of the functions for the hand-held device. (For example, the specification mentions without detail input” of existing printed data, e.g. bar codes, text, and graphical information on p. 11. On page 13, it adds as possible, associated. input/output devices “a touch keyboard, digitizer tablet means, printers, laser bar code readers, FR modules, smart card interfaces, disk systems, full travel keyboards, larger displays, local area network interfaces, et cetera.”) None of these are considered generally to be suitable for direct digital analysis or scientific use. In fact, this patent's discussion on “Digital Signal Processing” is related to barcode reading (pages 17-23). Unfortunately, the real world is not universally pre-equipped with Universal Product Codes.
There are a number of more limited, special-purpose inventions that are not coincident with this invention, though they at first glance may seem to interact with particular claims herein. For example, U.S. Pat. No. 5,876,351 is a claim to a “portable and modular electrocardiogram (ECG) medical device.” Despite the language immediately prior to it in the specification, which argued that the patent should be extended to “any predetermined diagnostic medical function, the claim explicitly limits it to ECG readings. In contrast, the present invention is not so limited. Even the subordinate claims for ‘medical applications’, that may cover ‘ECG, other heart function, and integrated heart-and-nervous system reading’ applications, are not just for isolated. single readings but involve their storage, analysis, comparison, and the interactivity with other software-based systems for decisive, expert interaction based on continuing readings rather than an individual, or even a series of individual and disconnected snapshots. Furthermore, the invention in U.S. Pat. No. 5,876,351 specifically requires both a single, particular user control (a joypad) and dictates the meaning and use of the arrow keys therein, rather than having the user interface conformable by software to the user's preferences within the constraints of the application's functional needs and possibilities. It requires, however, not merely that the device contain a joypad (which is but one possible control for a user interface) but specifies precisely the limited functions which can be invoked by the pairs of keys therein. Moreover, this invention fails to disclose any means for modifying the user interface (or even the conception of such a possibility), means for interacting with the user during the analytical and action processes after an observation, or engaging in a dynamic process of continuous or event-driven monitoring, as opposed to simply taking a single reading (or even a discrete set of single readings). And. to the extent any heart-monitoring use claim exists for this invention that does not depend upon incorporation of an ECG, this patent will neither be limiting nor apply.
Similarly, U.S. Pat. No. 5,827,179 makes claims for a “personal computer card for collection for real-time biological data.” The invention again depends upon an external computer system which, when the card is added to it, becomes a ‘powerful diagnostic instrument’, The problem is, of course, that while the PC card itself that is the subject of this patent may be lightweight and portable, the system-as-a-whole which necessarily includes a personal computer fails such a test, As all the user-interface, interaction, storage, analysis, display, and interactivity depend upon the externally-linked personal computer, this patent fails to comprehend the present invention. Moreover, to the extent that its claims specifically require the incorporation of at least one, and in some cases several, particular and specific sensors (including a pressure transducer), it fails to limit the present invention which has no such specific requirement for full functionality.
The claims in U.S. Pat. No. 5,410,141 belie the broader text of the title and specification, for they require both a wireless transfer and an external host for what is a ‘data terminal’ rather than a self-contained unit. Moreover, that patent focuses entirely on data capture, without any user interactivity concerning the process. And as the specification makes clear, the focus of the entire patent is on a generalized claim to a bar-code reader (see e.g. claim 57's recitation of ‘removable and replaceable code scanner module means’), rather than any external analog-to-digitally based environmental sensor(s).
The separation stated in U.S. Pat. No. 5,220,522 not only focuses on the keyboard port and bus, but also lacks the integrated performance and package of the current embodiment of this invention. Instead, its claims recite a clear separation between peripheral sensor, host computer, and peripheral data acquisition, monitor, and control device.
The invention and claims of U.S. Pat. No. 5,587,577 for a ‘Modular Scanner with Hand-Held Data Terminal,’ not only focus on the physical aspects of a data acquisition system, but lack any element or claim as to synergistic operation with the data during or after its acquisition; for, as the title suggests, the machinery and process attached to the user is merely for data acquisition by the machinery, with the human playing little more than the role of a carrier.
Remote acquisition is disclosed in U.S. Pat. No. 5,790,977, “Data Acquisition from a Remote Instrument Via the Internet.” The invention requires a host and remote system. However, and lacks any personal portability. The separation between display, analysis, and sensing subjects the user to all the uncertainties of a less-than-stable internet, which means that failures from mere communication mishaps now are added to the existing environmental uncertainties.
This embodiment of the invention focuses on bringing the human observer back into the heart of the process, making it feasible for him (or her) to interact with the data observation, analysis, and consequent action. Because the invention can be readily carried, and linked to any of a number of types of sensors (any that can be governed by analog-to-digital conversion, that is), its applications are not limited to a particular field. Because software governs the translation of the digital signals, the operation of the hand-held device itself, and the display of the results (or of the analysis, or even the guidance to the user), the inflexible limitations of hardware are overset; rather than being limited to a specific-purpose device or instrument, the observer has the set of potential instruments (or the set of potential reading).
This invention generally relates to a device and method for data acquisition, comparison, and analysis that comprises a hand-held computer device with an attachable module having its own associated analog sensor, and having stored software suited or adaptable to the sensor, the attachable module, and that particular user for that particular use. The invention includes hardware and software that lets the human interact with computer and sensor combinations, that permits event—rather than clock—driven observations, that allows human senses, capabilities, and judgment to serve as an extension to the computer and digital processing synergistically with the computer's and sensor's capabilities, thereby most readily and flexibly extending that of the human. A further extension of the invention specifically allows the annotation of both the process and environmental context of the data acquisition beyond the limitations of the particular sensor and attachable module, capturing the ‘between the ears’ processing of the human observer, which can be as critical to ensure that the meaning is correctly ascribable to the results of the process as the data observations themselves are. A still further extension of the invention specifically allows interaction with the hand-held computer and associated attachable module with its sensor in order to calibrate and examine the combination, creating the potential for correctly evaluating the entire environment including the sensor and the process on which the data acquisition and analysis depends. And a still-further extension of the invention allows the interaction between the external conditions and the computer or the human or both together, thereby creating the opportunities for feedback loops and dynamic adaptation in real time. Once the data-acquisition dynamic is thus made available to the hand-held computer's user, because there is the potential for further linkage to additional, externally-based resources (such as an external computer. additional memory, further sensor, or additional programming), the flexibility and adaptability become limited only by what the communication channels can bear or the human user can manage, operate, handle, or is carrying to which the hand-held can be also adapted or modified to process and control. (No claim is made for using additional human observers in the data acquisition, comparison, or analysis; the use of trainees or graduate student assistants is already well-known, however imperfectly practised.)
For example, a human user could carry the hand-held computer device with an associated attachable module and sensor attached thereof for testing soil pH into the field to track and trace a suspected spill from an underground pipeline of an environmentally-hazardous substance. By calibrating the sensitivity to match the current environmental conditions, ranging from the day's temperature to the mix of soils subject to being tested, the most useful range of readings may be assured. Simultaneously, the user can be adding annotations as to smells, or sights, which the analog sensor and hand-held computer are not currently equipped to detect yet which may add or explain significance of readings, such as noting a ‘discoloration’ on the surface of the soil or indeed on the probe when withdrawn from the soil. The user could also correctly annotate the calibration process such that conversion from one set of records to the next could be accurately made. And, by using the connection to a network, the user might even be able to order the shutdown of the nearest valves to the greatest concentration of the spill and the opening of all those ‘downstream’ in the pipeline, thereby allowing the maximum safe drainage and minimizing the total extent of the spill. What previously may have required teams of workers, fields of sensors, repeated runs of sensor placements, test-runs, and off-site analysis, taking hours or days or weeks, can now become a single interactive process that combines the best of both human and digital capabilities in a portable, flexible, synergistic format.
It is this combination of capabilities that makes the difference from prior inventions in this field. Most leave out the human operator or human element, leaving themselves vulnerable to any internal failure in the sensor or the system, leaving themselves blind and deaf to elements of the environment forming the context for their data acquisition that may affect the process and the results, separating themselves from the human-based capabilities and skills and therefore having to repeat or re-run or re-evaluate the data acquisition results when problems or concerns or questions about the process arise, rather than solving them on the spot at the time. Many leave out the capability to act in real-time and on the spot as a consequence of immediate analysis, thereby abandoning all potential for a feedback-based interactive process. Also left out are the capabilities (1) to annotate the process and observations on a real-time basis by the in-the-field user or operator of the device; and (2) to calibrate the device/sensor/software combination for each particular context, whose absence either requires blind faith in the perfectibility of human-driven data acquisition and computer-monitored analog sensing, or risks creating an unreviewable and irreproducible (and hence, unusable) recording lacking contemporaneous evaluation of its accuracy, validity, and completeness.
As the user's input is a crucial portion of this invention, there are many formats which may be known by or familiar to a particular user. A user interface that can be customized by an individual user to that format which is most suitable for his current need (for sometimes accuracy, sometimes a more general summary, and sometimes the merest of assurances, are needed for the annotative, analytical, and active phases of the entire feedback process; but a user who cannot chose which is most appropriate at the moment will find his capacity limited by the harshly imposed process constraints of the system. Accordingly, an aspect of this invention is the deliberate capacity for flexibly altering the user interface according to the particular needs of the user as determined by the context and time of his interaction with the entire process.
The combination of human and computer capabilities synergistically permits adaptation to and awareness of contextual events, giving the user and the system a flexibility not feasible otherwise. Most data collection can be more readily made duplicable and analyzable if first digitized, yet the process of collection and digitization may require supervision or calibration to cope with differences between expectations and real-world conditions. Transitory or unpredictable conditions that otherwise may create tremendous difficulties for computer handling, or which may otherwise require invalidation of an entire run due to concerns over sensor or digitization flaws, can be annotated by a user or subjected to immediate re-evaluation. Analysis that suggests intervention or alteration can be performed by the human user who is on the spot, or signaled by or through him to handle problems beyond the scope of a particular specialized computer program or hardware. Both digitized analog sensors taking readings and human actions can be controlled either by the hand-held computer or user, while supported by a computer or networks worth of specialized records, programming, and actions. The capability to sense, evaluate, compare analyze, and act based on contextual cues from sensors not present or currently activated in the hand-held computer, or from machine-based sensor readings not otherwise accessible to a human user, allows immediate context-based feedback on both the process and the results. Experimental corrections can be made that incorporate local or remote judgment and analysis without delay or separation between the observer and the actor through the intermediary of the hand-held device with the appropriate attachable module and associated sensors.
The process of effectively transforming data or observations into knowledge, and then putting knowledge into action, works best when the three interrelated activities (acquiring observations, monitoring the data-acquisition process, and acting upon an analysis arising from that data) are integrated, allowing direct, immediate, and localized feedback.
Errors can arise in perception, in analysis, and in action Anyone of these, if not immediately subject to review and correction, can turn feedback from a virtuous to a vicious cycle. The closer and more accessible the process is to a human observer, the more opportunity for immediate correction or reaction to prevent such a problem from arising. The greater the separation in either time or space between the sensor, the computer, and the human observer, the less efficient and effective the scientific method becomes.
Embodiments of the invention are described by way of example with reference to drawings, in which:
[copy from other spec-no claims]
Both 8A and 86 show that at many steps in the process, the user may interleave annotations (60, 68, 74, 76, 82, 88, 108, 114, 116, 122) which may refer to that step or to conditions outside the process.
Because the application software expands the flexibility of the hand-held computer device, by allowing the set of physical controls to he configured to the needs of the application and the user, and by allowing interactive controls to exist within the display, a series of drawings indicating the type, nature, and effects of the software are included. These, are not meant to be exhaustive as their nature depends upon the particular application and sensor.
A different display is shown in
In one embodiment, a hand-held computer device includes a display screen, an operating system, a central processing unit, memory for storage of programs and data (both dynamic and static), means for connecting itself to an external computer, a set of controls for user interaction, means for interactive data acquisition, and an attachable module for data acquisition which includes an analog sensor an analog-to-digital converter, and means for communication with the hand-held computer device.
The hand-held computer device (also referred to as a ‘shirt-pocket’ computer device) contains in one compact and lightweight package the entirety of the operating system/input/display/recording/programmable and long-term memory features of any general-purpose and interactive computer that, with the loading and operation of the appropriate application software, can become any of an expandable set of specific-purpose tools. As the hand-held computer device is both readily portable and fully functional, it combines into one exceedingly useful and accessible package the heretofore separated sensor/computer/display/input-output functional elements that allow the mixing of the best of human adaptivity and computer precision. It can be carried where a laptop cannot. And, because it has such a minimalist weight and bulk penalty, it will be carried when heavier, bulkier, more exhausting and more awkward combinations are left behind.
Because this embodiment of the invention has a display screen, the user/observer of the hand-held computer device has a view into the ongoing process, whether that be of data acquisition or analysis of the results. Without such a ‘window’ (presumably the generic term is still within the public domain) on the process, none of the benefits of synergy or interactivity are possible. The display, moreover, is one that is modified by the software which is discussed in further detail below.
Because this embodiment of the invention has an operating system, central processing unit, and memory for storage of programs and data, it can do more than one particular task, and retain the results for more than one observational run. Moreover the hand-held computer device can use application software to compare and analyze observational records. These processes, and the results, can be displayed as they occur via the already-mentioned display screen; which can mean that an intelligent human observer may be able to perceive patterns or trends in ways that the computer has not yet been programmed to do, even in the middle of a series of observations; which, in turn, can lead to on-the-spot decisions that benefit the observer, the record, and the process. For example, a human observer might note that soil pH readings link with a particular flora in the Amazon rainforest, and actively seek out or avoid areas with a higher density of such plants. Or a disturbing trend in chemical analysis, though it has not yet led to a disruptive event, may alert the observer to the potential for a crisis before it erupts, allowing him to either leave or intervene to prevent it in the first place.
Because the hand-held computer device has means for connecting itself to either an external (but, please note, not necessarily a host) computer, or a network (which in turn may include multiple computers, peripherals, or other hand-held computer devices), two major advantages ensue. First, the flexibility of the hand-held computer device is not limited to its memory capacity, as the desired application software can be transferred from outside.
A crucial distinction between this embodiment of the invention and several others disclosed above is that the hand-held computer device, as it has the general flexibility of any computer, can and does use the software to modify its controls to the needs of the application and the user. The display on the display screen can similarly be modified by internal software fitted to the user's need. As these controls include any software-configurable input (or output) means, the device is no longer a Procrustean bed that forces the user to its limitations; but, rather, a user-friendly and adaptive tool that can be modified to meet the needs of the user at the moment.
For example, one user may feel most comfortable using a stylus and alphabetic listing to activate an iconic software control or to ‘pick out’ annotative notes. Another may wish to use a particular stylus-driven alphabetic shorthand (or even the aged Gregg shorthand, depending only on finding a software implementation thereof). A third may wish to use layered menu choices, while a fourth may prefer a hierarchical tree of possible interactions.
Some users may wish the controls or the process (as opposed to the controls for, say altering the scale of the display) to be displayed on the screen for direct interaction. This embodiment of the invention permits that; moreover, it would allow (as long as the enabling software has been loaded) a user to specify which controls appear on the left, right, top, bottom, or center of the display screen. Not everyone wants the ‘trash can’ at the bottom right corner of their display! Enabling that flexibility of the user interface and surpassing the physical limitations of any device's hardware is a significant aspect of this embodiment of the invention. Others may want the controls to be linked in a particular fashion to the hardware controls available on the handheld computer device. Others may want to change the alphabet, or even the language—and again, as long as the enabling software has been loaded onto the hand-held computer device, they may get what they want or need.
Furthermore, the controls needed will depend in part on the task(s) to be performed. Taking a series of temperature readings? Do you want the results to be displayed in Celsius, Kelvin, or Fahrenheit? Do you want readings every second, every tenth of a second, or every 10 seconds? Do you want a bar graph, a linear graph, or a maximum/minimum range over the total observations displayed? Because there are means for interactive data acquisition, the user and the use together determine the set of controls, their layout, and their usage and effect. Change the task, however (soil pH, or blood glucose level, or light intensity), and the controls and interactions may need alteration as well—alteration which is driven by the application software, running within the limitations of the hand-held computer device's operating system and hardware controls.
Because the hand-held computer device also includes means for interactive data acquisition by its user, the user of the hand-held computer device is also the observer (due to the display screen. central processing unit, and memory) of the data acquisition process, able to interact with the process (thanks to the set of controls), and can use the entirety for any of a wide variety of specific purposed when combined with the appropriate attachable module and application software, as detailed below. Once again his actions, decisions, and observations become part of the entire process. Rather than having analog sensor, digital recording, computer analysis, human contextual observation all separated, this embodiment of the invention joins them all together in a process of synergistic, context-sensitive interaction that uses the best strengths of both humans and computers.
Because a particular attachable module can be configured to a particular task, while a wide variety of possible observations and interactions can be enabled through the appropriate set of attachable modules, this embodiment of the invention maintains the flexibility and adaptivity of the general-purpose computer. As the attachable module contains both an analog sensor and an analog-to-digital converter, and means to communicate with the hand-held computer device, the entire combination retains its adaptivity subject solely to the potential software limitations of the particular hardware. No longer would a technician necessarily have to carry three different sensors with varying levels of sensitivity, if a software filter can be devised to adapt the raw analog data to the appropriate digital output. One sensor can be set to detect, for example, a range of lumens from 1 to 1,000, while the same sensor can be programmed on the next run to detect a range of lumens from 500 to 5.000. Any sensor that measures within the limitations of the hand-held computer device's and attachable module's hardware and software (in the best embodiment these are from 0 to 5 volts with power requirements not exceeding 200 mA) can be attached, and these already include at least the following: accelerometer, barometer, biology gas pressure sensor, colonmeter, C02 gas sensor, conductivity probe, current and voltage probe, dissolved Oxygen probe, EKG sensor, flow rate sensor, heart rate sensor, light (presence, intensity), magnetic field, pH, pressure, relative humidity, respiration, force, thermocouple, and voltage sensor.
In one embodiment, the hand-held computer device further includes an internal on-board clock which the operating system and application software can integrate into an application, to allow a user to view time-based observations or to enter as part of user-driven annotation. This allows the data acquisition to be driven by relative time (intervals of length×between observations) or by absolute time (readings to be taken at time X GMT). The hand-held computer device also has an on-board battery rendering it independent of external power sources, and can obtain power from and provide power to external connections, including the capability of providing power to the attachable module, which in turn can provide power to the attached sensor within certain ranges.
As the hand-held computer device has both the capabilities of its internal memory, cpu, and display, and can communicate with an external computer, this embodiment of the invention is capable of running an expert system software program that can guide the user or data analysis interactions, with the location of the software depending upon that program's memory or that communication means' transmission limitations.
When carried into the field by the user, the hand-held computer device (in the best embodiment, a Palm-based operating system hand-held computer such as the Palm Pilot or Handspring Visor), attachable module, and external analog sensor are combined with the appropriate application software, so the user had the means for data capture, analysis, annotation, and calibration (of the sensor or analog-to-digital translation) as necessary for immediate, interactive, and synergistic observation, analysis, annotation, and action. Data readings are taken in a one or more trials (or experiment runs). As the observations progress the user is able to annotate the process, the results, or even the actions taken both within and without the hand-held device's limitations. The user may choose to control the data acquisition process manually, with inputs being taken each time he activates the—sensor; or he may set up a series of observations which will then be run by the hand-held computer device. The interval between observations, the scale of measurement, the sampling rate, and the measurement units may be selected by the user through the controls physically present in the hand-held computer device or as virtually present through the display screen if it has been made interactive.
Additionally, the user may view the data as it is acquired, changing the display on the display screen to meet his needs. He can alter the scale, select an individual reading or set of readings, move the time interval displayed (forward or backward), or even change the trial currently being displayed. Annotations may be entered linked to particular observations, to a trial, or to the process as a whole, which allows the user to add to the observations of the data observations on the process, and to add to the analysis observations on the analysis, the results, or the process. Anyone of the annotations may include elements within the power of the human user that are not within the existing limitations of the hardware and software he is using. Light sensors may be affected by clouds or by the temporary shadow of a moving object in the environment (such as a jetliner flying overhead) which was not predicted when the trial was first proposed; yet by annotating the event otherwise anomalous or difficult data readings may be accounted for.
Furthermore, if there is a need for more than a particular sensor (more than an individual sensor, or more than one type of sensor) to jointly cooperate and interact, this embodiment of the invention allows them to be conjoined via any network. The network in turn could be monitored or driven by any particular hand-held computer device or by any intermediary computer; alternatively, the intermediary computer or other hand-held computer devices could be used to provide additional processing power or memory under the direction and control of the single hosting hand-held computer device. Therefore this embodiment of the invention includes the method of using a human plus a hand-held computer device with an attachable module with the appropriate sensor and application software to take necessary scientific, technical, or other instrument readings in one or a series of observations, and then, in a further extension, in using the same human plus computer combination to interact with the environment under the direction of any software-based or human-based advice communicable through the network to the individual human on-the-spot where the readings are being taken. For example, a low-level medical technician may use a hand-held computer device with the attachable module with its sensor, and application software, to track the blood glucose level of a borderline diabetic patient. Upon receiving a reading which is out-of-bounds (too high or too low), the technician could be alerted through the hand-held computer device to administer the appropriate corrective treatment.
Additionally, if the attachable module and application software are properly configured (which may be done by downloading from an external computer the appropriate software) the hand-held computer device and attachable module may be used to drive a direct interaction with a further peripheral connected to the outlet(s) of either the attachable module or the hand-held computer device itself. For example, a small group of technicians may be sent along a stream in which a pollution trace must be followed, which runs parallel to a pipeline. Upon finding the greatest concentration of the pollutant with the upstream not being polluted, that individual can be directed to the pipeline where, either through expert-system advice communicated through the hand-held computer device or, through software empowering an attachable module that can be linked to the pipeline controls directly, the closest upstream valve to the pipeline can be closed to shut down the leak causing the pollution.
In some embodiments, the hand-held computer device includes central processing, memory, and a display unit from a commonly available platform such as the Palm Pilot (available from 3Com Corporation) or the Visor (available from Handspring Corporation), either of which uses the Palm Operating System for its basic operating system software. These also permit communication with external computers through ASCII text and browsable HTML files, and link with such commonly available external computers as PC-compatible. Windows OS based (95/98 or NT) personal computers of Macintosh (Mac OS 7.5.3 or greater) personal computers, commonly available.
There are a number of analog-to-digital sensors that have already been created and available on the marketplace from a variety of vendors. Any sensor that measures in the range of 0 to 5 volts, with power requirements that do not exceed 200 mA, or have their own power source, and provides an input signal as specified by the application software or Palm Operating System, can be used. These sensors should have 2 input channels, a modular ‘telephone jack’ connector (DIN5 adapters are commonly available), permit user configurable sampling rates of up to 400 samples per second for 1 sensor or 200 samples per second for 2 sensors, have a 12 bit resolution, and an input voltage range of 0 to 5 volts, single ended. A sample of the governing software for translating analog-to-digital signals from a sensor to the hand-held computer device can be found in an article published by Gary T. Derosiers, “Pilot Hardware Add-Ons,” Handheld Systems 6.4, July/August 1998.
Among the specific embodiments and examples are the use of a Palm Pilot, a sensor, and an attachable module and application software for detecting the pH of a liquid sample to assess water quality and trace variations through a stream; the use of a Palm Pilot, a sensor, and an attachable module and application software for detecting heartbeats for monitoring, athletic activity; the use of a Palm Pilot, a sensor, and an attachable module and application software for detecting acceleration changes during rides (thereby allowing assessment of curves and possible metal stresses); the use of a Palm Pilot, a sensor, and an attachable module and application software for detecting light to evaluate photographic conditions, light leaks, or office cubicle conditions; and the use of a Palm Pilot, a sensor, and an attachable module and application software for detecting temperature to assess the efficiency of particular insulation and radiation placements in design of an office or home.
Although the present invention has been described in considerable detail with reference to certain preferred versions thereof, other versions are possible. As long as analog readings from an external sensor can be converted to digital signals, the combination of hardware and software in the appropriate attachable module for that sensor makes the entire combination with the hand-held computer device feasible. One could add to the list of sensors above, for example, radiation monitors, particular chemical sensors, biological sensors (DNA or gene-fragment presence/absence), weight, bar code, inventory tag, infrared, motion—if the attachable module can be created and then the sensor plus module joined to the hand-held computer device, the observer can bring the strength of computer analysis to the flexibility of human contextual assessment together in an interactive fashion, using this embodiment of the invention. Therefore, the spirit and scope of the appended claims should not be limited to the description of the preferred versions contained herein.
Disclosed in this invention is a hand-held computer device which accommodates and communicates with a variety of attachable modules for data acquisition (each module having an analog-to-digital converter and at least one connection for an associated analog sensor), wherein said hand-held computer device includes hardware controls and application software for interactive, synergistic, and real-time annotation by the user of the device during the data acquisition process, and wherein said hand-held computer device can communicate with another computer or network of computers for greater processing power, memory, or embedded programs. A further modification of the device disclosed herein allows calibration of the sensor readings to accommodate environmental or sensor-hardware variations that might otherwise complicate or even invalidate the data acquisition, or limit its utility. The hand-held computer device can be used by an individual to acquire data, annotate the process, calibrate the process, and analyze and act upon the results, thereby allowing an interactive, synergistic, real-time and event-driven approach to applications in the real world, including many specific uses in the fields of scientific inquiry, medical evaluation and treatment, lifestyle and health management, technical and inventory control, and security endeavors. Further disclosed are both the device and method for immediate analysis, feedback, and interaction action amongst the user, the hand-held computer device, a remote computer or network, or any combination thereof, whether such action is to correct failings in the sensors, the conversion process, the software, or the data acquisition process as a whole, or to put the analysis performed into immediate effect. Also disclosed are means for adapting the user-interface of the hand-held computer, whether such adaptation be to adapt to the preferences or needs of the user, the particular attachable module, the current or expected data acquisition process, or the method used to annotate, analyze, or act upon the results of data acquisition, analysis, or to communicate with a remote computer or network. Also disclosed is the use of more than one hand-held computer with a attachable module and associated sensor(s) (whether the combination of each particular hand-held computer and attachable module and associated sensor(s) is identical or complementary), to gain greater real-time and context-based knowledge of and control over the environmental conditions and process than can be managed by a single point of static or dynamic reference. Finally, also disclosed are methods to use each of the above interactively, allowing synergistic adaptation on a real-time, real-world basis of computer and human sensing, evaluation, and action driven by a contextual awareness of both the environment and the process, which grants a flexibility that is impossible to obtain by any of automatic, remote, or manual methods acting separately.
The reader's attention is directed to all papers and documents which are filed concurrently with this specification and which are open to public inspection with this specification, and the contents of all such papers and documents are incorporated herein by reference.
All the features disclosed in this specification (including any accompanying claims, abstract, and drawings) may be replaced by alternative features using the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.
This is a continuation and claims the benefit of U.S. patent application Ser. No. 09/479,031, filed on Jan. 1, 2000, the entirety of which is hereby incorporated by reference.
| Number | Date | Country | |
|---|---|---|---|
| Parent | 09479031 | Jan 2000 | US |
| Child | 11893139 | Aug 2007 | US |
