Patent Application
20060227640
This present invention is related to a sensing devices which capture images and/or sound, such as optical sensing devices, (e.g., cameras), audible sensing devices, (e.g., sound recorders, telephones, personal data assistants (PDAs)), or both optical and audible sensing devices, (e.g., video recorders, videophones, camera phones). More particularly, the present invention is related to a sensing device which emits one or more detectable warning signals, (e.g., sound, light, radio signals), which indicate that a sensing functionality associated with the sensing device has been activated.
Sensing functionality includes the capture of image and sound. Sensing device functionality has become miniaturized so that people may not be aware of the physical presence of a sensing device, such as cameras, microphones and speakerphones, which may be embedded in phones, PDAs, and watches. Sensing devices are also being integrated with other functionalities such as storage and transmission functionalities. This allows for the increased quantity and timeliness of sensed data. The surreptitious use of these sensing devices in designated restricted areas poses obvious security and privacy concerns.
The problem with this situation is that these devices may be used to locally record or relay sounds and/or images to another device, without proper authorization. The restriction of bringing these devices into certain areas is posted, or a search is made for their existence.
Emissions from electronic equipment may be used as a basis for detecting the existence and use of such equipment. By increasing the strength of these emissions and by giving them a more distinctive pattern, it would be possible to make it easier for a device at a distance to detect this signature. However, if the equipment is shielded in accordance with TEMPEST specifications, signals emitted from the equipment would be weak and their pattern can not be used to detect a particular type of device by using a detection device.
The physical dimensions of sensing devices have been shrinking and the basic functionality can be incorporated into devices with various forms so that the physical presence of a sensing device may not be detectable. Any existing indications that sensing is occurring, such as the light on a video camera can be covered and/or be imperceptible within the ambient conditions.
In conventional landline or wireless communication recording devices, a warning sound or message is sometimes emitted by telephone conversation recorders. This warning is either directly or indirectly mandated by Federal and State law.
There are systems which broadcast radio frequency beacons that tell devices, such as camera phones, to disable their camera function. For example, Iceberg Intellectual Properties Limited has implemented this approach, as disclosed in International Application No. WO 2004/089021 A2 by Patrick Snow entitled “Portable Digital Devices”.
Prior art sensing device detection schemes assume a degree of cooperation from the users of the recording devices. They also in general require that people do not attempt to defeat signaling mechanisms that disable the recording functions. The exception to this is the use of a camera function to detect signals that disable recording functions. This implementation however does not alert the target of the pictures of the occurrence of the recording.
The flash of a camera would be rather easy to detect, and utilize as an indication of picture taking. Unfortunately, flash usage does not always occur, and may be disabled. Given modern low level light detectors and the post processing capabilities of digital image software, this often is not an issue in regards to picture quality.
What is needed is a way to alert a person(s) or monitoring equipment to the existence or utilization of such equipment that does not rely on explicit signaling generated by the location protection/privacy system, (i.e., sensor/beacon system), but rather have elements of the physical process of recording, (e.g., core technology), be detectable through remote sensing. Simple modification of the transducers (audio, image/video) that deal with the core technology can be made more tamper proof.
The prior art uses the core recording functionality to generate an alert signal, such as a flash, sound or the like, to alert a target that it is being recorded. However, there is a possibility that these signals may not be strong or reliable enough to be received by the target. Furthermore, the target may not distinguish this signal from an “innocent” signal, or may be misguided by a false alarm.
It would be desirable to incorporate a warning mechanism in sensing devices to facilitate searching for the devices, identifying the devices or the users thereof, confiscating the devices and/or taking action against the users of the devices.
The present invention is related to a sensing apparatus having a sensing functionality controller and a method of detecting the sensing apparatus is disclosed. A signal generation unit embedded in the sensing apparatus operates in conjunction with the sensing functionality controller to support the sensing functionality of the sensing unit. The signal generation unit generates one or more sensing functionality support signals used to enhance content obtained by the sensing unit, (e.g., a flash for illuminating a sensed target such that content obtained by the sensing unit is not too dark, or a sound or light-based auto focus signal to make sure that the content is not out of focus). When the sensing functionality controller is activated, the same signal generation unit generates a detectable warning signal, (e.g., sound, light, radio wave), which indicates that the sensing functionality controller has been activated and/or the sensing unit is sensing.
A more detailed understanding of the invention may be had from the following description, given by way of example and to be understood in conjunction with the accompanying drawings wherein:
The present invention counteracts the surreptitious aspect of sensing devices by having the sensing devices emit a warning signal which indicates that a sensing function is being performed. A sensing device can include any device which records and/or transmits images and sound, or a device which gathers samples, records and/or transmits information relating to the presence of biochemical substances.
Referring to
A warning signal detector 110, (or a hearing aid as described below), can detect the warning signal 105B and then create a human detectable warning signal 115 or a machine detectable warning signal 120.
Still referring to
Referring to
The warning signal 105 may be considered as interference for a sound sensing device 100. The warning signal filter 150 will filter out this warning signal 105 using signal processing before images and/or sound captured by the sensing unit 145 are stored in the sensed data memory 155. The user interface 165 enables a user of the sensing device 100 to input settings and store identification information in the ID data memory 170. The optional watermarking unit 175 watermarks the warning signals 105 emitted by the sensing device 100.
The timing unit 135 is used in conjunction with the audible generator 133. The timer 135 is activated via a signal 134 which is output when the audible generator 133 emits either the human detectable sound signal 1054 or the ultrasonic sound signal 1055.
Aspects of the warning signal 105 should be proportional to the strength of the sensing functionality. The degree of strength can be represented by volume, pitch, or the time spacing of on and off periods. Volume may be the most suitable parameter since the volume will naturally decrease in proportion to the distance from the source of the sound. The sound can also indicate the class of a sensor device. The volume of the warning sound signal may need to be adjusted to remain noticeable within the ambient sound.
In one embodiment, it may be advantageous to time multiplex each sound in order to allow for sounds to remain distinct, since sounds may be emitted by other devices and other sources. Alternatively, a scheme similar to carrier sense multiple access/collision detection (CSMA/CD) can be applied to the transmission of sound signals. A sound sensor (not shown) may be incorporated within the sensing device 100 such that sound collisions may be detected.
Sound sensing devices include some form of microphone. If a warning sound signal 105 is emitted from the microphone, then any attempt to muffle the warning sound signal 105 will also muffle the sound being sensed. As stated with the emission of radio frequency (RF) signals below, a signal that is produced as an innate characteristic of a sensor is more difficult to disable. Substances can be made to vibrate via a number of mechanisms including pressure, light, heat, and sound. This vibration can create a sound with a particular pattern.
Interference caused by an audible warning signal 105 may be reduced or eliminated by placing the warning signal in a band outside of the range of human hearing, i.e., ultrasound. A specialized hearing aid may then be used to pick up the sound. It may be the case that in the future, hearing aids will become a common accessory. The term hearing aid usually refers to the reception of sound waves, whereas a speaker and headset usually refer to the conversion of electric signals into sound. From the perspective of the user, though, information is being conveyed in the form of sound and the term hearing aid will be used hereafter. A hearing aid can generally allow for hands free operation, including the frequently mentioned hands free driving. It can perform sound enhancement or sound cancellation. It can also pick up various forms of warning signals that involve a degree of personal preference. However, a signal such as a fire alarm signal would not be subject to someone's personal preferences.
The hearing aid can convert the out-of-band sound to an in-band sound including a spoken word representation of the signal. The sound signal can also be modulated with digital information which can be decoded by the hearing aid or by a receiver with some other form factor that has communication abilities. Thus a data modulated sound signal can act as a warning signal that is tailored to both the capabilities of humans and of machines. A hearing aid device can have the general functionality of converting data generated in various ways into sound, such as with the reception and conversion of the RF warning signal discussed below.
Light can also be used as a warning signal. A light signal may be emitted from the area of the lens of the camera, via the light generator 132 of the signal generation unit 130. Thus, any attempt to shield this warning signal will also result in covering the lens. Any distortion that this warning light will create will be filtered out of the image via signal processing. The light will be made noticeable but not annoying or dangerous to people. Therefore, a light sensor may be required in order to determine the adjustment to the intensity of the light warning signal such that a controlled relationship to ambient light is maintained. Similar to sound, the light may be modulated with data so that the signal is tailored to both the capabilities of humans and of machines. The light signal can also be emitted in a band out of the range of human perception.
An RF, digital or analog signal may be emitted to inform devices that monitor this warning signal. For example, the RF transmitter 131 in the signal generation unit 130 emits an RF signal 1051. Irrespective of whether the RF signal 1051 is digital or analog, the strength of the signal is proportional to the strength of the sensing. A digital signal may indicate various data items related to the sensing function, as stored in the ID data memory 170. These data items may include the type of sensor, the strength of sensor, the location of sensor and the sensing directional pattern. It is also possible that more personal/identification data may be revealed such as the serial number of the sensing device, the name of the sensing device, and contact information for the person responsible for the sensing device 100. These same data items may be used to modulate a sound or light warning signal generated by the signal generation unit 130. Some data items, (e.g., personal data), may only be revealed based on the sensing device 100 receiving an input, via the user interface 165, indicating the context or explicit instructions for operations. Such instructions may come from multiple parties. The combination of multiple instructions to form executable instructions will require a combining algorithm. This algorithm may require the use of artificial intelligence including the use of logic.
The information content of the RF signal may be contained in its analog characteristics. The amount of information contained may be less than that of a digital signal but may indicate the broad class of the sensing device 100 and may allow for enough uniqueness such that a particular device can be identified. The RF transmitter 131 in the signal generation unit 130 may be used to transmit the RF analog signal. This signal can generally be an analog signature but it may be possible to include digital information. By having the signature originate as an innate characteristic of the key component of a sensing device, makes it more difficult to disable the generation of the signature.
The warning signals 105 can be fully perceptible by humans and/or a monitor device can first receive the signal. The device which monitors the warning signal can be portable or stationary. It can be implanted, worn, or carried. It can convey the state of alarm to the user via various vibrations, various sounds (coded or language message), various light patterns, or can output a digital message to another device.
In one embodiment, the sensing device warning signals 105 may be coded by watermarks using the optional watermarking unit 175. These watermarks can represent data structures with sufficient information in them to uniquely identify the sensing device 100. The sensing device 100 is assigned watermarks based on the sensing functions of the sensing device 100. Each different type of sensing functionality has distinctive characteristics, (e.g., type of sensor, strength of sensor, location of sensor, sensing directional pattern, serial number of the sensing device, and the name, ID, and contact information for the person responsible for the sensing device, or the like). An administrative organization will have the responsibility to assign identifying information to each sensing functionality on each sensing device 100, as stored in the ID data memory 170.
Before the sensing function takes place, an appropriate watermarked warning signal may be emitted to provide sufficient time for the detecting device to react accordingly. Each detecting device has the capability to determine all known watermarks of sensing devices. The universe of sensing devices is either stored on the detecting device or special servers with adequate resources, (e.g., a computer processor unit (CPU), memory, cache, hard drive), to process look-up operations in a timely fashion. A search algorithm may be executed based on a number of schemes such as hashing, binary search and btree, or a combination thereof.
On the detecting device, there would be various converters that have different functions for solely transforming each watermark to an alert signal. An alert signal could be generated by a sound system, a vibrating system, a lighting system and/or just an interface that may be connected to other devices. Irrespective of the form of the alert system, it shall be customizable according to personal preferences. For example, a user may be able to choose a loud music selection when his/her picture is about to be taken and, if an unauthorized recording is about to take place, another kind of music could alert the user.
The function of warning that sensing is occurring or is about to occur is currently realized as a side effect of the existence or operation of the sensing device 100. Currently, sensing devices emit TEMPEST signals. A camera may send out a sound wave to perform auto-focusing. To enhance the amount of light on a subject, a flash may be used. The shutter of a camera can make a sound when light is admitted during the taking of a photo. The physical dimensions of a sensing device provide an indication of its presence. Video cameras have traditionally had a light visible to someone in front of the camera. The light informed the talent in front of multiple cameras which of the cameras is “live”. This feature has been carried over to camcorders.
In one embodiment of the present invention, a camera's flash device in a non-flash mode is used as the light generator 132 in the signal generation unit 130 to signal picture taking or camera existence via a sensing functionality support signal 106 that is generated, for example, by the main flash. Rather than generate a high lumens output as needed for illumination via the main flash signal 106, the light generator 132 can emit a low level light signal 1053, in or out of the optical band, at specific repetition rates, and/or at specific frequencies to be detectable by suitably designed equipment, but not necessarily detectable by people or animals. The light generator 132 may include an ancillary flash bulb which may be used to generate the low level light signal 1053, such that it is triggered each time a picture is taken. Modulation could be superimposed to the flash using Ferroelectric light valves, modulated mirror arrays, piezoelectric shutters, Kerr cells, Pockels cells, Bragg cells, liquid crystals or the Digital Light Processing™ (DLP) technology from Texas Instruments.
Thus, various sound and electromagnetic signal generating mechanisms may be used a means to alert a sensor to the existence of a recording device. This alert may occur whenever the sensing device 100 device is turned on via the activator, or when the sensing device 100 passes through a portal and detects an “identify yourself” signal, or when it receives a broadcast “identify yourself” signal. The mechanisms may be activated periodically, as long they are powered up, or whenever sensing functionality is initiated. A warning signal detection device may be worn by a person, or be incorporated in an area detection device. The warning signal 105 is detected and appropriate alerting takes place. The alert may be to a specific person, to a centralized monitor station, relayed to a remote station or person, or its occurrence merely recorded. Any of these alerts may cause secondary functions to occur. For example, a security guard may be summoned, an audio signal may be generated, a picture may be taken of the area to identify the recorder's carrier, a watermark broadcast may be generated, and so on.
Another method is to use a microphone or speaker to generate a background predictable and modulated signal. The use of a modulated signal with a specific training-like sequence will increase the probability of detection at the location surveillance system. For example, coding techniques, such as the one used by NASA for deep space communication, may be used.
All licensed recording devices will emit electromagnetic or ultra sound signals to indicate the activation of a recording activity to alert the monitoring device from the venue or a person of such an occurrence. These signals will indicate the “electronic serial number” of the device for identifying the device and providing a signal for locating the position of the activated recording device.
In one embodiment, a monitoring device network may be provided to compute the position of the recording device based on the measured signal arrival time differences (in a large open space). The signal will be transmitted at certain frequency with a certain range at a certain modulation scheme to minimize interference to other devices. It transmits periodically to minimize multiple device emission collisions and to save device power. Frequency hopping and a randomly selected period between signals could be considered for multiple-device handling capacity. The offending device is either static or moving slowly, (e.g., less than or equal to 3 km/h). For short distance position locating, ultra sound may be better because it requires less timing accuracy requirement. This is only possible if an ultra sound wave could be used to carrying a device code.
In order for the relative timing measurements to be sufficiently accurate, the various detectors must interpret the same timing point or points within the signal. This can be accomplished by encoding modulation transitions or data strings within the transmissions that are uniquely identifiable.
In an area where recording is not allowed, a monitoring device will be installed for detecting the presence and the location of offending devices that have activated unauthorized. At least three (3) monitoring devices in a guarded area are networked for measurement data gathering and synchronized for timing accuracy. Once the reliable measurement data is available, (the device ID is distinguishable from three monitoring devices), the position of the offending device can be located using a reversed time difference on signal arrival (TDOA) position locating method.
In the following description, the principle of the position locating method is illustrated. The Ricean fading model is assumed (only electromagnetic), i.e., the signal path between the monitoring device and the offending device is line of sight (LOS) dominated.
As shown in
At the time of the device-id reception, the offending device is in a position that is d1 distance from M1, d2 from M2 and d3 from M3. (Note d1, d2 and d3 are not measured, just a quantity used in the later equations).
The time differences between the arrival moments of two monitoring devices will then be (for simplicity only two shown) obtained at the monitoring network
T21=|t2−t1|, and
T31=|t3−t1|
The product of this time difference (Tij) and the speed of the transmitting media S, (e.g., electromagnetic wave 3×108 meters, sound waves 3.4×102 meters in air), represent a distance (Dij=S×Tij). This distance defines the curve (a hyperbola) that at any point of the curve the differences between the distances to the two monitoring devices is a constant, or
Dij=S×Tij=|dj−di|
For example in the above figure, the distance differences are determined as follows:
D21=S×T21 where T21=|t2−t1|, and
D31=S×T31 where T31=|t3−t1|.
Alternatively, the distance differences may be intuitively obtained as follows:
D21=d2−d1| and D31=d3−d1|.
Since all the monitoring devices and the offending device can be put in the same coordinate system, the following relationships may be established:
d1=√ (x0−x1)2+(y0−y1)2)};
d2=√ (x0−x2)2+(y0−y2)2)}; and
d3=√ (x0−x3)2+(y0−y3)2)}, where
D21=d2−d1=√{square root over ((x0−x2)2+(y0−y2)2)}−√{square root over ((x0−x1)2+(y0−y1)2)}; and
D31=d3−d1=√{square root over ((x0−x3)2+(y0−y3)2)}−√{square root over ((x0−x1)2+(y0−y1)2)}.
Since D21 and D31 are known quantities, and xi and yi, (where i=1, 2, 3), are also known quantities in the system, only x0 and y0 are the unknown coordinates of the offending device position to be solved.
Thus we have 2 unknowns and 2 equations (they are non-linear) for the computation of (x0, y0), the position of the offending device can be determined by carrying out the non-linear equations to linear computations, as is well known in the art.
In another embodiment, a monitoring device grid net covering positions on sensed signal energy may be used, as shown in
Each monitoring device 5051-50512 is installed to monitor a certain range of area 5101-51012. When the offending device 520 is activated, the nearest device 5052, 50512 senses the signal and alert the authority for appropriate actions by emitting respective warning signals 5152 and 51512.
According to the geography of the monitored area, the deployment of the monitoring device 505 and coverage grid may vary. The monitoring device 505 may also be more sophisticated to use the directional technology for pointing to the direction of the offending device 520.
Although the features and elements of the present invention are described in the preferred embodiments in particular combinations, each feature or element can be used alone without the other features and elements of the preferred embodiments or in various combinations with or without other features and elements of the present invention.
This application claims the benefit of U.S. Provisional Patent Application No. 60/633,527 filed Dec. 6, 2004, which is incorporated by reference as if fully set forth.
| Number | Date | Country | |
|---|---|---|---|
| 60633527 | Dec 2004 | US |
