The present invention relates generally to device for measuring 3D coordinates associated with an object in an environment, and a method of using such a device. In particular, embodiments of the invention relate to alignment of the device in an environment.
A 3D reflected wave measuring device can be used to identify the height of locations from which waves are reflected to the measuring device (and detected as reflected wave measurements). This may be achieved by subtracting a vertical coordinate representing a surface (with respect to which the height is to be referenced) from a vertical coordinate that corresponds to the reflection location.
However, for a 3D reflected wave measuring device having a three dimensional coordinate system the reflected wave measuring device may be installed in an orientation that is not aligned with an expected a frame of reference within the environment, which may result in incorrect determinations of heights.
The inventors have identified that 3D reflected wave measuring devices have applications in the field of fall detectors or security. As such, determining a correct height metric associated with an object of interest within an environment may be important.
Therefore, the present invention provides a device for measuring 3D coordinates associated with an object in an environment, and the method of using such a device, in accordance with the independent claims appended hereto. Further advantageous embodiments are also defined in the dependent claims appended hereto.
More specifically, the present invention provides a device for measuring 3D coordinates associated with an object in an environment, comprising: a 3D reflected wave measuring device having a 3D coordinate system for measuring 3D coordinates of an object in the environment, the 3D reflected wave measuring device having a first frame of reference; and an orientation sensing device to indicate an orientation of the 3D reflected wave measuring device in an environment for aligning the first frame of reference with a second frame of reference that is associated with the environment.
The orientation sensor advantageously enables the 3D reflected wave measuring device to be aligned correctly to a frame of reference within an environment. This in turn enables a more accurate measurement of 3D reflected wave data points from an object within the environment to be made.
The device may comprise a processor coupled to the orientation sensing device for receiving orientation data indicative of the orientation of the device.
In one embodiment, where a physical correction to the alignment of the device may be made, the device may be connected to one or more actuators for adjusting the orientation of the device in respective one or more axes, and wherein the processor drives the one or more actuators in response to the orientation data to align the first frame of reference to the second frame of reference.
In an alternative embodiment, the processor may output an indication of the orientation of the device in response to the orientation data, the indication for alerting a user at least one of audibly and visually. The indication may indicates at least one of an angular direction and an angle in one or more axes in which the device should be moved in order to align the first frame of reference to the second frame of reference; and/or whether or not the device is aligned.
In a further alternative embodiment, employing a mathematical method of aligning the device, the processor may be configured to align the first frame of reference with the second frame of reference by using the orientation data to compensate for a difference between the first frame of reference and the second frame of reference.
In this embodiment, the processor may configured to compensate for a difference between the first frame of reference and the second frame of reference by applying a rotation matrix to one or more axes of the first frame of reference and/or to one or more coordinates of 3D reflected wave measurements received by the 3D reflected wave measuring device.
The processor may be configured to determine a height metric of an object in the environment relative to the second frame of reference using 3D reflected wave measurements received by the 3D reflected wave measuring device.
The processor may receive height data, the height data indicating a height of the device in the environment, and wherein the processor is configured to determine the height metric of the object in the environment using the height data and the 3D reflected wave measurements received by the 3D reflected wave measuring device.
In any of the above-mentioned embodiments, the 3D reflected wave measuring device may be a radar device. The orientation sensing device may comprise at least one of a gyroscope, a 3 axis accelerometer or a level sensor.
The device may comprise a fall detector.
The present invention also provides a method of using a device for measuring 3D coordinates associated with an object in an environment, the device comprising a 3D reflected wave measuring device for measuring 3D coordinates of an object in the environment, the 3D reflected wave measuring device having a 3D coordinate system having a first frame of reference, and an orientation sensing device to indicate an orientation of the 3D reflected wave measuring device in an environment for aligning the first frame of reference with a second frame of reference that is associated with the environment, the method comprising: installing the device in the environment by mounting the device on a wall; and aligning the first frame of reference with the second frame of reference.
In the above method, the device may be connected to one or more actuators for adjusting the orientation of the device in respective one or more axes, and wherein the step of aligning the first frame of reference with the second frame of reference comprises driving the one or more actuators in response to the orientation data to align the first frame of reference to the second frame of reference.
In an alternative method, the method may comprise outputting an indication of the orientation of the device in response to the orientation data, the indication for alerting a user at least one of audibly and visually. The indication may indicate: at least one of an angular direction and an angle in one or more axes in which the device should be moved in order to align the first frame of reference to the second frame of reference; and/or whether or not the device is aligned.
In the above methods, the step of aligning may be performed one or more times after installation of the device.
In a further alternative method, the step of aligning the first frame of reference with the second frame of reference comprises compensating for a difference between the first frame of reference and the second frame of reference using the orientation data.
In this further alternative method, compensating for a difference between the first frame of reference and the second frame of reference may comprise applying a rotation matrix to one or more axes of the first frame of reference and/or to one or more coordinates of 3D reflected wave measurements received by the 3D reflected wave measuring device.
In any of the above-mentioned methods, the method may further comprise determining a height metric of an object in the environment relative to the second frame of reference using 3D reflected wave measurements received by the 3D reflected wave measuring device.
The installed height of the device on the wall may be stored in memory in the device as height data, and the method comprises determining a height metric of the object in the environment relative to the second frame of reference using the height data and 3D reflected wave measurements received by the 3D reflected wave measuring device.
The step of aligning the first frame of reference with the second frame of reference may be performed at least one of one or more times after installation, or prior to each determination of the vertical displacement of an object in the environment
In any of the methods, the 3D reflected wave measuring device may be a radar device. The orientation sensing device may comprise at least one of a gyroscope, a 3 axis accelerometer or a level sensor.
In any of the methods, the device may comprise a fall detector.
Any instructions may be provided on one or more carriers. For example there may be one or more non-transient memories, e.g. a EEPROM (e.g. a flash memory) a disk, CD- or DVD-ROM, programmed memory such as read-only memory (e.g. for Firmware), one or more transient memories (e.g. RAM), and/or a data carrier(s) such as an optical or electrical signal carrier. The memory/memories may be integrated into a corresponding processing chip and/or separate to the chip. Code (and/or data) to implement embodiments of the present disclosure may comprise source, object or executable code in a conventional programming language (interpreted or compiled) such as C, or assembly code, code for setting up or controlling an ASIC (Application Specific Integrated Circuit) or FPGA (Field Programmable Gate Array), or code for a hardware description language.
These and other aspects will be apparent from the embodiments described in the following. The scope of the present disclosure is not intended to be limited by this summary nor to implementations that necessarily solve any or all of the disadvantages noted.
The present invention will now be described, by way of example only, and with reference to the accompanying drawings, in which:
In the following detailed description, reference is made to the accompanying drawings that form a part hereof, and in which is shown by way of illustration specific embodiments in which the inventive subject matter may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice them, and it is to be understood that other embodiments may be utilized, and that structural, logical, and electrical changes may be made without departing from the scope of the inventive subject matter. Such embodiments of the inventive subject matter may be referred to, individually and/or collectively, herein by the term “invention” merely for convenience and without intending to voluntarily limit the scope of this application to any single invention or inventive concept if more than one is in fact disclosed.
The following description is, therefore, not to be taken in a limited sense, and the scope of the inventive subject matter is defined by the appended claims and their equivalents. In the following embodiments, like components are labelled with like reference numerals.
In the following embodiments, the term data store or memory is intended to encompass any computer readable storage medium and/or device (or collection of data storage mediums and/or devices). Examples of data stores include, but are not limited to, optical disks (e.g., CD-ROM, DVD-ROM, etc.), magnetic disks (e.g., hard disks, floppy disks, etc.), memory circuits (e.g., EEPROM, solid state drives, random-access memory (RAM), etc.), and/or the like.
As used herein, except wherein the context requires otherwise, the terms “comprises”, “includes”, “has” and grammatical variants of these terms, are not intended to be exhaustive. They are intended to allow for the possibility of further additives, components, integers or steps.
The functions or algorithms described herein are implemented in hardware, software or a combination of software and hardware in one or more embodiments. The software comprises computer executable instructions stored on computer readable carrier media such as memory or other type of storage devices. Further, described functions may correspond to modules, which may be software, hardware, firmware, or any combination thereof. Multiple functions are performed in one or more modules as desired, and the embodiments described are merely examples. The software is executed on a digital signal processor, ASIC, microprocessor, or other type of processor.
Specific embodiments will now be described with reference to the drawings.
As shown in
In some embodiments, the CPU 202 is configured to detect the presence of a person in the monitored space 104, and if a person is present, classify the state or postural position (standing, sitting or having fallen) of the person based on an output of the active reflected wave detector 206.
The active reflected wave detector 206 may operate in accordance with one of various reflected wave technologies.
Preferably, the active reflected wave detector 206 is a radar sensor. The radar sensor 206 may use millimeter wave (mmWave) sensing technology. The radar is, in some embodiments, a continuous-wave radar, such as frequency modulated continuous wave (FMCW) technology. Such a chip with such technology may be, for example, Texas Instruments Inc. part number IWR6843. The radar may operate in microwave frequencies, e.g. in some embodiments a carrier wave in the range of 1-100 GHz (76-81 Ghz or 57-64 GHz in some embodiments), and/or radio waves in the 300 MHz to 300 GHz range, and/or millimeter waves in the 30 GHz to 300 GHz range. In some embodiments, the radar has a bandwidth of at least 1 GHz. The active reflected wave detector 206 may comprise antennas for both emitting waves and for receiving reflections of the emitted waves, and in some embodiment different antennas may be used for the emitting compared with the receiving.
The active reflected wave detector 206 is not limited to being a radar sensor, and in other embodiments, the active reflected wave detector 206 is a lidar sensor, or a sonar sensor.
The active reflected wave detector 206 being a radar sensor is advantageous over other reflected wave technologies in that radar signals can transmit through some materials, e.g. wood or plastic, but not others—notably water which is important because humans are mostly water. This means that the radar can potentially “see” a person in the environment 100 even if they are behind such an object. This is not the case for sonar.
In embodiments, the active reflected wave detector 206 is used to determine whether the person is in a posture that may be relate to them standing, sitting, or having fallen. This may be achieved for example by detecting a height associated with a certain location on their body, e.g. a location above their legs.
In operation, the active reflected wave detector 206 performs one or more reflected wave measurements at a given moment of time, and over time these reflected wave measurements can be correlated by the CPU 202 with the presence of a person and/or a state of the person and/or a condition of the person. In the context of the present disclosure, the state of the person may be a characterization of the person based on a momentary assessment. For example, a classification passed on their position (e.g. in a location in respect to the floor and in a configuration which are consistent or inconsistent with having fallen) and/or their kinematics (e.g. whether they have a velocity that is consistent or inconsistent with them having fallen, or having fallen possibly being immobile). In the context of the present disclosure, the condition of the person may comprise a determination of an aspect of the person's health or physical predicament, for example whether they are in a fall condition whereby they have fallen and are substantially immobile, such that they may not be able (physically and/or emotionally) to get to a phone to call for help. In some embodiments this involves an assessment of the person's status over time, such as in the order or 30-60 seconds. However, the condition of the person may in some contexts be synonymous with the status of the person. For example, by determining that the person is in a safe supported state or a standing state, it may be concluded that the person is not currently in a fall condition, whereby they are on the floor and potentially unable to seek help. It may additionally or alternatively be concluded that they are in a resting condition because of their status being determined to be in a safe supported state, e.g. lying on a bed. In another example their condition may be classified as active and/or mobile based on a determination of a walking status.
For each reflected wave measurement, for a specific time in a series of time-spaced reflective wave measurements, the reflective wave measurement may include a set of one or more measurement points that make up a “point cloud”. Each point 302 in the point cloud may be defined by a 3-dimensional spatial position from which a reflection was received, and defining a peak reflection value, and a doppler value from that spatial position. Thus, a measurement received from a reflective object may be defined by a single point, or a cluster of points from different positions on the object, depending on its size. In some embodiments the point cloud is prefiltered to exclude static points, for example by using a moving target indication as is known in the art.
When a cluster of measurement points are received from an object in the environment 100, a location of a particular part/point on the object or a portion of the object, e.g. its centre, may be determined by the CPU 202 from the cluster of measurement point positions having regard to the intensity or magnitude of the reflections (e.g. a centre location comprising an average of the locations of the reflections weighted by their intensity or magnitude). As illustrated in
The object's centre or portion's centre may be a weighted centre of the measurement points. The locations may be weighted according to an Radar Cross Section (RCS) estimate of each measurement point, where for each measurement point the RCS estimate may be calculated as a constant (which may be determined empirically for the reflected wave detector 206) multiplied by the signal to noise ratio for the measurement divided by R4, where R is the distance from the reflected wave detector 206 antenna configuration to the position corresponding to the measurement point. In other embodiments, the RCS may be calculated as a constant multiplied by the signal for the measurement divided by R4. This may be the case, for example, if the noise is constant or may be treated as though it were constant. Regardless, the received radar reflections in the exemplary embodiments described herein may be considered as an intensity value, such as an absolute value of the amplitude of a received radar signal.
In any case, the weighted centre, WC, of the measurement points for an object may be calculated for each dimension as:
N is the number of measurement points for the object;
Wn is the RCS estimate for the nth measurement point; and
Pn is the location (e.g. its coordinate) for the nth measurement point in that dimension.
The CPU 202 may determine a height metric associated with at least one measurement of a reflection from the person conveyed in the output of the active reflected wave detector 206 and compare the height metric to at least one threshold.
The height metric may be a height of a weighted centre of the measurement points of a body or part thereof (where each measurement is weighted by the RCS estimation), and the CPU 202 may compare this height metric to a threshold distance, D, from the floor (e.g. 30 cm) and determine that the person in the environment is in a standing (non-fall) position if the height metric exceeds the threshold distance, this is illustrated in
The height metric is not limited to being a height of a weighted centre of the measurement points of the person's body or part thereof. In another example, the height metric may be a maximum height of all of the height measurements associated with the person's body or part thereof. In another example, the height metric may be an average height (e.g. median z value) of all of the height measurements of the person's body or part thereof. In the case of using a weighted centre or average height, the “part thereof” may beneficially be a part of the body that is above the person's legs to more confidently distinguish between fall and non-fall positions.
If the height metric (e.g. weighted centre, average height and/or maximum height) is within (less than) the threshold distance, D, from the floor (e.g. 30 cm), the CPU 202 may determine that the person in the environment is in a fall position, this is illustrated in
When the device is a fall detector, the detector could be a part of the device or a module of the code or part of its functionality.
As can be seen from the above methods, a reliable and accurate determination of the value for a height metric from the Active Reflective Wave Detector is important if a reliable determination of a position or posture of the person is to be made based on data received from the active reflected wave detector. That position may, for example, be a related to postural information indicative of a person standing position, sitting position or having fallen over. As mentioned above, such data may be in the form of sets of points defined by 3D coordinates and intensities or magnitudes of the reflective wave for each of the points.
The device may or may not be installed in an orientation and position such that its active reflected wave detector is not aligned with an expected frame of reference (for example a horizontal plane lying a height Hi above a point on a surface such as the floor, which in some embodiments is a horizontal surface). Thus, the expected frame of reference may have an axis x2,y2,z2, which is associated with an environment. In particular, x2 and y2 are horizontal and z2 is vertical, where the x2-y2 plane lies a distance H above the floor. A horizontal floor may thus lie in a plane defined by the equation z2=−H. The device's active reflected wave detector has an actual frame of reference (i.e. how it is actually installed) having an axis x1, y1 and z1, in which the z axis is only nominally (but not necessarily in practice) vertical. If the nominal axis (x2, y2, z2) is not aligned with the expected frame of reference (x1,y1,z1), both in terms of orientation (or at least the orientation of the z axis) and position (or at least the height position), then the device may incorrectly determine a height metric corresponding to coordinate associated with a measured reflect point. This is because the coordinate is measured with respect to the nominal frame of reference, but the height metric is with respect to the actual environment (i.e. a frame of reference of the environment; also referred to herein as an expected frame of reference).
Of course, misalignments may also occur in the x-y plane (where the device rotates about the z axis). However, misalignment by rotating in the z axis would not influence a vertical measurement (but simply misalign the useful field of view about the x-y axis). Therefore, such misalignments may be ignored in some embodiments.
In the case of the in
As will be appreciated, in addition to or instead of using the height D1 of the point 306 as the height metric, the height metric used for the comparison with the threshold may be associated based on one or more points associated with the person 106. For example, the height metric could correspond to a spatial position of a weighted centre of the measurements (308 in
The embodiments disclosed herein facilitate or cause alignment of the 3D reflected wave measuring device (i.e. the active reflected wave detector 206) with a frame of reference of the environment. This is done using data from the orientation sensing device 210, which, in practice may be at least one of a gyroscope, a 3 axis accelerometer and a level sensor. In some embodiments the orientation sensing device 210 be 6-axis orientation sensing devices, the 6 axis consisting of 3 from a gyroscope and 3 from an accelerometer.
There are two main implementations of an alignment correction of the device in order that the device is correctly aligned to a frame of reference within the environment: 1) physical alignment; and 2) mathematical alignment. In either case, the purpose of the alignment correction is to re-align the orientation of the z-axis of the device with the z axis of the frame of reference in the environment, where misalignment has occurred because of a rotation of the device about the x and/or y axes. The alignment correction may also, or instead, be performed to re-align the x and/or y axes in the case where there has been a rotation of the device about the z axis. However, in some embodiments, misalignment in the x and/or y axes as a result of rotation of the device about the z axis is ignored.
The physical alignment is discussed first.
In this embodiment, and in its most fundamental configuration, the device utilises the orientation sensor 210 for the processor to provide feedback for a user to manually adjust a position of the device when installing the device.
Most simply the indication could be provided by one or more level meters (or representations of level meters). Two level meters respectively lying in perpendicular planes are preferable. As such, the user would adjust the pan and tilt of the device during installation in order to achieve the desired position as indicated by the level meters. The indication could in its most basic form be a binary output of level (to within a predefined accuracy) or not level.
In a related embodiment, the processor may give an indication when out of alignment of an extent and preferably also direction of the misalignment so that a person could easily align the device as needed. The indication may be provided by a visual and/or audio means on the device itself, or communicated to another device, for example a smartphone, tablet or computer.
In alternative embodiment where physical re-alignment occurs, the processor 202 may be coupled to drive actuators that control the position (pan and tilt) of the device. A closed loop control system may be implemented where the processor utilises the data from the orientation sensing device 210 in order to drive the actuators to move the device to minimise the misalignment between the device and the frame of reference (for example a horizontal plane lying at height Hi above a point on a surface such as the floor, which in some embodiments is a horizontal surface).
The alignment of the device may be performed one or more times after installation of the device. The device may alert the user periodically if the position of the device has become misaligned with the intended frame of reference. With the motorised embodiment, the device may continuously monitor its orientation relative to the intended frame of reference.
Now discussed is the mathematical method of aligning the frame of reference of the orientation sensing device with the frame of reference of the environment.
In this embodiment, which is the preferred embodiment, the device is not physically reoriented, but rather, the angular difference (at least with respect to the z axis) between the second frame of reference and the first frame of reference (of the orientation sensing device) is compensated for so that the measurements made with respect to the first frame of reference are interpreted with respect to the second frame of reference.
The compensation can be achieved, for example using a rotation matrix. For example, the rotation matrix can be applied to either coordinates of respective measured wave reflections from the 3D reflected wave device, or to the first frame of reference.
Rotation matrices are well known in the in the operation of 3D coordinate systems.
Once the rotation matrix has been applied, the measurements made by the 3D reflected wave device are aligned to the intended frame of reference. Height metric measurements made based on the reflected wave measurements therefore provides an accurate determination of the height metric of an object, irrespective of the physical orientation of the device once installed.
The compensation may occur periodically, or before each determination of a height metric. It is also noted that the compensation, may be applied firstly on the raw data being received from the active reflective wave detector before any processing of the data to determine a weighted centre point is performed. Alternatively, the raw data may be processed to determine a weighted centre point (or other spatial location based on the raw data), and then compensation may be applied.
In either of the above implementations of the alignment, installation height data (H) may be stored in the memory of the device. The installation height data may comprise a height value at which the device is instructed to be installed on a wall. Alternatively installation height data may be data that is input by the installer of the device once the device has been installed. Furthermore, the installation height data may be data that is measured using the 3D reflected wave detector 206 (once correctly aligned with the intended frame of reference), for example based on a measured z-displacement from one or more identifiable reflection points on the floor, for example by a retroreflector placed on, or moving along, the floor.
As such, the vertical displacement of the 3D measurements with respect to the second frame of reference may be determined based on a vertical difference between the reflected data from the object and the installation height data.
Where it is referred herein to “aligning the frame of reference of the 3D reflected wave measuring device”, this means aligning the frame of reference of the 3D reflected wave measuring device 206. It is the orientation sensing device 210 that provides the orientation data. However, the orientation sensing device package has a fixed and known positional relationship with respect to the 3D reflected wave measuring device, and so the orientation of the 3D reflected wave measuring device 206 may be determined with respect to the orientation sensing device 210 to that any differences between their respective orientations may be calibrated out (e.g. by mathematical alignment such as by using a rotation matrix, for example as described herein). Such a calibration may be performed in during manufacture of the device 102 for example, prior to its installation for use in the environment. It will be appreciated, however, that the aligning the frame of reference of the 3D reflected wave measuring device 206 to the frame of reference associated with the environment may be equivalently performed by aligning the orientation of the sensor sensing device 210 with the frame of reference of the environment, either physically or by mathematical manipulation, in conjunction with aligning the orientation of the sensor sensing device 210 with the frame of reference of the environment 3D reflected wave measuring device 206, either physically or by mathematical manipulation.
The phrase “A and/or B”, “at least one of A and B” and “at least one of A or B” are each intended to include the following options, unless the contexts clearly requires otherwise: “A”, “B”, “at least A”, “at least B”, “at least one of A”, “at least one of B”, and all combinations thereof relating A with B, for example, “at least one of A and at least one of B”. Where used in a claim, the feature of the claim should be taken to be fulfilled if any one or more of those options is satisfied. If in a particular context any of those options are nonsensical or result in a duplicated option then, in that particular context, only the nonsensical option or duplicated option may be disregarded.
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.
Number | Date | Country | Kind |
---|---|---|---|
2008033.9 | May 2020 | GB | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/IL2021/050593 | 5/21/2021 | WO |