Patent Application
20230323723
The present disclosure relates to an automatic door system and more specifically, to an automatic door system that includes at least one radar sensor to sense a door area.
Business buildings often control physical access using an automatic door. The automatic door is configured to open and close based on measurements from one or more sensors at, or around, the automatic door. The automatic door opening and closing typically conforms to an industry standard for safety to prevent injury (e.g., by a door closing on a person). As a result, an automatic door may include a first sensor (i.e., activation sensor) configured to detect a person approaching (or leaving) a door and a second sensor (i.e., presence sensor) configured to detect when a person is at the door (e.g., in the doorway). The sensing capability of the automatic door is typically limited to detecting (i) a bidirectional movement (e.g., to/from doorway) and (ii) a presence (e.g., in the doorway). Additional sensing capabilities could enhance door control.
In at least one aspect, the present disclosure generally describes an automatic door system. The automatic door system includes a door that is controllable to at least an open position and a closed position. The automatic door system further includes a plurality of radar sensors. Each radar sensor includes a millimeter wave (mmW) radar configured to generate mmW radar data corresponding to a door area. Each radar sensor further includes an inertial measurement unit (IMU) that is configured to generate IMU data corresponding to an orientation of the mmW radar with respect to a reference coordinate system. The automatic door system further includes a door control unit (DCU) that is communicatively coupled to the door assembly and the plurality of radar sensors. The DCU is configured (e.g., by software instructions) to control the door based on combined mmW radar data, which includes mmW radar data from each of the plurality of radar sensors.
In another aspect, the present disclosure generally describes a radar sensor for an automatic door, the radar sensor includes a mmW radar that is configured to generate mmW radar data corresponding to a door area. The radar sensor further includes an IMU that is configured to generate IMU data corresponding to an orientation of the mmW radar. The radar sensor further includes a sensor control unit (SCU) that is configured by software instructions to receive the mmW radar data and the IMU data; register the mmW radar data to a reference coordinate system; and transmit the registered mmW radar data to a DCU of the automatic door.
In another aspect, the present disclosure generally describes a method for controlling an automatic door. The method includes collecting first mmW radar data from a first mmW radar and collecting first IMU data, which corresponds to a first orientation of the first mmW radar, from a first IMU. The method further includes collecting second mmW radar data from a second mmW radar and collecting second IMU data, which corresponds to a second orientation of the second mmW radar, from a second IMU. The method further includes registering the first mmW radar data and the second mmW radar data to a reference coordinate system. The method further includes combining the first mmW radar data and the second mmW radar data to obtain combined mmW radar data. The method further includes analyzing the combined mmW radar data to track one or more targets in a combined door area, which corresponds to the combined mmW radar data, and controlling the automatic door based on the tracking of the one or more targets in the combined door area.
The foregoing illustrative summary, as well as other exemplary objectives and/or advantages of the disclosure, and the manner in which the same are accomplished, are further explained within the following detailed description and its accompanying drawings.
The components in the drawings are not necessarily to scale relative to each other. Like reference numerals designate corresponding parts throughout the several views.
Conventionally, two sensors have functioned to activate the open/close function of an automatic door (i.e., activation sensor) and to prevent the automatic door from closing on an object or entity in the doorway (i.e., presence sensor). An object may be inanimate, such as the floor, a wall, or a piece of furniture, while an entity may be animate, such as a person, animal, or otherwise moving object. Objects and entities may be referred to generally as targets. For example, a microwave radar sensor may be configured to detect motion of a target in an activation zone of a door area. While a conventional microwave radar sensor may be configured to determine if a movement is towards the door or away from the door, it may not be configured to detect other directions of movements, differentiate between multiple movements, or detect stationary objects/people. For example, the microwave radar sensor may not be able to detect when a customer stands motionless in the doorway. As result, most conventional doors utilize an infrared (IR) sensor that can detect the presence of a person or object in a presence zone (i.e., safe zone) aligned with the doorway even when the person is motionless. The IR sensor, however, may not be sensitive enough to differentiate between stationary objects and stationary people. Further, the IR presence sensor is not configured to provide any understanding of where an object/person is located other than within a presence zone of a door area. Automatic door control could be greatly enhanced if the sensors were more sensitive and/or could gather more information.
An automatic door may use a video camera as the sensor for door operation. A video camera can be configured to capture video in the door area at the automatic door, and image processing may be applied to the captured video in order detect motion and presence for door operation. This image approach to door control, however, has technical problems. For example, reduced light and/or direct/reflected sunlight can reduce image quality, which can affect the image processing necessary for door control. Further, weather conditions (e.g., fog, snow, rain) can also affect image quality in exterior installations. Accordingly, a video camera may not be practical for some door installations. Further, the resolution of the video camera may not be fine enough to capture small movements and therefor can lack sensitivity. For example, there may be risk of classifying a stationary person as a stationary object. Thus, a need exists for a door sensor that can facilitate the detection, classification, and tracking of multiple targets in a door area that is also versatile enough to be used in various environmental conditions.
To address at least the technical problems presented above, the present disclosure describes an automatic door system (i.e., automatic door) that includes a millimeter wave (mmW) radar sensor (or sensors) that can provide data to detect, classify, and track multiple targets in a door area in various environmental conditions. Millimeter wave (mmW), as described herein, includes any frequency of the electromagnetic spectrum having a wavelength that is on the order of a millimeter (e.g., 1 mm≤λ≤10 mm). For example, any frequency in a range of approximately 30 gigahertz (GHz) to approximately 300 GHz can be used for millimeter wave radar sensing. The disclosure recognizes that some advantages may be achieved by sensing frequencies higher than conventional door microwave radar sensors, which may have less resolution than mmW radar sensors, and door optical sensors, which can be affected more by environmental conditions than mmW radar sensors. In particular, mmW radar sensors may operate in a portion of the electromagnetic spectrum ranging from approximately 40 GHz to approximately 75 GHz (i.e., V band), with a frequency (or frequencies) around 60 GHz (e.g., 60 GHz≤f≤64 GHz) being possible for the implementations described herein.
Millimeter wave radar can measure position and velocity information at very high resolutions. Accordingly, multiple objects/entities in a door area can be detected simultaneously, and stationary objects may be determined as living based on subtle movements (e.g., breathing). Further, mmW radar is immune to lighting conditions and environmental conditions that negatively affect video (i.e., vision) sensors. Data (e.g., position, velocity) collected by a mmW radars can be processed to classify, track, and predict movement for multiple targets in a door area, which can enhance door control in a variety of ways.
The sensor system 120 may include one or more sensors that are configured to sense a door area 130. The door area 130 may be divided into zones based on the resulting door operation caused by an object/entity (i.e., target) detected in the zone. For example, the door area 130 may include an activation zone 131 (i.e., shown as dotted) and a presence zone 132 (i.e., shown as hashed). Sensing a movement of an entity (e.g., approaching the door) in the activation zone 131 may trigger the automatic door to perform an open/close sequence and sensing an object/entity in the presence zone 132 may open the door (e.g., open a closing door) or suspend an open door from closing.
An advantage of the disclosed mmW radar sensor system approach is that the zones are all sensed using the same technology, as opposed to different sensor types for different zones. Accordingly, each zone's size, shape, and location may be changed (e.g., by software/firmware) without adjustment to the sensor system. Additionally, new zones may be added and/or existing zones may be removed. For example, a living-object zone may be designated in the door area 130 to help tracking and classification. In other words, the disclosed automatic door includes configurable zones for door control. Accordingly, it may be understood that the door area 130 and zones shown in
Another advantage of the disclosed mmW radar sensor system approach is that multiple mmW radar sensors may be used to extend and/or reshape the door area 130. For example, the use of multiple mmW radar sensors can extend the presence zone 132 over the threshold 115 and/or expand the activation zone 131 so that a width of the activation zone 131 matches or exceeds the width of the automatic door 100. In other words, the disclosed automatic door system has a combined door area corresponding to the coverage of multiple mmW radar sensors in a door sensor system, where the combined door area is larger than a door area that could be sensed by any one of the multiple mmW radar sensors alone.
The door area 130 may be located on the interior or the exterior of a building, room, or hallway that is accessed by the automatic door 100. In some implementations, the automatic door 100 may include an interior sensor system to sense an interior door area and an exterior sensor system to sense an exterior door area. The interior sensor system and the exterior sensor system can operate identically. Accordingly, the present disclosure will discuss one sensor system for the automatic door, but it should be understood that the principles of the disclosure can be adapted to automatic doors with multiple sensor systems, such as described for interior and exterior.
The door interface 260 may include processing and communication circuitry to enable the DCU 210 to communicate with a network 261 (e.g., internet) or a computing device 262 (e.g., tablet computer). For example, a technician operating the computing device 262 may access the automatic door via communication with the door interface using a wireless communication protocol (e.g., Bluetooth). This access may be used to test, calibrate, or otherwise operate the subsystems of the automatic door system. For security, the door interface 260 can include physical or software security components to prevent unwanted access. In a possible implementation, the DCU 210 may store data collected by the automatic door system 200 and may transmit the data, or messages related to the data, to another computer coupled to the automatic door system 200 via the network 261 (e.g., cloud). In another possible implementation, the processing of the DCU 210 may be supplemented or replaced by a processor, or processors, coupled to the automatic door system via the door interface 260. For example, analysis of door sensor data may be carried out using a cloud computing approach. In another possible implementation, the door interface 260 may be configured to communicate with other automatic door systems. For example, it may be possible for classification from a first door to be shared with a second door to aid in tracking an entity. The door interface may communicate wirelessly via a variety of wireless technologies, including (but not limited to) WiFi, 4G/5G wireless, Bluetooth, and ultra-wideband (UWB).
The automatic door system 200 further includes a door sensor system 220. As discussed, the door sensor system 220 can include one or more mmW radar sensors (i.e., mmW sensors, radar sensors) configured to transmit an electromagnetic signal in a millimeter wave frequency band (e.g., 60-64 Gigahertz (GHz)) into door areas and to receive reflected electromagnetic signals back from objects/entities in the door areas. Based on these reflected signals, the one or more radar sensors can detect objects/entities and their movement. For example, each radar sensor may generate mmW radar data (i.e., radar data) that corresponds to the targets (i.e., entities/objects) measured in its door area (i.e., in its field-of-view).
The automatic door system 200 further includes a door assembly 240. The door assembly 240 can include an automatic door (i.e., door 245). The door 245 can be one of a variety of types including (but not limited to) sliding, folding, and swinging and can include both moving and non-moving door panels. The door 245 can be configured by a door motor (i.e., motor 246), or other device configured to apply hydraulic or electromechanical force, into an open position, a closed position, or one or more positions therebetween (i.e., partially open). The door 245 may be mounted to door panel carriers that slide or roll in the header 110 to open/close. The door assembly 240 can further include a door operator 230. The door operator 230 can include a controller and circuitry to drive the motor 246 based on signals from the DCU 210. The door assembly 240 can further include a door encoder 250 that is coupled to the door and configured to measure the position of the door. The measured position of the door can be fed back to the DCU 210 to provide a closed loop control of the door 245.
The implementation shown in
The radar sensor 400 further includes an inertial measurement unit (i.e., IMU 510). The IMU 510 includes a combination of sensors to measure the orientation of the IMU (e.g., with respect to the earth). The combination of sensors can include accelerometers, a gravitometer, gyroscopes, and/or a magnetometer. The IMU is mechanically coupled (to the mmW radar 600 so that IMU data returned by the IMU 510 corresponds to the orientation of the mmW radar 600.
The radar sensor 400 further includes a sensor control unit (SCU 520). The SCU 520 can be configured to communicate radar data and IMU data with the DCU and/or with other sensors, as described above. In a possible implementation, the SCU 520 can be configured to determine the orientation of the mmW radar from the IMU data and register the mmW radar data based on the determined orientation. The SCU 520 may include memory 530 on which to store (e.g., buffer) radar/IMU data and/or software for radar/IMU data processing.
The configuration shown in
The transmitter 610 includes a synthesizer to produce a transmit signal (i.e., Tx) that changes frequency over time according to a ramp profile. For example, the transmitter 610 can include a ramp generator 611 to repeatedly generate a voltage that steadily increases (or decreases) with time for a period. The changing (i.e., ramp) voltage may then be coupled to an input of a voltage-controlled oscillator (i.e., VCO 612) or other oscillator that has a mmW frequency that is proportional to the input voltage. The resulting frequency modulated (i.e., chirped) transmit signal (i.e., Tx) is then coupled to a field of view via a transmit antenna 613. The mmW can be referred to as a frequency modulated continuous wave (FMCW) radar.
Each receiver 620 of the mmW radar 600 includes an antenna 621 configured to receive signals reflected from an object in the field of view. A range to an object can cause a delay between the transmitted signal (i.e., Tx) and the received signal (i.e., Rx), which for a chirped signal, corresponds to a frequency difference. As a result, the processor 601 of the mmW radar 600 can estimate distance to a target (i.e., range) based on the frequency difference between the transmitted signal and the received signal. Accordingly, each receiver of the mmW radar 600 further includes a mixer 622 configured to determine the frequency difference between the received signal (Rx) and the transmit signal (Tx). Each receiver may further include a filter 623 (e.g., low-pass filter, band-pass filter). The filter may help to isolate an intermediate frequency (i.e., IF) signal from harmonics resulting from the mixing process. The IF signal can include multiple tones, where the frequency of each tone is proportional to the distance (i.e., range) of a corresponding target (i.e., object). Each receiver may further include an analog to digital converter (i.e., ADC 624) that is configured to digitize the IF signal. The digitized signal may then be coupled to the processor 601 where it can be processed (e.g., transformed) in order to generate the mmW radar data which includes the range information to each target. For example, a Fourier transform (e.g., fast Fourier transform (FFT)) may be performed on the ADC data to obtain a frequency domain representation (i.e., spectrum) of the IF signal. The peaks of the spectrum can correspond to the range of the objects. Accordingly, the processor may be configured to identify range of each target based on peak detection of a spectrum. The mmW radar data may include a range to an object in the field of view.
The mmW radar data may further include an angle 635 of the received signal from the object with respect to the mmW radar. A coordinate system 630 of the mmW radar may be based on a direction of the antenna array of the receiver and the angle 635 may be relative to an axis of the coordinate system. Because of the angle 635, the received signal at each of the receivers will be shifted in phase with respect to each other. In other words, the range from the object to each of the receiver antennas is slightly different. To determine the angle, the processor 601 may be configured to determine the phase of corresponding spectral peaks at each receiver to determine the phase shifts between antennas. In other words, the processor may be configured to determine how a phase of a spectral peak changes over position to determine a spatial phase shift. The processor 601 may then be configured to relate the spatial phase shift to an angle to the target.
The processor 601 may be further configured to combine the range and the angle of mmW radar data to determine the position (i.e., location) of an object in the field of view (i.e., door area). When multiple objects/entities are in the field of view the processor 601 may be configured to determine positions of each. Multiple object positions may be resolved based on the angular resolution of the mmW radar. The angular resolution can correspond to the number of transmitters/receivers in the mmW radar.
The mmW radar data may further include a velocity measured for a moving object in the field of view. In order to measure velocity (i.e., speed), the processor 601 may be configured to estimate how the phase shift of a spectral peak changes over time. In other words, the processor may be configured to determine how a phase of a spectral peak changes over time to determine a temporal phase shift. The temporal phase shift may correspond to the velocity. Objects at the same range (i.e., having the same spectral peak) can be resolved using a frame of chirp signals and then performing a second transform (i.e., doppler-FFT) to resolve the objects by peaks corresponding to their different temporal phase shifts (i.e., velocity).
Each mmW radar may generate mmW radar data based on a coordinate system corresponding to the mmW radar. In other words, each mmW radar may generate mmW radar data relative to its own coordinate system. In an automatic door system, it may be desirable to use multiple mmW radars to expand a sensed area. Each coordinate system of the multiple mmW radars may be different. To compared and/or combined mmW radar data from different mmW radars, it may be necessary to convert the mmW radar data from the mmW radars to a common (i.e., reference) coordinate system. For example, mmW radar data from each mmW radar may be converted to a reference coordinate system (e.g., corresponding to the door). In another possible implementation mmW radar data from a first mmW radar may be converted to the coordinate system of a second mmW radar. Comparing and/or combining mmW radar data from multiple mmW radars may include a coordinate transformation (i.e., conversion) of the mmW radar data to a common (i.e., reference) coordinate system. The converted mmW radar data may be referred to as registered mmW radar data.
Returning to
When mmW radar data is converted to the reference coordinate system (i.e., common coordinate system) it is said to be registered. For example, mmW radar data from a first mmW may be registered to a door coordinate system and mmW data from a second mmW radar may be registered to the door coordinate system. The registered mmW radar data can be combined to formed combined radar data. The combined radar data can be analyzed to locate a target in a combined door area.
Registering first mmW radar data (from a first mmW radar) and second mmW radar data (from a second mmW radar) to a reference coordinate system can include determining a first coordinate conversion between a first coordinate system (of the first mmW radar) and the reference coordinate system and a second coordinate conversion between a second coordinate system (of the second mmW radar) and the reference coordinate system. Next, the first mmW radar data may be modified according to the first coordinate conversion to obtain registered first mmW radar data, and the second mmW radar data may be modified according to the second coordinate conversion to obtain registered second mmW radar data. The registered first mmW radar data and the registered second mmW radar data can then be combined to obtain combined mmW radar data.
The reference coordinate system may be selected variously. In a possible implementation the reference coordinate system is a coordinate system aligned with the physical shape of the automatic door. For example, a horizontal axis (e.g., x-axis) of a reference coordinate system can be in a direction along which the door panels open and close (i.e., door opening 103), and a vertical axis (e.g., y-axis) of the reference coordinate system can be along a direction from the threshold 115 to the header 110 of the sliding door. In another possible implementation, the reference coordinate system is a coordinate system of one of the mmW radars (e.g., the master sensor). In this implementation, mmW radar data from other radar sensors are converted to the coordinate system of the master sensor. This conversion may be aided by communication of IMU data amongst radar sensors.
The method 900 includes collecting 911, at the first radar sensor (S_1), IMU data using IMU_1 and collecting 912 mmW radar data using mmW_radar_1. The method 900 further includes collecting 921, at the second radar sensor (S_2), IMU data using IMU_2 and collecting 922 mmW radar data using mmW_radar_2. The data from each IMU and each mmW radar is then communicated to a door control unit (i.e., DCU) of the automatic door system. For example, SCU_1 and SCU_2 may each be configured to communicate the data over one of the topologies described previously (see
The method 900 further includes determining 931 a conversion for the radar data (e.g., range, angle, position, velocity) based on the IMU_1 data and the IMU_2 data. For example, the DCU 930 may be configured to receive the IMU data, determine the relative orientation and positions of S_1 and S_2 based on the IMU data, and determine the data conversion based on the relative orientations and positions. Based on the determined conversion, the method 900 then includes registering 932 the mmW radar data to a common coordinate system. For example, the DCU may be configured to register the mmW radar data to a reference coordinate system (e.g., automatic door coordinate system) and combine the mmW radar data into combined mmW radar data 940. This method 900 for generating the combined mmW radar data may have the advantage of performing more complex algorithms for data registration at the DCU which can centralize the processing requirements, which may provide cost, complexity and power savings.
The method 1000 further includes generating registered mmW radar data at the first radar sensor (S_1) and the second radar sensor (S1). The method 1000 further includes determining 1013 the conversion and registering 1014 the first mmW radar data (i.e., mmW_radar_1). The first SCU (i.e., SCU_1) can be configured to determine the conversion based on the first IMU data (IMU_1) and second IMU data (IMU_2) received form the second radar sensor (S_2). For example, the relative orientation of the coordinate systems for each radar sensor may be calculated to determine the orientation. The method further includes determining 1023 the conversion and registering 1024 the second mmW radar data (i.e., mmW_radar_2). For example, the second SCU (i.e., SCU_2) can be configured to determine the conversion based on the second IMU data (IMU_2) and first IMU data (IMU_1) received from the first radar sensor (S_1).
The method 1000 further includes communicating the registered mmW radar data to the DCU of the automatic door system. The communication may be accomplished according to one of the topologies described previously, such as shown in
The conversions described may include selecting a reference coordinate system upon which to compare the orientation of a radar sensor (i.e., IMU data). The reference coordinate system may be a fixed coordinate system, such as described by the dimensions of an automatic door. In this case, no IMU data may be necessarily collected for the door because its orientation and position are fixed and known. Alternatively, the reference coordinate system may be a coordinate system of a particular radar sensor (e.g., master sensor). In this case, the IMU data from the particular radar sensor may be collected and used as a reference to which the IMU data from other radar sensors may be compared to determine the necessary conversion for registration. Variations to these approaches may exists and are within the scope of the present disclosure.
The first door area 1101 and the second door area 1102 can be included in a combined door area including combined mmW radar data. The combined mmW radar data may be the union of radar data from the first radar sensor and radar data from the second sensor. The combined door area may include a region including radar data from the first radar sensor (and not the second radar sensor) and a region including radar data from the second sensor (and not the first radar sensor). In the overlap region, the registered radar data from each sensor should ideally match. For example, a target located at a position in the overlap region should be measured at the same position by each radar sensor after the data is registered. Accordingly, combining radar data in the overlap region may include selecting one of the radar sensor registered data or processing (e.g., averaging) the registered data from the two sensors for each point in the overlap region. Zones can be defined in the combined door area so that when a target is detected at a position in the combined door area that corresponds to a zone, a corresponding action (or inaction) may be triggered. Further key positions may be defined in the combined door area, such as a center of the door.
The combined mmW radar data can be a three-dimensional array (i.e., point cloud) of received (i.e., returned) signals from the radar sensors. Each point in the point cloud can be associated with a range to a target from the door and an angle to the target with respect to the door. A strong received signal at a point may indicate a target located at the point. A target may include a group of high received signal points that are clustered in a shape corresponding to the target. In other words, multiple points in the point cloud may correspond to a target. A range to the target may be determined from the ranges of the points in the point cloud corresponding to the target and an angle of the target may be determined from the angles of the points in the point cloud corresponding to the target. The ranges and the angles of the target can change as the target moves. The combined mmW radar data may be analyzed to track one or more targets in a combined door area corresponding to the combined mmW radar data.
A moving target can be tracked through the combined area. For example, the target at a first time is determined to be at a first position 1110 that is included in the first door area 1101, the target at the second time is at a second position 1120 that is included in both the first door area and the second door area, and the target at a third time is at a third position 1130 that is included in the second door area. Tracking may include determining which door area (i.e., radar sensor) includes the target and handing off the tracking between door areas (i.e., radar sensors) as the target moves.
The IMU data for each radar sensor 400 may be expected to remain the same after a service process, such as installation, commissioning, calibration, or re-calibration, is performed. Accordingly, each radar sensor 400 may be configured to monitor its position over time after the service process. which may be useful in generating a notification for a new service process, such as re-calibration.
In the method 1200, baseline IMU data is captured 1210. The capture of the baseline IMU may be triggered manually. For example, a service technician may trigger an SCU 520 to capture and store baseline IMU data in a memory 530 as a baseline orientation 1260. The capture of the baseline IMU data may also be triggered automatically. For example, the SCU 520 may be configured to capture and store a baseline orientation 1260 (e.g., in a memory 530 of the SCU) periodically (e.g., quarterly, yearly) to account for drifting measurement results due to external factors, such as age and temperature. After storing the baseline orientation (i.e., baseline IMU data), the method 1200 may include capturing 1220 IMU data. For example, the SCU 520 may be configured to receive the IMU data captured by the IMU regularly as part of the target sensing for the automatic door operation. The method 1200 may further include comparing 1230 the baseline IMU data (i.e., baseline orientation) and the IMU data to determine if a change 1240 in orientation of the mmW radar has occurred. For example, the SCU 520 may be configured to receive IMU data from the IMU and recall baseline IMU data (i.e., baseline orientation) from a memory 530 of the SCU and perform a comparison. If the comparison satisfies a criterion (e.g., exceeds a threshold) then the SCU may be configured to transmit 1250 a signal based on the change. The method may repeat the comparison (e.g., regularly, as needed, etc.) in order to monitor the position/orientation of the mmW radar.
The signal may be used to trigger events such as a service alert, a recalibration (e.g., automatic recalibration), or the like. In a possible implementation, a change in the position and/or orientation of the mmW radar 600 can trigger the sensor control unit (i.e., SCU 520) to transmit a signal to the DCU 210 to indicate that the IMU data for a radar sensor in the door sensor system 220 has changed. This signal may trigger the DCU 210 to recalibrate the door sensor system and/or to transmit an alert via the door interface 260. For example, a possible alert can be a notification that service of the automatic door system 200 is required. In another example, the signal may trigger the DCU to determine and update values used for the conversion to register and combine mmW radar data.
The combined mmW radar data can include more information than required simply to detect motion and presence. Control of the door can be controlled according to more conditions (i.e., enhanced) based on the additional information. The enhanced door control can be tailored to match a target and its movement and can be less prone to errors (e.g., false opening/closing). Further, this control can account for multiple targets and can adapt to changing conditions. This may have the technical effect of increasing the safety and reliability of an automatic door.
Returning to
The results of the target tracking module can be transmitted to a door control module 1510. The door control module 1510 can be configured to process the information about the one or more targets to determine how, and when, the door should be controlled. Additionally, the results of the target tracking module 1400 can be transmitted to a door filter module 1530. The door filter module 1530 can be configured to process the information about the one or more targets and to identify certain conditions that could trigger certain door operation (or prevent certain door operations). For example, the door operations triggered by the door filter module 1530 may be associated with an operating requirement for safety and/or may be associated with an operating requirement for improving the reliability of the door. For example, the door filter module may be configured to prevent the automatic door from closing on a person in the doorway or from opening in response to movements that should not open the door.
Information from the door control module 1510 and the door filter module 1530 can be received by a door operation module 1550. The door operation module 1550 can be configured to determine when and how to open/close the automatic door based on the information from the door control module 1510 and the door filter module 1530. Some determinations made by the door operation module can include an amount of a door opening/closing, a position of the door opening/closing, a speed of the door opening/closing, and/or a time of the door opening/closing. The door operation module 1550 may be further configured to translate the details regarding the door opening/closing into signals that can be transmitted from the DCU to the door assembly (e.g., the door operator) for door control. Information from the door assembly may be fed back to the door control and/or door filter to help operate the door. For example, the determination of how and when to open/close the automatic door may be based on the state (e.g., opening, opened, closing, closed, etc.) of the door assembly.
After a target is detected, the target tracking module may be further configured to determine 1430 physical characteristics of the target. For example, the target data may be analyzed to determine a shape of the target. The shape of the target may include a width of the target. Additionally, the floor maybe detected from the combined mmW radar data and the height of the target off the floor may be determined. Physical characteristics of the target may help to classify the target. For example, the target may be classified as living (e.g., a person) or as an inanimate (e.g., an object) based on its physical characteristics (e.g., height). These physical characteristics are provided as examples. It is recognized that other physical characteristics and/or information related to physical characteristics (e.g., inanimate/animate) may be determined for a target. Accordingly, this discussion is not intended to limit the present disclosure to these examples.
After a target is detected, the target tracking module may be further configured to determine 1440 a position of the target. The position of the target may be mapped to a position, or region, in the combined door area. Further the position of the target may include a mapping to a particular door area in the combined door area. For example, the position of the target may be designated as corresponding to a particular radar sensor. Further, the position of the target may include a mapping to a zone (e.g., activation zone 131, presence zone 132, etc.) within the combined door area. These positions are provided as examples. It is recognized that other positions and/or information related to a position (e.g., zone) may be determined for a target. Accordingly, this discussion is not intended to limit the present disclosure to these examples.
After a target is detected, the target tracking module may be further configured to determine 1450 a motion of the target. The motion of the target may include a velocity of the target, which includes a speed of the motion and a direction of the motion. When the motion of the target is combined with a position of the target, a path of the target may be determined. In some implementations, the mmW radar data may be analyzed to determine motion on a very small scale. For example, a breathing motion of a person may be detected and used to classify the target (e.g., as animate). These motions are provided as examples. It is recognized that other motions and/or information related to motion (e.g., breathing) may be determined for a target. Accordingly, this discussion is not intended to limit the present disclosure to these examples.
The target tracking module 1400 may be further configured to record (i.e., store) the physical characteristic information, the position information, and/or the motion information (i.e., target information) in a target database 1500. In a possible implementation, the target database 1500 may be queried based on the physical characteristic information, the position information, and/or the motion information to determine if the target is new or if the target is already recorded in the target database. For a target recognized as being in the database, the target tracking module 1400 may be configured to update target information stored for the target. The updates may also add or create new target information corresponding to the target. For example, previously obtained positions may be combined with presently obtained positions to compute a path of the target.
After the target database is updated for a detected target, the mmW radar data may be further analyzed for other targets and the process may be repeated. In other words, detecting targets and determining the corresponding target information may be carried out in series until all targets are recorded. In another possible implementation, segmenting the mmW radar data may include detecting multiple targets in one step and determining the target information for each of the multiple targets may be carried out in a parallel fashion.
As mentioned previously, the door control module 1510 can be configured to process target information to determine how, and when, the door should be controlled. For this, the door control module 1510 may include a crossing point estimator 1511. The crossing point estimator 1511 can be configured to estimate if the target will cross the threshold 115 of the door opening 103. When it is determined that the target will cross the threshold 115 of the door opening 103, the crossing point estimator 1511 can be further configured to estimate where, within the door opening 103, the target will cross the threshold 115 of the automatic door. These estimates may be based on the current position of the target and the target's direction of movement obtained from the target database 1500 (e.g., by query).
The door control module 1510 may further include a time of arrival estimator 1512. The time of arrival estimator 1512 can be configured to estimate when (i.e., a time) the target will arrive at the threshold of the automatic door. This estimate may be based on the position and velocity (i.e., speed, direction) of the target.
The door control module 1510 may further include a door opening estimator 1513. The door opening estimator 1513 can be configured to estimate how large a door opening is needed to accommodate the target. The estimated door opening can be less than a full (i.e., 100%) door opening. This estimate may be based on the size (e.g., width, height), and shape of the target. In some implementations, the door opening estimator 1513 can be configured to estimate where in a range of possible door openings the door opening should be located. This estimate may be based on the crossing point estimated for the target. The results of the estimators may be transmitted to the door operation module 1550. Additionally, the results of the estimators may be transmitted to the target database 1500 to further characterize the target.
As mentioned previously, the door filter module 1530 can be configured to process target information to identify certain conditions that could trigger certain door operations (or prevent certain door operations). These identifications can be used to filter the target. In other words, based on the identified conditions, the target may be considered or ignored for door control. The door filter module 1530 may include a parallel traffic identifier 1531 (i.e., parallel traffic filter). The parallel traffic identifier may determine motion of a target that is moving in a direction other than towards the door. For example, the parallel traffic identifier 1531 may be configured to determine that the target is moving in a direction that is substantially parallel with the door opening 103. The output of the parallel traffic identifier can be a signal having a signal level that indicates that parallel traffic has been identified (e.g., HIGH level) or that parallel traffic has not been identified (e.g., LOW level). The signal may trigger the door operation module 1550 to ignore a target that is not expected to interact with the door.
The door filter module 1530 may further include a nuisance identifier 1532 (i.e., nuisance filter). The nuisance identifier may be configured to determine motion of a target that can be ruled out as actionable movement for the automatic door. For example, the nuisance identifier may determine that the target is moving in a repetitive way, such as motion from a tree blowing in the wind. The output of the nuisance identifier 1532 can be a signal having a signal level that indicates that a nuisance has been identified (e.g., HIGH level) or that a nuisance has not been identified (e.g., LOW level). The signal may trigger the door operation module 1550 to ignore a nuisance target.
The door filter module 1530 may further include an inanimate identifier 1533. The inanimate identifier 1533 may determine that a target is inanimate (i.e., not living). For example, the inanimate identifier may determine that no breathing motion is associated with the target and/or that the height of the target is smaller than a threshold. The output of the inanimate identifier 1533 can be a signal having a signal level that indicates that the target present is inanimate (e.g., HIGH level) or that the target present is animate (e.g., LOW level). The signal may trigger the door operation module 1550 to ignore inanimate targets.
The door filter module 1530 may further include a gesture identifier 1534. The gesture identifier 1534 may identify motion as a particular gesture. For example, the gesture identifier may determine that a person is waving. The output of the gesture identifier 1534 can be a signal having a signal level that indicates that a motion of a target is a gesture (e.g., HIGH level) or that the motion of the target present is not a gesture (e.g., LOW level). Alternatively, the signal may indicate which gesture from a plurality of possible gestures is detected. The signal may trigger the door operation module 1550 to control the door (e.g., open the door) according to the gesture. The results of the identifiers (i.e., filters) may be transmitted to the door operation module 1550. Additionally, the results of the estimators may be transmitted to the target database 1500 to further characterize the target.
As mentioned previously, the door operation module 1550 can be configured to process one or more estimates generated by the door control module 1510, one or more identifications generated by the door filter module 1530, and door information 1560 to determine an action for the automatic door. The door information 1560 may include a state (e.g., percent opened) of the automatic door. The state of the door may be determined by a mathematical door model with inputs corresponding to the inputs of the door. The door operation module 1550 may include an action classifier 1551 that is configured to generate a door action based on the estimates, identifications, and the door information. In a possible implementation the action classifier 1551 is a machine learning model (e.g., neural network) configured to output a signal corresponding to an action of the door based on the estimates and identifications. The door operation module 1550 may further translate this signal corresponding to the action of the door into a signal, or signals, to control the door assembly.
A crossing point estimator 1511 may use the position change vector ({right arrow over (dP)}) to determine where the target will cross the threshold of the automatic door. As shown in
A radar sensor 1720 may be position directly above the center of the door (i.e., may be aligned with the center of the door). The radar sensor can be configured to directly measure velocity (i.e., speed) of a target; however, this speed, which is known as the measured doppler speed (VD), is in a direction towards the radar sensor. Accordingly, a mean speed (Vm) based on a calculation using positions measured by the radar sensor may be different from a measured doppler speed (VD) based on a direct measurement of speed by the radar sensor.
Control of the automatic door may also be based on an estimated time of arrival (Ta) of the target to the door from its current position (PT) and a time of arrival estimator 1512 can be configured to calculate a time of arrival using the mean speed (VM) and/or the measured doppler speed (VD). In other words, a time to arrive at the door can be computed as the time required to reach the crossing point (Cx, Cy, 0) or the time required to reach the center of the door (Cx0, Cy0, 0).
The time of arrival estimator may compute (i) a mean time of arrival (Tm) based on the mean speed (VM) and (ii) a doppler time of arrival (TDp) based on the projected doppler speed. These arrival times may be different because the mean speed can be towards a crossing point, while the projected doppler speed can be towards a center of the door. Accordingly, the time of arrival estimator 1512 may be configured to derive a time of arrival (Ta) for a door opening based on these estimated times of arrival.
Returning to
As shown in
The method 2100 includes determining if a crossing point (Cx, Cy) is in the safe zone (S) 2110. If the crossing point is in the safe zone (S) then the target is not ignored (i.e., no ignore signal). The method 2100 further including determining if a location (Px, Py) of the target is in the safe zone (S) 2120. If the target is in the safe zone (S) then the target is not ignored (i.e., no ignore signal). The method 2100 further includes determining if a projected doppler speed (VDp) is above a threshold speed (Vmin) 2130. If the target is above the threshold speed (i.e., minimum speed), then the target is not ignored. Otherwise, if none of the conditions are satisfied, then the target is ignored (i.e., ignore signal). In other words, if (i) the projected crossing point is not in the safe zone, (ii) the current position of the target is not in the safe zone, and (iii) the target speed is not above a minimum speed, then the target is classified as parallel traffic and an ignore signal is sent so that the door is not triggered to open/close based on the target.
The three-dimensional tracking over a combined area provided by the disclosed approach can enable enhanced door control. The enhanced door control can include opening and closing based on object tracking and object classification (i.e., identification, filtering). In a possible implementation, the door may be controlled to not open for a child or animal based on height measurement. In another possible implementation, the door may be controlled to not open to a person walking near the door but not towards it. In another possible implementation the door may be controlled to partially open based on a computed size of the target. For example, a door opening may be based on the size of the target. In another example, a position of the door opening within the doorway may be based on a crossing point of the target. In another possible implementation, the door may be controlled to not close on a target identified as a human. For example, a breath movement may be identified in order to classify the target as a human. In another implementation, the door may be controlled based on an identified gesture from a target. For example, a motion may be identified as a gesture (e.g., hand waving) to open the door.
The enhanced door control can further include controlling a timing and/or speed of a door opening/closing based on tracking the target. In a possible implementation, the door may be controlled to open at a time corresponding to a time a target will arrive at the door. In another possible implementation, the door may be controlled to open at a rate that corresponds to a speed (i.e., velocity) of a target.
The mmW radar sensors may detect the door and its movement. For example, the mmW radar sensors may detect the left/right panels as the move in an out of the doorway. In a possible implementation, the sensing of the door panels may be used to replace or supplement the information about the door state provided by the door encoder 250. For example, combined mmW radar data may be processed to determine a state of the door (e.g., open, closed, percentage open, etc.). The state of the door may be used to determine an opening time, such as described in conjunction with
The present disclosure describes automatic door systems utilizing mmW radar sensors. However, the use of mmW radar sensors does not preclude an automatic door system from using data from other types of sensors as well. For example, the mmW radar door sensing described here may be combined with optical (i.e., visual) door sensing. One skilled in the art may recognize additional combinations and alternatives. For example, it is possible for mmW radar sensor data to be combined with other sensor data in a sensor fusion approach for door control. It is also possible for mmW radar sensors and other sensors to be used alternatively (i.e., switched). For example, door control may be switched (e.g., automatically switched) between sensing modalities as conditions dictate.
In the specification and/or figures, typical embodiments have been disclosed. The present disclosure is not limited to such exemplary embodiments. The use of the term “and/or” includes any and all combinations of one or more of the associated listed items. The figures are schematic representations and so are not necessarily drawn to scale. Unless otherwise noted, specific terms have been used in a generic and descriptive sense and not for purposes of limitation.
While certain features of the described implementations have been illustrated as described herein, many modifications, substitutions, changes and equivalents will now occur to those skilled in the art. It is, therefore, to be understood that the appended claims are intended to cover all such modifications and changes as fall within the scope of the implementations. It should be understood that they have been presented by way of example only, not limitation, and various changes in form and details may be made. Any portion of the apparatus and/or methods described herein may be combined in any combination, except mutually exclusive combinations. The implementations described herein can include various combinations and/or sub-combinations of the functions, components and/or features of the different implementations described.
As used in this specification, a singular form may, unless definitely indicating a particular case in terms of the context, include a plural form. Spatially relative terms (e.g., over, above, upper, under, beneath, below, lower, and so forth) are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. In some implementations, the relative terms above and below can, respectively, include vertically above and vertically below. In some implementations, the term adjacent can include laterally adjacent to or horizontally adjacent to.
