The subject matter disclosed herein generally relates to aircraft landing zone classification, and more particularly to environmentally-aware landing zone classification for an aircraft.
Optionally-piloted vehicles (OPVs) and unmanned aerial vehicles (UAVs) can operate without a human pilot using autonomous controls. As OPVs and UAVs become more prevalent, they are being operated in less restricted and controlled areas. When OPVs and UAVs are operated autonomously in flight, they must identify a landing zone prior to landing. To account for unpredictable landing zone conditions, OPVs and UAVs typically use an image-based system to identify geometric factors that may impede a safe landing. Current art on autonomous landing zone detection has focused on three-dimensional (3D) terrain-based data acquisition modalities, such as LIght Detection and Ranging scanners (LIDAR), LAser Detection and Ranging scanners (LADAR), and RAdio Detection And Ranging (RADAR) for autonomous landing zone detection. While images can be valuable in identifying a safe landing zone, geometric factors may not provide enough information to determine whether a seemingly flat surface is a suitable landing site. For example, it may be difficult for image-based systems to discriminate between a dry field and a surface of a body of water from only image information. Additionally, in a catastrophic area, other factors can impact landing zone safety.
According to an aspect of the invention, a method of performing environmentally-aware landing zone classification for an aircraft includes receiving environmental sensor data indicative of environmental conditions external to the aircraft. Image sensor data indicative of terrain representing a potential landing zone for the aircraft are received. An environmentally-aware landing zone classification system of the aircraft evaluates the environmental sensor data to classify the potential landing zone relative to a database of landing zone types as environmentally-aware classification data. Geometric features of the potential landing zone are identified in the image sensor data as image-based landing zone classification data. The potential landing zone is classified and identified based on a fusion of the environmentally-aware classification data and the image-based landing zone classification data. A final landing zone classification is provided to landing zone selection logic of the aircraft based on the classifying and identifying of the potential landing zone.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where acquisition of the environmental sensor data and the image sensor data is time synchronized to correlate the environmentally-aware classification data and the image-based landing zone classification data during the fusion of data.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where evaluating the environmental sensor data to classify the potential landing zone further includes comparing time correlated values from multiple environmental sensors in combination with mapping values of the environmental sensor data and differences between the environmental sensor data to the database of landing zone types.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where a safety assessment is performed by comparing, for each type of environmental factor, the environmental sensor data against acceptable limits and making a safety determination based on collective results of the comparing.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where the safety determination is provided directly to the landing zone selection logic of the aircraft.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where one or more environmental sensors of the aircraft are controlled to target one or more features associated with the potential landing zone to perform the safety assessment.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where the classifying and identifying of the potential landing zone further includes making an initial landing zone classification based on the image-based landing zone classification data, and reclassifying and identifying the potential landing zone as an adjustment to the initial landing zone classification based on the environmentally-aware classification data.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include receiving position data for the aircraft, and determining a geographic location of the potential landing zone based on the position data. The position data can be incorporated with the environmentally-aware classification data and the image-based landing zone classification data. A record of the environmentally-aware classification data and the image-based landing zone classification data can be stored based on the position data.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where the environmental sensor data are received from one or more of: a noncontact pyrometer, a thermal-imaging camera, a noncontact infrared temperature sensor, a wind speed sensor, an ambient temperature sensor, a moisture sensor, radiation level detector, and a population-detection camera; and the image sensor data are received from one or more of: a LIght Detection and Ranging scanners (LIDAR) scanner, a video camera, a multi-spectral camera, a stereo camera system, a structure light-based 3D/depth sensor, a time-of-flight camera, a LAser Detection and Ranging scanners (LADAR) scanner, and a RAdio Detection And Ranging (RADAR) scanner.
In addition to one or more of the features described above or below, or as an alternative, further embodiments could include where the aircraft is autonomously controlled during landing based on a final landing zone selected by the landing zone selection logic in response to the final landing zone classification.
According to further aspects of the invention, a system for environmentally-aware landing zone classification for an aircraft is provided. The system includes a processor and memory having instructions stored thereon that, when executed by the processor, cause the system to receive environmental sensor data indicative of environmental conditions external to the aircraft. Image sensor data indicative of terrain representing a potential landing zone for the aircraft are received. The environmental sensor data are evaluated to classify the potential landing zone relative to a database of landing zone types as environmentally-aware classification data. Geometric features of the potential landing zone are identified in the image sensor data as image-based landing zone classification data. The potential landing zone is classified and identified based on a fusion of the environmentally-aware classification data and the image-based landing zone classification data. A final landing zone classification is provided to landing zone selection logic of the aircraft based on the classifying and identifying of the potential landing zone.
The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The foregoing and other features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings in which:
In exemplary embodiments, environmentally-aware landing zone classification is provided for an aircraft. The environmentally-aware landing zone classification operates in conjunction with other landing zone classification systems, such as image-based classification, to increase the probability of selecting a safe landing zone based on observed environmental factors. Examples of environmental factors that can be observed include fire, water, wind, radiation, population, and other such factors that could impede a safe landing on what appears to be otherwise unobstructed terrain. Environmentally-aware classification reduces the risk of potentially landing in a location that was determined acceptable based on geometric factors alone, but in reality would be a less desired and potentially catastrophic area. Embodiments do not rely upon environmental observations alone; rather, environmental data are used to augment geometric information captured from other sensors and/or databases, such as LIDAR, LADAR, RADAR, cameras, Digital Terrain Elevation Data (DTED), and other such systems known in the art. Data acquisition can be synchronized to fuse geometric image-based and environmental data.
The inclusion of environmental factors in landing zone selection further assists in determining a landing zone where an aircraft can potentially land and whether the landing zone appears safe. Environmentally-aware landing zone classification may be implemented in autonomous aircraft, such as optionally-piloted vehicles (OPVs) and unmanned aerial vehicles (UAVs), and/or may be provided to assist in human-piloted aircraft landing zone selection.
Referring now to the drawings,
The environmentally-aware landing zone classification system 106 includes an aircraft computer system 118 having one or more processors and memory to process sensor data acquired from a sensing system 120. The sensing system 120 may be attached to or incorporated within the airframe 108. The sensing system 120 includes one or more environmental sensors 122 and one or more imaging sensors 124. The aircraft computer system 118 processes, in one non-limiting embodiment, raw data acquired through the sensing system 120 while the autonomous UAV 100 is airborne. An environmental sensor processing system 126 interfaces with the environmental sensors 122, while an image sensor processing system 128 interfaces with the imaging sensors 124. The environmental sensor processing system 126 and the image sensor processing system 128 may be incorporated within the aircraft computer system 118 or implemented as one or more separate processing systems that are in communication with the aircraft computer system 118 as part of the environmentally-aware landing zone classification system 106. The environmental sensors 122 can include but are not limited to: noncontact pyrometers for temperature measurement, thermal-imaging cameras, noncontact infrared temperature sensors, wind speed sensors, ambient temperature sensors, moisture sensors, radiation level detectors, and population-detection cameras. Accordingly, the environmental sensors 122 can be used to detect a variety of environmental factors external to the autonomous UAV 100, such as temperature, wind speed, population (human/animal), radiation (nuclear, electromagnetic), and the like, while the autonomous UAV 100 is airborne and in search of a landing site.
The imaging sensors 124 can capture image sensor data of a terrain 130 for processing by the aircraft computer system 118 while the autonomous UAV 100 is airborne. In an embodiment, the imaging sensors 124 may include one or more of: a downward-scanning LIDAR scanner, a video camera, a multi-spectral camera, a stereo camera system, a structure light-based 3D/depth sensor, a time-of-flight camera, a LADAR scanner, a RADAR scanner, or the like in order to capture image sensor data indicative of the terrain 130 and determine geometric information of one or more potential landing zones 132A, 132B, and 132C for the autonomous UAV 100. Additionally, the autonomous UAV 100 may include a navigation system 134, such as, for example, an inertial measurement unit (IMU) that may be used to acquire positional data related to a current rotation and acceleration of the autonomous UAV 100 in order to determine a geographic location of autonomous UAV 100, including a change in position of the autonomous UAV 100. The navigation system 134 can also or alternatively include a global positioning system (GPS) or the like to enhance positional awareness of the autonomous UAV 100.
In exemplary embodiments, the aircraft computer system 118 of the environmentally-aware landing zone classification system 106 performs an analysis of one or more potential landing zones 132A, 132B, and 132C based on both geometric and environmental factors. For example, terrain 130 that is observed by the environmentally-aware landing zone classification system 106 may include geometric impediments 136 (e.g., structures, trees, building, rocks, etc.), such as those depicted near potential landing zone 132C that may clearly rule out potential landing zone 132C as a safe landing zone. While potential landing zones 132A and 132B may both appear to be substantially flat surfaces, geometric analysis alone may be unable to accurately discern that potential landing zone 132A is located upon a water body 138. Using environmental factors extracted from the environmental sensors 122, such as temperature and moisture levels, potential landing zone 132A can be identified as water and therefore an unsafe landing zone. Landing zone classification and identification can perform a number of comparisons to determine suitability of multiple potential landing zones as further described herein.
The system 200 may include a database 212. The database 212 may be used to store potential landing zone profiles, safety limits, position data from navigation system 134, geometric profiles, environmental profiles, and the like. The data stored in the database 212 may be based on one or more other algorithms or processes for implementing the environmentally-aware landing zone classifier 202. For example, in some embodiments data stored in the database 212 may be a result of the processor 204 having subjected data received from the sensing system 120 to one or more filtration processes. The database 212 may be used for any number of reasons. For example, the database 212 may be used to temporarily or permanently store data, to provide a record or log of the data stored therein for subsequent examination or analysis, etc. In some embodiments, the database 212 may store a relationship between data, such as one or more links between data or sets of data acquired through the modalities onboard the autonomous UAV 100 to support data fusion.
The system 200 may provide one or more controls, such as vehicle controls 208. The vehicle controls 208 may provide directives based on, e.g., data associated with the navigation system 134. Directives provided by the vehicle controls 208 may include navigating or repositioning the autonomous UAV 100 to an alternate landing zone for evaluation as a suitable landing zone. The directives may be presented on one or more input/output (I/O) devices 210. The I/O devices 210 may include a display device or screen, audio speakers, a graphical user interface (GUI), etc. In some embodiments, the I/O devices 210 may be used to enter or adjust a linking between data or sets of data. It is to be appreciated that the system 200 is illustrative. In some embodiments, additional components or entities not shown in
The direct safety assessment logic 308 makes a direct safety assessment of the potential landing zones 132A-132C of
The environmental landing zone classification logic 310 can evaluate the environmental sensor data to classify potential landing zones 132A-132C of
Image sensor data indicative of terrain 130 (
Landing zone classification fusion and identification logic 320 can classify and identify the potential landing zones 132A-132C of
Data fusion can also combine geographic location information with geometric and environmental features. For example, geographic locations of the potential landing zones 132A-132C of
As this data is collected over a period of time, profiles can be constructed to determine classification and identification confidence of the potential landing zones 132A-132C of
The landing zone classification fusion and identification logic 320 provides a final landing zone classification to the landing zone selection logic 312 of the autonomous UAV 100 based on the classifying and identifying of the potential landing zones 132A-132C of
Technical effects include potential landing zone selection for an aircraft based on environmental factors and geometric factors of the potential landing zone.
While the invention has been described in detail in connection with only a limited number of embodiments, it should be readily understood that the invention is not limited to such disclosed embodiments. Rather, the invention can be modified to incorporate any number of variations, alterations, substitutions or equivalent arrangements not heretofore described, but which are commensurate with the spirit and scope of the invention. Additionally, while various embodiments of the invention have been described, it is to be understood that aspects of the invention may include only some of the described embodiments. Accordingly, the invention is not to be seen as limited by the foregoing description, but is only limited by the scope of the appended claims.
This application claims the benefit of U.S. provisional patent application Ser. No. 62/027,321 filed Jul. 22, 2014, the entire contents of which are incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
62027321 | Jul 2014 | US |