Patent Application
20230410272
This application claims priority to, and the benefit of, India Patent Application No. 202241029166, filed May 20, 2022 (DAS Code 5FC7) and titled “SYSTEMS AND METHODS FOR AIR CARGO CONTAINER DAMAGE MONITORING,” which is incorporated by reference herein in its entirety for all purposes.
The present disclosure relates generally to aircraft cargo management, more specifically, identification of cargo damage.
Aircraft cargo compartments are used to carry luggage and other cargo during a flight. Air cargo containers, such as Unit Load Devices (ULDs), are used for loading cargo into aircraft. ULDs and cargo pallets are typically made of aluminum and come in standardized shapes and sizes configured for bulk loading a large quantity of cargo. They also allow cargo to be efficiently loaded and fastened inside the aircraft, reducing loading time and the risk of cargo and/or aircraft damage.
Many cargo and passenger aircraft are equipped with semi-automatic Cargo Loading System (CLS) in the compartment. The CLS is an electrically powered system that allows Unit Load Devices (ULDs) to be carried into the aircraft cargo compartment. The primary benefit of the CLS is to reduce the manpower and loading/unloading time during shipment. One of the many risks to shipping cargo via aircraft is cargo damage. Improper cargo stacking, damaged cargo containers, and improper fastening of cargo have damaged aircraft fuselages, on-board cargo equipment, aircraft doors, and the like. Moreover, damaged ULDs with uneven surfaces may block movement by jamming the CLS motor drives, causing unexpected delays. Accordingly, ULDs need to be visually inspected before being loaded into the aircraft.
A method for monitoring cargo loading is disclosed herein. In various embodiments, the method may comprise receiving, by a field device, image data of cargo from a camera. In various embodiments, the method may comprise interfacing, by the field device, the field device with an on ground infrastructure network. In various embodiments, the interfacing may comprise sending by the field device, the image data to the on ground infrastructure network. The interfacing may further comprise receiving, by the field device, feedback from the on ground infrastructure network, wherein the feedback may comprise a damage classification. The method may further comprise instructing, by the field device, a cargo loading system to halt cargo loading in response to the damage classification.
In various embodiments, the method may comprise processing, by the field device, the image data. The processing may further comprise filtering, by the field device, the image data. The processing may further comprise compressing, by the field device, the image data. The processing may further comprise performing, by the field device, image segmentation and representation. The processing may further comprise performing, by the field device, image extraction.
In various embodiments, the interfacing may further comprise transmitting, by the field device, a unit load device configuration to the on ground infrastructure network, wherein the on ground infrastructure network may be configured to select a trained AI-based Analytics Model based on the unit load device configuration. In various embodiments, the sending may comprise sending, by the field device, image data processed by the field device to the on ground infrastructure network, wherein the on ground infrastructure network may be configured to process the image data and may be further configured to perform a damage classification using the selected trained AI-based Analytics Model.
In various embodiments, the method may comprise receiving, by the field device, a damage report from the on ground infrastructure network. The interfacing may further comprise receiving, by the field device, an instruction from the on ground infrastructure network for a unit load device scan. In various embodiments, the receiving, by the field device, image data from the camera may comprise the field device electronically communicating with the camera. In various embodiments, the method may comprise commanding, by the field device, the camera to adjust a view of the cargo.
A method for monitoring cargo loading is also disclosed herein. The method may comprise receiving, by an on ground infrastructure network, image data of cargo from a camera. In various embodiments, the method may comprise directing, by the on ground infrastructure network, the camera to activate, wherein the camera may be configured to capture image data. In various embodiments, the method may comprise initiating, by the on ground infrastructure network, a sensor to scan a unit load device.
The method may further comprise interfacing, by the on ground infrastructure network, the on ground infrastructure network with the field device. The interfacing may further comprise receiving, by the on ground infrastructure network, the image data from the field device. In various embodiments, the method may comprise sending, by the on ground infrastructure network, feedback from the field device, wherein the feedback may comprise a damage classification. In various embodiments, the method may further comprise monitoring, by the on ground infrastructure network, loading of cargo based on the damage classification. The method may further comprise instructing, by the on ground infrastructure network, a cargo loading system to halt cargo loading in response to the damage classification.
In various embodiments, the interfacing may further comprise receiving, by the on ground infrastructure network, a unit load device scan from the field device. The interfacing may further comprise extracting, by the on ground infrastructure network, a unit load device configuration. The interfacing may further comprise selecting, by the on ground infrastructure network, a trained AI-based Analytics Model based on the unit load device configuration. The interfacing may further comprise processing, by the on ground infrastructure network, the image data. The interfacing may further comprise performing, by the on ground infrastructure network, a damage classification using the selected trained AI-based Analytics Model. In various embodiments, the initiating may further comprise selecting, by the on ground infrastructure network, one of an RFID scan, barcode scan, or text scan.
A method for monitoring cargo loading is also disclosed herein. In various embodiments, the method may comprise receiving, by a field device, image data of cargo from a camera. In various embodiments, the method may comprise interfacing, by the field device, the field device with an aircraft avionics system. In various embodiments, the method may comprise interfacing, by the field device, the field device with on ground infrastructure network. In various embodiments, the interfacing may comprise sending by the field device, the image data to the on ground infrastructure network. The interfacing may further comprise receiving, by the field device, feedback from the on ground infrastructure network, wherein the feedback may comprise a damage classification. The method may further comprise instructing, by the field device, a cargo loading system to halt cargo loading in response to the damage classification.
In various embodiments, the instructing may comprise instructing, by the field device, a movement controller of the cargo loading system. In various embodiments, the instructing may further comprise reversing, by the field device, the movement controller of the cargo loading system. In various embodiments, the method may comprise generating, by the field device, an alarm based on the damage classification. The generating may further comprise transmitting, by the field device, the alarm to the aircraft avionics system.
The detailed description of exemplary embodiments herein makes reference to the accompanying drawings, which show exemplary embodiments by way of illustration. While these exemplary embodiments are described in sufficient detail to enable those skilled in the art to practice the disclosure, it should be understood that other embodiments may be realized and that logical changes and adaptations in design and construction may be made in accordance with this disclosure and the teachings herein. Thus, the detailed description herein is presented for purposes of illustration only and not of limitation. The scope of the disclosure is defined by the appended claims. For example, the steps recited in any of the method or process descriptions may be executed in any order and are not necessarily limited to the order presented. Furthermore, any reference to singular includes plural embodiments, and any reference to more than one component or step may include a singular embodiment or step. Also, any reference to attached, fixed, connected or the like may include permanent, removable, temporary, partial, full and/or any other possible attachment option. Additionally, any reference to without contact (or similar phrases) may also include reduced contact or minimal contact.
As used herein, the term “network” includes any cloud, cloud computing system, or electronic communications system or method which incorporates hardware and/or software components. Communication among the parties may be accomplished through any suitable communication channels, such as, for example, a telephone network, an extranet, an intranet, internet, point of interaction device (point of sale device, personal digital assistant (e.g., an IPHONE® device, a BLACKBERRY® device), cellular phone, kiosk, etc.), online communications, satellite communications, off-line communications, wireless communications, transponder communications, local area network (LAN), wide area network (WAN), virtual private network (VPN), networked or linked devices, keyboard, mouse, and/or any suitable communication or data input modality. Moreover, although the system is frequently described herein as being implemented with TCP/IP communications protocols, the system may also be implemented using IPX, APPLETALK® program, IP-6, NetBIOS, OSI, any tunneling protocol (e.g., IPsec, SSH, etc.), or any number of existing or future protocols. If the network is in the nature of a public network, such as the internet, it may be advantageous to presume the network to be insecure and open to eavesdroppers. Specific information related to the protocols, standards, and application software utilized in connection with the internet is generally known to those skilled in the art and, as such, need not be detailed herein.
“Cloud” or “Cloud computing” includes a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. Cloud computing may include location-independent computing, whereby shared servers provide resources, software, and data to computers and other devices on demand.
As used herein, “electronic communication” means communication of electronic signals with physical coupling (e.g., “electrical communication” or “electrically coupled”) or without physical coupling and via an electromagnetic field (e.g., “inductive communication” or “inductively coupled” or “inductive coupling”). In this regard, “electronic communication,” as used herein, includes wired and wireless communications (e.g., Bluetooth, TCP/IP, Wi-Fi, etc.). Additionally, as used herein, “Bluetooth” is a low-power, short-range wireless radio communication.
As used herein, “transmit” may include sending electronic data from one system component to another over a network connection. Additionally, as used herein, “data” may include encompassing information such as commands, queries, files, data for storage, and the like in digital or any other form. As used herein, “Machine Learning” may be, for example, a method of data analysis that automates analytical model building. For example, a method of data analysis that automates analytical model building may be referred to as an “AI-based Analytics Model.” Additionally, as used herein, “Deep Learning” may be a subset of Machine Learning, comprising multiple layers of analysis.
As disclosed herein, “cargo,” “cargo containers,” and “unit load devices (ULDs)” may be interchangeably used to refer to aircraft cargo. As will be discussed in further detail below, an aircraft and aircraft cargo loading systems may contain multiple image sensors, cameras, and motion sensors placed at different locations within the aircraft or on cargo loading systems near the aircraft, such that the sensors and cameras may cover a 360-degree view of cargo as the cargo is loaded into the aircraft. These systems may also be placed on a tarmac, along a cargo loading ramp, and/or along the cargo bay door of the aircraft to scan cargo as it moves into and out of the cargo bay. These systems may be commanded by an onboard device, such as, for example, a field device, to scan the cargo as cargo enters and traverses within the cargo bay. The field device may be, for example, an aircraft interface device (AID). These systems may capture image data and relay the data to the field device. Image data may be, for example, images, three-dimensional scans, and video feeds of cargo. The field device may process the data and may interface with and relay data to on board video storage systems, avionics systems aboard the aircraft, and on ground infrastructure network. On ground infrastructure network may use Machine Learning and/or pre-trained AI-based Analytics Models to identify and classify cargo damage and defects. The on ground infrastructure network may do this by comparing the cargo being scanned to an appropriate predetermined ULD configuration. ULD configuration information may be stored with the on ground infrastructure network and may be selected by the on ground infrastructure network based on cargo details retrieved from a QR (Quick Response) code, barcode, text, and/or RFID-based (radio-frequency identification) tag. Cargo details may be model numbers and serial numbers associated with the ULDs. On ground infrastructure network may be, for example, the cloud.
As will be described in further detail below, the AI-based Analytics Model may be trained to identify cargo damage. This model may be trained to detect irregularities in shape and size of cargo or unit load device (ULD) containers and classify the cargo as damaged. The on ground infrastructure network may provide feedback to the field device, which may, in turn, instruct a cargo loading system (CLS) to halt cargo loading or reverse cargo loading, enabling timely removal of damaged cargo.
Accordingly, a method of monitoring cargo loading by the field device may be used to detect damage to cargo and ULDs, reducing the need for manual inspection of cargo on a cargo loading ramp or in the cargo bay and/or eliminate the need for constant human monitoring of cargo loading.
Referring to
An shown in
There are several ULD types and configurations approved to be loaded as aircraft cargo. Accordingly, since each ULD 104 differs in shape and size, the on ground infrastructure network 106 may first identify the particular ULD configuration of the scanned ULD 104 before classifying damage to the ULD 104. The on ground infrastructure network 106 may choose a configuration from a configuration database 110 stored in the on ground infrastructure network 106. In identifying the associated ULD configuration, the on ground infrastructure network 106 may select a pre-trained AI-based Analytics Model associated with the configuration. This may be an AI-based classification application 112 that is specific to the ULD configuration. Accordingly, a ULD damage classification analysis 114 may be tailored to the specific ULD configuration.
As will be discussed in further detail below, the on ground infrastructure network 106 may comprise a damage classification database that enables the AI-based application 112 (e.g., the AI-based Analytics Model) to perform the damage classification analysis 114, analyzing the scanned ULD for irregularities. The on ground infrastructure network 106 may comprise a high performance computer 118 that may provide computing power for processing and analysis.
As shown in
Proceeding with a QR/barcode scan (step 308), the field device may direct a camera to capture (step 320) an image and transmit the image to the field device. The field device may perform (step 322) image segmentation and decode (step 324) the letters in the image. The field device may consolidate, format, and perform (step 326) an XML message formation for the serial and model numbers.
Proceeding with a text scan (step 310), the field device may direct a camera to capture (step 328) an image and transmit the image to the field device. The field device may perform (step 330) image segmentation. The field device may then preprocess and enhance (step 332) the image. The field device may perform feature extraction and filter the image (step 334). The field device may classify (step 336) the image, identifying the model number and serial number of the ULD. The field device may consolidate, format, and perform (step 338) an XML (extensible markup language) message formation for the serial and model numbers.
Proceeding with either the RFID scan (step 306), QR/barcode scan (step 308), or the text scan (step 310), the field device may transmit (step 340) an XML message to on ground infrastructure network (e.g., the cloud) to obtain a ULD configuration. The field device and the on ground infrastructure network may be connected via a web interface. The on ground infrastructure network may extract (step 342) ULD configuration information from a database and send the configuration to the field device. The field device may validate and process (step 344) the received configuration information and send the processed information to a high performance computer (HPC) of the on ground infrastructure network. The HPC of the on ground infrastructure network may be configured for AI-based model selection, which may be used to compare the ULD to a pre-set scan of the ULD.
Referring to
The method 501 for training and validating the AI-based Analytics Model for damage identification is further shown in
To increase training precision, the sensors may also be used to capture the ULD at various areas and near various components of a hypothetical cargo loading system and/or cargo bay, such as, for examples, near forklift edges, rails, rivets, joints, doors, latches, and the like. ULDs used during this training may be normal, undamaged ULDs, as well as damaged ULDs to enable the on ground infrastructure network to classify the ULDs as normal or damaged. In various embodiments, the training and validating of the AI-based Analytics Model may occur in real-time, simultaneous with real-time monitoring. For example, as a new kind of defect or damage is identified, the AI-based Analytics Model may be trained for a new damage classification, and the on ground infrastructure network may be updated based on the new training data. In various embodiments, scanned images of the ULDs may be supplemented with computer generated images. These images may be fed into the AI-based Analytics Model of the on ground infrastructure network. This may enable the network to avoid under fitting or over fitting the classification data and increase classification precision.
For every type of ULD, a respective AI-based Analytics Model may be identified. These models may be trained on the type of ULD for which training is in progress. The various trained models may be stored along with the associated ULD configuration in the on ground infrastructure network. This may enable the on ground infrastructure network to activate the relevant trained model during real time damage detection.
Referring to
Referring to
In various embodiments, the method may comprise receiving (step 802), by a field device, image data of cargo from a camera. In various embodiments, the method 800 may comprise interfacing (step 806), by the field device, the field device with on ground infrastructure network. In various embodiments, the interfacing (step 806) may comprise sending (step 808) by the field device, the image data to the on ground infrastructure network. The interfacing (step 806) may further comprise receiving (step 810), by the field device, feedback from the on ground infrastructure network, wherein the feedback may comprise a damage classification. The method 800 may further comprise instructing (step 812), by the field device, a cargo loading system to halt cargo loading in response to the damage classification. Instructing the cargo loading system to halt cargo loading may allow removal of a damaged unit load device from the cargo loading system.
In various embodiments, the interfacing (step 806) may further comprise transmitting (step 807), by the field device, a unit load device configuration to the on ground infrastructure network, wherein the on ground infrastructure network may be configured to select a trained AI-based Analytics Model based on the unit load device configuration. In various embodiments, the sending (step 808) may comprise sending, by the field device, image data processed by the field device to the on ground infrastructure network, wherein the on ground infrastructure network may be configured to process the image data and may be further configured to perform a damage classification using the selected trained AI-based Analytics Model.
In various embodiments, the method 800 may comprise receiving (step 811), by the field device, a damage report from the on ground infrastructure network. The interfacing may further comprise receiving (step 813), by the field device, an instruction from the on ground infrastructure network for a unit load device scan. In various embodiments, the receiving (step 802), by the field device, image data of cargo from the camera may comprise the field device electronically communicating (step 815) with the camera. In various embodiments, the method 800 may comprise commanding (step 817), by the field device, the camera to adjust a view of the cargo.
With reference to
A method for monitoring cargo loading is also disclosed herein. The method (10,
The method may further comprise interfacing (step 20), by the on ground infrastructure network, the on ground infrastructure network with the field device. The interfacing (step 20) may further comprise receiving (step 22), by the on ground infrastructure network, the image data from the field device. In various embodiments, the method 10 may comprise sending (step 24), by the on ground infrastructure network, feedback to the field device, wherein the feedback may comprise a damage classification. In various embodiments, the method 10 may further comprise monitoring (step 26), by the on ground infrastructure network, loading of cargo based on the damage classification. The method 10 may further comprise instructing (step 28), by the on ground infrastructure network, a cargo loading system to halt cargo loading in response to the damage classification. Instructing the cargo loading system to halt cargo loading may be configured to allow removal of a damaged unit load device from the cargo loading system.
In various embodiments, the interfacing (step 20) may further comprise receiving (step 21), by the on ground infrastructure network, a unit load device scan from the field device. As shown in
Additional embodiments of the present disclosure may comprise a method (50,
In various embodiments, the instructing (step 64) may comprise instructing (step 66), by the field device, a movement controller of the cargo loading system. In various embodiments, the instructing (step 66) may further comprise reversing (step 68), by the field device, the movement controller of the cargo loading system. In various embodiments, the method 50 may comprise generating (step 70), by the field device, an alarm based on the damage classification. The generating (step 70) may further comprise transmitting (step 72), by the field device, the alarm to the aircraft avionics system. In various embodiments, an alarm may be generated by the field device and transmitted by the field device to a device controlled by cargo loading personnel, such as, for example, a cargo loading control panel and/or handheld device. It may be understood that an alarm may be transmitted to any device used to manage and/or monitor cargo loading. The present disclosure is not limited in that regard.
Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. Furthermore, the connecting lines shown in the various figures contained herein are intended to represent exemplary functional relationships and/or physical couplings between the various elements. It should be noted that many alternative or additional functional relationships or physical connections may be present in a practical system. However, the benefits, advantages, solutions to problems, and any elements that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or elements of the disclosure. The scope of the disclosure is accordingly to be limited by nothing other than the appended claims, in which reference to an element in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more.” Moreover, where a phrase similar to “at least one of A, B, or C” is used in the claims, it is intended that the phrase be interpreted to mean that A alone may be present in an embodiment, B alone may be present in an embodiment, C alone may be present in an embodiment, or that any combination of the elements A, B and C may be present in a single embodiment; for example, A and B, A and C, B and C, or A and B and C.
Systems, methods and apparatus are provided herein. In the detailed description herein, references to “various embodiments”, “one embodiment”, “an embodiment”, “an example embodiment”, etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. Further, when a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to affect such feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. After reading the description, it will be apparent to one skilled in the relevant art(s) how to implement the disclosure in alternative embodiments.
Furthermore, no element, component, or method step in the present disclosure is intended to be dedicated to the public regardless of whether the element, component, or method step is explicitly recited in the claims. No claim element herein is intended to invoke 35 U.S.C. 112(f), unless the element is expressly recited using the phrase “means for.” As used herein, the terms “comprises”, “comprising”, or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202241029166 | May 2022 | IN | national |
