Patent Application
20230156055
The present disclosure relates generally to systems for transferring data and, more specifically, systems that transfer data from a vehicle to a remote node in a secure and real-time manner.
Various types of vehicles include video surveillance systems that provide for monitoring activities that occur on or with the vehicle. The video is saved in local storage and archived. The relevant video can be accessed and retrieved if there is an event that requires a review of the activities of the vehicle.
An issue with these systems is the amount of video data collected is too large to simply be uploaded continuously to the cloud. The process of uploading, storing, and subsequently accessing the video data is inefficient and untimely. This type of system can also be expensive to implement and maintain. In one application for aircraft used for long-haul flights, operators rely on SATCOM for uploading the video during flight. While this system does work, there is significant cost associated with moving data to and from the vehicle.
Another issue is the delay in recording and accessing the video data. There are events in which it is necessary to access the data in a real-time period. For example, video may be needed in near real-time to address events involving terrorism, human trafficking and many other international crimes. Current systems introduce a lag in accessing the information which may require the event to conclude before the data is available to be retrieved. Further, some events may prevent the video data from being uploaded and thus the data may not be recoverable.
In addition to video data, there is also other types of data that are recorded at a vehicle and stored. Examples include but are not limited to phase-of-flight transitions for an aircraft, security event markers, audio markers, and various sensor data. This data should also be managed in a manner that can provide for real-time review.
One aspect is directed to a computing device positioned in a vehicle to stream data from the vehicle to a monitoring node. The computing device comprises communications circuitry and processing circuitry operatively connected to the communications circuitry. The processing circuitry is configured to: establish a secure communication tunnel with the monitoring node while the vehicle is moving; obtain data to be streamed to the monitoring node; translate the data into a message using a publish-and-subscribe protocol; initiate a communication session with the monitoring node; and transmit the message from the vehicle to the monitoring node via the secure communication tunnel.
In another aspect, the processing circuitry is configured to obtain the data from storage on the vehicle.
In another aspect, the processing circuitry is configured to obtain the data from one or more cameras in the vehicle.
In another aspect, the processing circuitry is configured to obtain the data in response to receiving a request from the monitoring node.
In another aspect, the processing circuitry is further configured to: encrypt the data; save the encrypted data at the vehicle; initiate the communication session with the monitoring node; after initiating the communication session, receive a request for the data; and in response to receiving the request, decrypt the data prior to translating the data into the message.
In another aspect, the publish-and-subscribe protocol is an MQTT protocol.
In another aspect, the secure communication tunnel is a TLS tunnel with a first endpoint at the vehicle and a second endpoint at the monitoring node.
In another aspect, the processing circuitry initiates the communication session based on an event that occurs with the vehicle.
One aspect is directed to a computing device to transfer data from a moving vehicle to a monitoring node located remotely from the vehicle. The computing device comprises a memory circuitry configured to store computer-readable program code, and processing circuitry configured to execute the computer-readable program code to cause the computing device to: establish a communication tunnel with the monitoring node; initiate a communication session with the monitoring node; after initiating the communication session, receive a request for the data from the monitoring node with the request being received through the communication tunnel; obtain the data from one or more cameras positioned in the vehicle; in response to the request, translating the data into a message using MQTT protocol; and transmit the data through the communication tunnel from the vehicle to the monitoring node.
In another aspect, the processing circuitry is further configured to obtain the data from a specific one of the cameras that is indicated in the request.
In another aspect, the image processing circuitry is configured to: obtain the data prior to receiving the request; encrypt the data; store the encrypted data; in response to the request, decrypt the encrypted data; and transmit the data through the communication tunnel from the vehicle to the monitoring node.
In another aspect, the processing circuitry is configured to transmit the data to the monitoring node prior to saving the data at the vehicle.
In another aspect, the vehicle is an aircraft and the processing circuitry is configured to transmit the data during flight.
One aspect is directed to a method of transmitting data from a moving vehicle to a remote monitoring node located away from the vehicle. The method comprises: capturing the data through one or more cameras located in the vehicle; after initiating communication, receiving a request to transfer the data to the monitoring node; establishing a secure communication tunnel between the vehicle and the monitoring node; and transferring the data from the vehicle to the monitoring node through the secure communication tunnel.
In another aspect, the method further comprises transferring the data from the vehicle through one or more satellites and then to the monitoring node.
In another aspect, the method further comprises storing the data at the vehicle prior to transferring the data to the monitoring node.
In another aspect, the method further comprises translating the data using MQTT protocol and subsequently transferring the data to the monitoring node.
In another aspect, the method further comprises receiving a request from the monitoring node to access the data and in response transferring the data from the vehicle to the monitoring node.
In another aspect, the method further comprises transferring the data through the secure communication tunnel separately as video packets and audio packets.
In another aspect, the method further comprises live streaming the data captured by the one or more cameras through the secure communication tunnel.
The features, functions and advantages that have been discussed can be achieved independently in various aspects or may be combined in yet other aspects, further details of which can be seen with reference to the following description and the drawings.
One or more control panels 60 are positioned on the vehicle 100 to allow persons onboard the vehicle 100 to view the image data. The control panels 60 can be positioned at one or more locations within the vehicle 100, including but not limited to the flight deck 102 for viewing by the flight deck personnel (e.g., pilot, co-pilot) and the cabin area 103 for viewing by other flight crew members. In one example, the control panels 60 are wireless tablets that can be carried by flight personnel throughout the vehicle 100.
The computing device 20 is configured to transmit the image data through a communication network 150 to a remote monitoring node 130.
The communication system 150 is configured to support offboard connectivity to transmit the image data while the vehicle 100 is moving. In one example with an aircraft 100, this occurs during flight. In one example, the image data is streamed in real time from the vehicle 100. Thus, personnel on the ground are able to receive images and/or video of events that are occurring on the vehicle 100 in real time. In another example, the image data is stored at the vehicle 100 and subsequently transmitted to the monitoring node 130 for access by the one or more nodes 154.
The computing device 20 establishes the tunnel 160 that includes certificate-based functionality that provides mutual authentication (e.g. according to 802.1x) of the end-points of the tunnel 160, establish the TLS tunnel 160, and authorize access to trusted services. The computing device 20 includes a Public Key Infrastructure (PKI) that manages and maintains the certificates, as well as installs, revokes, and securely stores the certificates.
The monitoring node 130 provides an opposing endpoint for the tunnel 160. The monitoring node 130 authenticates inbound connection requests from the vehicle 100. Once authenticated, the image data is received at the monitoring node 130 and transmitted or otherwise made accessible to the one or more nodes 154. The image data can also be stored by the monitoring node 130 for later access. The monitoring node 130 can encrypt the image data prior to providing access to the nodes 154 and/or transmitting the image data to the nodes 154. In one example, the image data is encrypted utilizing a single-use AES 256 key and encrypting that AES 256 key with a public key.
In one example, the monitoring node 130 includes image player circuitry for viewing the image data. The monitoring node 130 receives the real-time image data pushed by the vehicle 100 and the image player circuitry provides for viewing the image data. In one example, the image player circuitry provides for web-based access for viewing of the image data by the one or more nodes 154. The image player circuitry provides for various viewing functionality, including but not limited to pausing, rewinding and fast forwarding up to the point of receipt of the image data.
The monitoring node 130 provides for remote control of the computing device 20 and/or cameras 21 onboard the vehicle 100. This control provides for remote personnel to select the applicable image data for remote viewing. The control can also provide for the remote user to adjust one or more of the cameras 21 to obtain the desired image data. The control of the computing device 20 and cameras 21 can be performed by personnel stationed at the monitoring node 130 and/or at one of the nodes 154.
In one example, the monitoring node 130 is configured to provide a web interface for access by the nodes 154. The monitoring node 130 is configured for accessing image data using a browser-based interface or an applications program interface (API). The browser-based interface can include a website through which the image data can be accessed. The website can be hosted by the monitoring node 130 or at another location accessible through the network 153.
The cryptographically secure point-to-point tunnel 160 is established when connectivity is available and the computing device 20 makes a request to connect to the monitoring node 130 and the identities of both endpoints is mutually authenticated. The computing device 20 and the monitoring node 130 participate in a certificate-based mutual authentication process to validate the identities of the endpoints. Once authenticated, the cryptographically secure TLS-based tunnel 160 is established and maintained via heartbeat signals or messages.
In one example, security restrictions require that the vehicle 100 initiate connections with the monitoring node 130. These restrictions prevent the use of conventional bidirectional connection negotiations that provide for the monitoring node 130 to initiate a connection. To satisfy these security restrictions, a publish-and-subscribe architecture is used for exchanging the image data between the vehicle 100, monitoring node 130, and nodes 154. The monitoring node 130 acts as a broker that relays image data downloaded from the vehicle 100 to the nodes 154. Further, the monitoring node 130 collects and stages messages from the nodes 154 intended for uploading to the vehicle 100. For example, in one example monitoring node 130 receives and stages the messages intended to be sent to vehicle 100. In one example, the messages may be staged in a local memory, or in memory accessible to the monitoring node 130. As stated above, monitoring node 130 does not initiate a connection with vehicle 100 and therefore does not send the messages to vehicle 100 until it receives a connection initiation request from the computing device 20 on the vehicle 100. Once the monitoring node 130 receives the connection initiation request from the vehicle 100 and the connection is established, the monitoring node 130 transmits the staged messages to the vehicle 100.
The publish-and-subscribe architecture further provides for each of the messages to be categorized by one or more topics. The image data messages are transmitted to the one or more nodes 154 that subscribe to the topic. For example, image data may be categorized by a topic of “cargo hold”. A node 154 for shipping analysis may be subscribed to this topic, and therefore would receive the image data for this topic through the monitoring node 130. A second node associated with safety functions on the vehicle 100 may not subscribe to this topic and therefore would not receive the “cargo hold” image data. The number of topics can vary.
In one example, messages received at the monitoring node 130 during times when the computing device 20 has not initiated a connection are timestamped and staged by the monitoring node 130. The messages are held by the monitoring node 130 until the vehicle 100 initiates a connection with the monitoring node 130 and requests the messages.
In one example, the publish-and-subscribe architecture uses Message Queuing Telemetry Transport (MQTT) protocol. The MQTT protocol is built on the TCP/IP protocol and provides real-time reliable messaging services to the nodes 154 with minimal code and limited bandwidth. The MQTT protocol runs on TCP and is an application layer protocol. Therefore, the MQTT protocol can be used in application scenarios that support the TCP/IP protocol stack. MQTT includes a publish-subscribe messaging pattern that uses the monitoring node 130 as a message broker to distribute messages to the applicable nodes 154 based on the topic of a message. Typically, there are multiple MQTT topics available, which are associated with different nodes 154.
The messages received at the vehicle 100 can includes requests for previously recorded data and/or real-time data.
After receipt, the image data is encrypted (block 202) and saved (block 204) by the computing device 20. At some point thereafter, a request is received from one or more of the nodes 154 for the image data (block 206). The request can be for the entirety of the image data or for one or more discrete portions. After receiving the request, the image data is decrypted (block 208) and transmitted to the monitoring node 130 (block 210). In one example, the image data is transmitted using the MQTT protocol.
The processing circuitry 23 can also receive requests for real-time image data. For these situations, the image data is not initially encrypted and/or stored at the vehicle 100.
In one example, the request is received prior to the image data being captured. In response to the request, the computing device 20 causes the one or more cameras 21 to capture the scene. In another example, the computing device 20 receives image data continuously and/or on periodic intervals. Prior to receiving the request, the receive image data is encrypted and stored and transmitted as disclosed in
The real-time image data includes one or more images, video, and/or audio of events that are currently occurring on the vehicle 100. The events are captured in real-time, although there may be delays (latency) caused by the process of capturing and transmitting/accessing the image data. The latency can be caused by one or more of the time for the cameras 21 to capture the images, the transmission of the image data to the computing device 20, the transmission to the monitoring node 130 due to the speed of data transfer (bps), and displaying the image data on a display screen (either at the monitoring node 130 or other node 154). In one example, what the viewer views on their display will have a delay from when the event actually occurred.
Nodes 154 can access the image data at the monitoring node 130 through a variety of devices, including but not limited to laptop computers, personal computers, personal digital assistants, mobile computing/communication, tablet devices, and various other-like computing devices. Each of the nodes 154 accesses the monitoring node 130 either directly through a dedicated network or through the network 153 (e.g., Internet). In one example, one or more of the nodes 154 accesses the monitoring node 130 through a separate portal. Each node’s portal can include a secure interface through which the node 154 can access the image data and have access to one or more of the vehicles 100. In one example, nodes 154 are assigned one or more vehicles 100 from which they can receive the image data.
In one example, the monitoring node 130 is configured for browser-based accessibility. The browser-based interface can support well-known browsers. Alternatively, or in conjunction the nodes 154 can obtain the image data using one or more APIs.
The aspects disclosed above include the transmission of image data that is captured by one or more cameras 21. The structures and processes are also applicable for gathering and transmitting other types of data. On example includes one or more sensors 105 (see
In one example, the computing device 20 assigns different topics to the data based on the data type. For example, a first topic includes image data which is assigned a topic for publication to a first set of nodes 154, and a second topic includes flight control data which is assigned a different topic for publication to a second set of nodes.
In one example, the vehicle 100 initiates the communication with the monitoring node 130. This can include the vehicle 100 periodically initiating communication at predetermined time periods. Additionally or alternatively, a specific predetermined event that occurs on the vehicle 100 can trigger or cause computing device 20 into initiating the establishment of a secure communications tunnel 160 with the monitoring node 130. In one example, an unplanned or sudden change in vehicle movement (e.g., change in flight path, change in elevation, change in course, change in speed) may trigger or cause the computing device 20 to initiate the establishment of a secure communications tunnel 160 with the monitoring node 130. In another example, detected motion in an area of the vehicle 100 not typically known for movement during flight operations (e.g., the cargo hold) can trigger the computing device 20 to initiate the establishment of a secure communications tunnel 160. Initiating the establishment of the secure communications tunnel 160 may be performed automatically, such as in response to a signal output by one or more sensors, or be performed manually, such as responsive to input by onboard personnel. For example, a flight crew member may input a command or provide a signal to the computing device 20 through a control panel 60. In response to receiving the input, computing device 20 would initiate the establishment of the secure communications tunnel with the monitoring node 130.
In one example, the computing device 20 is configured to automatically trigger data transfers to the monitoring node 130 at predetermined times and/or predetermined events. Specific examples include but are not limited to automatically pushing one or more images or video during takeoff and landing of the aircraft 100.
The computing device 20 is configured to communicate with the monitoring node 130 through the communication network 150 by available connections. These connections are requested by the vehicle 100 when available. Once proper credentials are established, the connection is established enabling bi-directional secure transmission of data using the MQTT protocol.
In one example, the hierarchy is established by a customer using the system (e.g., airline, airport). For example, a customer defines the hierarchy that includes satellite communications for airports A, B, and C. Communications at airports A and B occur through the available satellite communications. However, airport C has spotty coverage through satellites and a connection may not be available. When not available, the vehicle 100 determines that a cellular connection is available and communication occurs through the cellular system. In this instance, the computing device 20 changes from the default satellite communications to cellular communications upon detection of the cellular signal.
The communication network 150 and architecture provide for a single monitoring node 130 to monitor multiple different vehicles 100. Further, the communications network 150 can include additional vehicles 100. A vehicle 100 can transmit and receive data from other vehicles 100 to each other while moving. In the specific application of aircraft, this includes the aircraft transmitting and receiving data during flight from other aircraft.
Communications circuitry 25, 135 provides for communication through the tunnel 160. Communications circuitry 135 of the monitoring node 130 also provides for communication with the nodes 154. Databases 26, 136 store information relevant and include a non-transitory computer readable storage medium (e.g., an electronic, magnetic, optical, electromagnetic, or semiconductor system-based storage device). The databases 26, 136 can be local or remote relative to the respective image processing circuitry 23, 133.
The secure tunnel establishing unit/module 30 comprises instructions that, when executed by the processing circuitry 23, causes the computing device 20 to initiate the establishment of a secure communications tunnel 160 with the monitoring node 130, as previously described. The data obtaining unit/module 31 comprises instructions that, when executed by the processing circuitry 23, causes the computing device 20 to obtain the data to be streamed from the computing device 20 to the monitoring node 130, as previously described. The data translation unit/module 32 comprises instructions that, when executed by the processing circuitry 23, causes the computing device 20 to translate the data obtained by data obtaining unit/module 31 into a message using a publish-and-subscribe protocol, as previously described. The session establishment unit/module 33 comprises instructions that, when executed by the processing circuitry 23, causes the computing device 20 to establish a communication session with the monitoring node 130 over the secure communications tunnel 160, as previously described. The communications unit/module 34 comprises instructions that, when executed by the processing circuitry 23, causes the computing device 20 to transmit the translated message to the monitoring node 130, and to receive data messages from the monitoring node 130, as previously described.
One or more control panels 60 are positioned on the vehicle 100 and provide for onboard monitoring of the captured data. The control panel 60 include a display for displaying the image data for viewing by onboard personnel. One or more input devices such as but not limited to a keyboard and mouse provide for a user to control the control panel. The control panel 60 can also include a browser for accessing the image data and/or controlling the transmission. The control panel 60 can also include a port to input data that is to be transmitted to the computing device 20. In one example, the port provides for input of image data stored on memory cards.
In one example as disclosed above, the vehicle 100 initiates the communications sessions that includes the transfer of data with the monitoring node 130. In another example, the communications provide for the monitoring node 130 to initiate communication and data transfer with the vehicle 100. This communication can be initiated in various manner, including but not limited to at predetermined timed intervals and upon receiving a request from a node 154.
The systems and methods described can be used on a variety of vehicles 100. Vehicles 100 include but are not limited to manned aircraft, unmanned aircraft, manned spacecraft, unmanned spacecraft, manned rotorcraft, unmanned rotorcraft, satellites, rockets, missiles, manned terrestrial vehicles, unmanned terrestrial vehicles, manned surface water borne vehicles, unmanned surface water borne vehicles, manned sub-surface water borne vehicles, unmanned sub-surface water borne vehicles, and combinations thereof.
The present invention may, of course, be carried out in other ways than those specifically set forth herein without departing from essential characteristics of the invention. The present embodiments are to be considered in all respects as illustrative and not restrictive, and all changes coming within the meaning and equivalency range of the appended claims are intended to be embraced therein.
