Patent Application
20240119736
The technology disclosed herein relates to combining video feeds from multiple video cameras into a single video stream that can be efficiently routed to various destinations, utilizing bandwidth optimization features for efficiency.
The issue of visual fatigue is an apparent consequence of viewing and monitoring data or footage without supplemental technology. A person viewing a video display may overlook specific essential details within the displayed field of view. With eyestrain, operators may lose sight of discrete exchanges or other angles of the video stream because they are hyper-fixated on one subject or action. The present invention is a more technical, holistic approach, as it does not merely produce video streams but a pre-programmable filtration method that can also analyze frames, as seen in highlighted areas of interest. This can be used in military environments, wherein an area of interest is a line of demarcation, such as a border, or other relevant, protected territory. The ability to alert to abnormal activity in a region of interest, trigger alerts, and capture clear, detail-oriented frames of the occurrence is a feature of origination in the present invention. For example, this feature would be immensely valuable in military surveillance applications.
The invention disclosed herein describes a means of combining video feeds from multiple video cameras into a single video stream that can be efficiently routed to various destinations, utilizing bandwidth optimization features for efficiency. The concatenated stream combines metadata relating to the source camera and the destination video display. The system implements AI, video analytics, and algorithms to trigger alerts on identified conditions and notify users. The invention enables sophisticated multicamera-multisite autonomous monitoring capabilities and provides a more dynamic and rich viewer experience in surveillance applications. The remote video monitoring and transmission system flourish in various industries, including but not limited to civil aviation remote Air Traffic Control (ATC) systems, apron monitoring systems, traffic command centers, military environments, and any destination wherein remote video monitoring is implemented and required.
The present invention describes a system and method for managing transportation hub assets by utilizing discrete intelligent video streams with optimized bandwidth and enhanced selectivity. Currently, remote video monitoring has been used in various fields and sectors. The purpose of remote monitoring is to surveil the remote situations through the local computer using network systems, remote sensing cameras, and other auxiliary equipment. Parts of the images and sounds are recorded to provide convenient reference material and an essential basis for handling incidents in airport environments. The remote video surveillance system has vast reach through standard telephone lines, networks, mobile broadbands, ISDN data lines, or direct connections. It also administers the pan, tilt, and lens of video surveillance images and preserves imagery.
The present invention offers a solution to transmit video information collected by one or more local cameras to the remote end in real-time, thus effectively utilizing the transmission bandwidth. It should be noted that the parameters that control the bandwidth are resolution, frame rate, color depth, color model and the selected area of interest (AOI), a mouse designated smaller area within the field of view, and other parameters relative to a codec that controls compression. The following system and methods remedy other issues, such as poor or reduced image resolution due to limited bandwidth or the problem of overburdening bandwidth resources with unnecessary or extraneous video footage.
The disclosed invention introduces several solutions for optimizing video bandwidth. Through AI video analysis, a user can utilize artificial intelligence to identify authorized activities, support multiple command centers, and utilize interactive and collaborative viewing functions. In short, the approach allows a user to designate an area to be monitored and the rules that trigger upon violation. As a result, the system responds in accordance with a pre-defined plan. For example, the system could be configured to notify users when a geofenced area is encroached. This area can be specified with a mouse click or through the network API definition. Additional customizations, like identifying markers such as abnormal weight, or other characteristics, may also be implemented at the user's discretion. Users can define rules for each camera and specify the transmission image resolution. Users can also specify an interval after which an alarm is automatically reenabled following a violation. The system supports real-time automatic alarms for multiple users at the remote end through the manifestations of sound, light, video, and screenshots. These novel camera settings enable more effective surveillance options with more efficient bandwidth utilization.
By way of example, some definable rules may include but are not limited to the following: unauthorized intrusion, recognition and tracking of specific moving objects, collision warnings, speed warnings and failures to follow established routes, alerts on designated targets, the position of the observation target changes. When a defined rule is violated, the corresponding actions taken by the system may include, but are not limited to: adjustments to resolution and refresh rates of the video transmissions with respect to presets, alerts to the pre-set remote user by marking or acousto-optic signals on the corresponding screen of the super frame, and automatic replays of transgressions.
The system's architecture uses a super frame to transmit video on a digital network. Encoding, transcoding, and decoding individual streams, especially multiple streams, can require extensive computational resources. In the aforementioned system, each camera streams image frames to a computer that compresses the stream for transmission across the network using individual bandwidth optimization settings for the respective camera. Users are also allowed the option to stream content in a lower bandwidth (“Quiet Surveillance”) mode, which varies compression settings, or a higher bandwidth (“Alerted/Zoom”) mode, which adjusts the camera's settings for higher resolution, frame rates, the field of view, full color and other elements. By combining streams into a larger super frame, the issue of encoding and decoding can be streamlined. Integration and encoding of multiple videos into one super frame for transmission enables a user to carry out high-level monitoring with optimal bandwidth. The super frame uses a grid layout, for example, a nine-square grid, to intuitively display real-time thumbnails of multiple videos on one screen. The thumbnails are transmitted at lower resolutions to conserve bandwidth and clickable to greater resolutions to provide greater detail as needed by the user. Each channel of the displayed video is rooted in the principle of effective use of resources and user preferences to institute the best video format. Each camera's video stream enables customization relative to resolution, frame rate, color depth, the overall or partial field of view, size, and positioning on the super frame. The architecture allows more user control over how recorded views are presented, combining individual views from the super frame in the desired arrangement.
As previously mentioned, the present invention is suitable for use in military environments for surveillance applications. With visual fatigue, operators may lose sight of discrete exchanges or other areas of interest that may be valuable to grasp. The present invention is a more technical approach to the problem. It does not merely produce video streams but a pre-programmable image processing method that can analyze frames and highlight areas of interest.
Local sites can forward footage to other remote locations in military applications, where an area of interest is a line of demarcation, such as a border, or other relevant, protected territory. The ability to alert to abnormal activity in a region of interest, trigger alerts, and capture clear, detail-oriented imagery of the occurrence is a feature of the present invention. For example, when a person or unauthorized object crosses a line of demarcation, a set of customizable, pre-set responses are triggered. The technology of the present invention also enables monitoring through an analysis of a set of characteristics, not limited to weight, height, size, and other markers. Having a local site that stores, encodes, and archives video streams that can be pushed to different destinations for surveillance is valuable in military operations, as centers can now communicate, exchange and monitor views and data without the concern of visual and individual fatigue. Centers can also access more curated feeds, which can help filter out obstruction, pinpoint anomalous activity, and ensure maximum efficiency and reporting.
When an automatic alarm or mouse click event is triggered, the next higher resolution real-time stream initiates, thus allowing viewers to use super frames to monitor large areas and increase the resolution of the displayed video as prompted. This feature aids transmission efficiency and reduces the workload associated with remote monitoring personnel. This process also streamlines the transmission of parts of each video frame that are points of interest to the user, as it ignores the transmission of the unrelated or non-selected elements. Users can request a sequence of scales and copies of source imagery from within the super frame to combine into a new mosaic to view. All streams that are sent are a transcoding of the original super frame. Each can combine all or any subset of the subframes, typically individual camera views or regions of interest, of the original, a key feature in bandwidth savings. As envisaged, there are also video recording DVR controls, bookmarks, and collaborative monitoring. In addition, it supports local backup of videos at full resolution and full refresh rate, as well as remote retrieval and playback. The footage from each camera is cached on the local site and can be paused, played forward and backward, and bookmarked at the remote end. This is all accomplished using a small collaboration window, which features drop-down boxes for sharing bookmarks and chat alerts with participants in multiple remote command centers.
Other features and aspects of the invention will become apparent from the following detailed description, taken in conjunction with the accompanying drawings, which illustrate, by way of example, the features in accordance with embodiments of the invention. The summary is not intended to limit the scope of the invention, which is defined solely by the claims attached hereto.
The various embodiments are illustrated by way of example, and not by way of limitation in the figures of the accompanying drawings. Having thus described the invention in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and wherein:
Before describing the invention in detail, it is useful to describe an exemplary environment in which it can be implemented.
One such example is that of a major metropolitan airport, an example of which is shown in
The local computer constructs each super frame video stream by formatting each of the cameras' feeds per the requirements for the destination and adding metadata as indicated in the camera's settings page 204. The local computer then concatenates all the formatted video feeds for a specific destination and compresses the video super frame stream for transport over digital networks. The system can be configured to identify particular trigger criteria to invoke an alert mode 205. Alerts can be generated by algorithms that apply artificial intelligence and change detection to alert viewers to abnormal conditions. The systems can also be configured for geofenced area monitoring such that when the system detects a change in the selected area 109, an alert is generated. For example, AI can validate moving objects using a library of authorized images; path prediction of moving objects could be applied to alert of potential collision; vehicle speeds and routes could be monitored.
The system can also be configured to invoke a quiet mode 206 in the absence of any triggering event. The destination settings page for each camera contains separate parameter settings for quiet mode and one or more alert modes. The local computer constructs the super frame video stream using the parameters specified for the current mode. The mode can change on the fly depending on whether a triggering event, algorithm, or user command invokes a change in mode.
A computer at the local site manages each camera's settings and ingests its video 204. As the stream is received, AI and image analysis algorithms are applied to trigger an alert on identified conditions, notify viewers of abnormal conditions, and optionally switch into a higher fidelity (“Alert/Zoom”) mode. The system can be configured to use a visual memory bank To alert users upon a “first time sighting” of a specific type of objects which merit a much higher degree of scrutiny. For example, a collection of trucks images would enable an AI engine to identify an object as a “truck.” Once an object has been classified, a historical archive of related object types can be searched for a match. If a match is found, statistics can be collected for future analytics. If the object is not found, a new entry is created, and an alert is issued for a “first time seen”.
Metadata is superimposed on each video frame, and the stream is recorded. Because video streams contain compressed data and are variable length, a synchronized companion file is generated that indexes the file for random access retrieval using metadata, time stamps, and any identified descriptors identified by the AI and analytic algorithms.
Quiet mode 206 may be used to conserve bandwidth or other system resources by varying settings for compression, resolution, frame rate, area of interest, color depth, and others. This system also allows a viewer or programmatic method to designate only a portion of the video frame, defined as the area of interest (AOI) to be transmitted 207. For example, a viewer can indicate an AOI by selecting a portion of the displayed image using a mouse interface 207. Transmitting only a portion of the frame can be done to conserve bandwidth. It can also be done to provide a zoomed-in image.
Alert mode, sometimes called zoom mode, can stream higher image fidelity by changing the camera's settings for higher resolution, frame rate, full FOV, full color, and so on. This allows the system to conserve resources by providing high-resolution images only when needed since a surveillance system typically spends most of the time in the untriggered quiet mode.
The system allows viewers to switch between modes when investigating events 208. Viewers can use DVR controls (pause, play, back. forward, bookmark) or add frame-accurate voiceover commentary to the metadata associated with the video stream.
A “chat” capability is provided for intra-site communication 209.
Because the video from each camera is cached at the local site, it is available for immediate pause, rewind, play forward, and bookmarking. The “DVR Live” capability provides immediate access to the recorded steam as it is being recorded. This enables viewers to pause, skip back, apply analytics (zoom in, watch in slow motion, apply algorithm(s)), jump forward, or back to live. Further, the DVR Live mode can be shared in real-time with other destinations assisted by the chat function as needed. The DVR Live function also supports Frame grabs and extracting clips as the video progresses and allows the extracted content to be routed (also in real time) to other network destinations. Viewers at the local site can collaborate and share bookmarks and chat alerts with participants at remote command centers. This feature also allows those in different bases of operation to exchange information with more immediacy, and in some instances, with more specifications depending on the course of action.
Additional requests can include higher resolution, faster frame rate, expanded view, etc. The local site computer 305 responds with a video selected 313 from the archive 306 and sends it to the request destination 309. Viewers at various destinations can communicate with each other using the systems chat 314 capabilities. As a result, viewers using the systems chat 314 may use these features and forward information, data, footage, or commentary to different bases of operation.
The system implements several bandwidth conserving techniques. The first is the SuperFrame manager function as shown in
Another technique to improve the efficient routing of video streams is to combine multiple camera views into one (or more) “Poster Frames” defined for each destination. Poster Frames allow a single encoded stream to transport multiple individually optimized camera views instead of multiple camera streams.
While various embodiments of the disclosed technology have been described above, it should be understood that they have been presented by way of example only, and not of limitation. Likewise, the various diagrams may depict an example architectural or other configuration for the disclosed technology, which is done to aid in understanding the features and functionality that may be included in the disclosed technology. The disclosed technology is not restricted to the illustrated example architectures or configurations, but the desired features may be implemented using a variety of alternative architectures and configurations. Indeed, it will be apparent to one of skill in the art how alternative functional, logical or physical partitioning and configurations may be implemented to implement the desired features of the technology disclosed herein. Also, a multitude of different constituent module names other than those depicted herein may be applied to the various partitions. Additionally, with regard to flow diagrams, operational descriptions and method claims, the order in which the steps are presented herein shall not mandate that various embodiments be implemented to perform the recited functionality in the same order unless the context dictates otherwise.
Although the disclosed technology is described above in terms of various exemplary embodiments and implementations, it should be understood that the various features, aspects and functionality described in one or more of the individual embodiments are not limited in their applicability to the particular embodiment with which they are described, but instead may be applied, alone or in various combinations, to one or more of the other embodiments of the disclosed technology, whether or not such embodiments are described and whether or not such features are presented as being a part of a described embodiment. Thus, the breadth and scope of the technology disclosed herein should not be limited by any of the above-described exemplary embodiments.
Terms and phrases used in this document, and variations thereof, unless otherwise expressly stated, should be construed as open ended as opposed to limiting. As examples of the foregoing: the term “including” should be read as meaning “including, without limitation” or the like; the term “example” is used to provide exemplary instances of the item in discussion, not an exhaustive or limiting list thereof; the terms “a” or “an” should be read as meaning “at least one,” “one or more” or the like; and adjectives such as “conventional,” “traditional,” “normal,” “standard,” “known” and terms of similar meaning should not be construed as limiting the item described to a given time period or to an item available as of a given time, but instead should be read to encompass conventional, traditional, normal, or standard technologies that may be available or known now or at any time in the future. Likewise, where this document refers to technologies that would be apparent or known to one of ordinary skill in the art, such technologies encompass those apparent or known to the skilled artisan now or at any time in the future.
This application claims the benefit of U.S. Provisional Patent Application Ser. No. 63/389,816, filed Jul. 15, 2022, which is incorporated herein in its entirety.
| Number | Date | Country | |
|---|---|---|---|
| 63389816 | Jul 2022 | US |
