Law enforcement agencies provide officers and agents with an assortment of devices—electronic and otherwise—to carry out duties required of a law enforcement officer. Such devices include radios (in-vehicle and portable), body-worn cameras, weapons (guns, Tasers, clubs, etc.), portable computers, and the like. In addition, vehicles such as cars, motorcycles, bicycles, may be equipped with electronic devices associated with the vehicle, such as vehicle cameras, sirens, beacon lights, spotlights, and personal computers.
It is increasingly common for law enforcement agencies to require officers to activate cameras (body-worn and vehicle-mounted) that enable officers to capture audio and/or video contents of incidents in which an officer is involved. This provides a way to preserve evidence, that would otherwise be unavailable, for subsequent legal proceedings. This evidence greatly aids in the investigation of criminal activities, identification of perpetrators of crimes, and an examination of allegations of police misconduct, to name a few advantages.
It is also desirable to further investigate the incidents based on the captured audio and/or video contents. In a network environment setting where different devices from different places can be used to consume and investigate the captured audio and/or video contents, there is a need to maintain consistent reliability of consumed audio and/or video contents.
The detailed description is described with reference to the accompanying figures, in which the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. The use of the same reference numbers in different figures indicates similar or identical items.
This disclosure is directed to techniques for a network operation center (NOC) server that integrates a model-controller-view (MCV) design pattern with an event streaming platform to support NOC workspace interoperability. The telemetry data streams may include data packet streams of audio content, video content, metadata, virtual reality or augmented reality data, and information that can be encoded in JavaScript Object Notation (JSON), Extensible Markup Language (XML), or other structured data modeling language. The MCV design pattern may include a structure that divides an application into three main logical components (e.g., model component, controller component, and view component) to handle specific aspects of the application (e.g., consumer application). As opposed to a Model-View-Controller (MVC) design pattern, the MCV design pattern includes the controller component that may be directly tied to the model component. In one example, telemetry data streams from heterogeneous sources (e.g., different types of media recording devices) are pushed into an event streaming platform such as Apache Kafka™, and the model component of the application decouples and stores the decoupled telemetry data streams. The controller component that is tied to the model component may then perform a filtering operation on queried decoupled telemetry data streams (sometimes also called models) to dynamically control views to be rendered on at least one display unit of the view component without losing control of continuity of the decoupled telemetry data streams. Alternatively, or in addition, the controller component may dynamically delete the views on some of the display units without affecting an order of telemetry data streams that are decoupled from the event streaming platform. In these cases, the dynamic rendering of views supports NOC workspace interoperability that can be critically important in law enforcement operations at least because they can enable performance of multi-functions without affecting the continuity (sometimes called configuration) of the decoupled telemetry data streams from the event streaming platform. Accordingly, while the model (e.g., queried decoupled telemetry data streams) can be independently filtered by the controller component, the decoupled telemetry data streams in the telemetry data storage may not be affected by filtering operations/functions.
In one example, the multi-functions may include listening to audio content in one display unit and watching video content on another display unit. In another example, the multi-functions may include watching images captured by a specific media recording device in some of the display units. In another example, images from different timestamps and from different media recording device identifications (IDs) may be watched and analyzed on some of the display units. In another example, the images are transformed into different scales or color formats to match a configuration of the display units. In these examples, the controller component that is tied to the model component in the MCV design pattern may perform the filtering operations on the same model to generate filtered telemetry data streams that can be rendered on the display units to support NOC workspace interoperability.
As used herein, the terms “device,” “portable device,” “electronic device,” and “portable electronic device” are used to indicate similar items and may be used interchangeably without affecting the meaning of the context in which they are used. Further, although the terms are used herein in relation to devices associated with law enforcement, it is noted that the subject matter described herein may be applied in other contexts as well, such as in a security system that utilizes multiple cameras and other devices.
The implementation and operations described above ascribed to the use of the server; however, alternative implementations may execute certain operations in conjunction with or wholly within a different element or component of the system(s). Further, the techniques described herein may be implemented in a number of contexts, and several example implementations and context are provided with reference to the following figures. The term “techniques,” as used herein, may refer to system(s), method(s), computer-readable instruction(s), module(s)m algorithms, hardware logic, and/or operation(s) as permitted by the context described above and throughout the document.
Example Architecture
The architecture 100 may include media recording devices 102(1)-102(N) of different types. Each of the media recording devices 102(1)-102(N) may be a video recording device, an audio recording device, or a multimedia recording device that records both video and audio data. The media recording devices 102(1)-102(N) may include recording devices that are worn on the bodies of law enforcement officers, recording devices that are attached to the equipment, e.g., motor vehicles, personal transporters, bicycles, etc., used by the law enforcement officers. For example, a law enforcement officer that is on foot patrol may be wearing the media recording device 102(1). In another example, a patrol vehicle of the law enforcement officer may be equipped with the media recording device 102(2), and so on.
The media recording devices 102(1)-102(N) may be connected to a NOC server 104 through a network 106. The NOC server 104 may be part of a facility that is operated by a law enforcement agency or a facility that is operated by a third-party that is offering services to the law enforcement agency. The NOC server 104 may include web sockets 108(1)-108(N), a receiving queue 110, and a consumer application that includes a model component 120, controller component 130, and a view component 140. The model component 120 may include a data query module 122, telemetry data storage 124, and a model module 126 that includes snapshots of decoupled data streams in the telemetry data storage 124. The controller component 130 may be directly connected to the model component 120 and further includes a filtering module 132 and a monitor selector 134. The view component 140 may receive signals from the controller component 130 and further includes a plurality of monitors 142. The model component 120, controller component 130, and the view component 140 may form the MCV design pattern of the consumer application. Each component or module of the NOC server 104 can be realized in hardware, software, or a combination thereof. For example, the web-sockets 108(1)-108(N) may be implemented by a software module designed to establish communications with the media recording devices 102(1)-102(N), respectively.
The network 106 may be, without limitation, a local area network (“LAN”), a larger network such as a wide area network (“WAN”), a carrier network, or a collection of networks, such as the Internet. Protocols for network communication, such as TCP/IP, may be used to implement the network 106. The network 106 may provide telecommunication and data communication in accordance with one or more technical standards.
Each one of the web-sockets 108(1)-108(N) may include an endpoint of a two-way communication link between two programs running on the network 106. The endpoint includes an Internet Protocol (IP) address and a port number that can function as a destination address of the web-socket. Each one of the web-sockets 108(1)-108(N) is bound to the IP address and the port number to enable entities such as the corresponding media recording device(s) to communicate with the web socket. In one example, the web-sockets 108(1)-108(N) may be set up to receive telemetry data streams from the media recording devices 102(1)-102(N). The received telemetry data streams may be pushed to the queue 110 before they are decoupled and stored in the telemetry data storage 124. In one example, the decoupled telemetry data streams are initially transformed to conform with a schema structure (not shown) in the telemetry data storage 124. The schema structure may include data fields that support sensor formats of the media recording devices 102(1)-102(N).
The queue 110 may include management software that processes data streams to or from the web-sockets. The queue 110 may be implemented by an event streaming platform that supports a publish-subscribe based durable messaging system. The event streaming platform may receive telemetry data streams and store the received telemetry data streams as topics. A topic may include an ordered collection of events that are stored in a durable manner. The topic may be divided into a number of partitions that can store these events in an unchangeable sequence. In this case, the event streaming platform may receive the telemetry data streams, store these telemetry data streams as topics, and different applications may subscribe to receive these topics in the event streaming platform. The subscription to receive the telemetry data streams (topics) may be based upon a time period (e.g., within a certain time range or frequency) or upon detection of an event.
In one example, a plurality of telemetry data streams from the web sockets 108 are pushed into the queue 110 and stored as topics. Given a situation that the telemetry data storage 124 has subscribed to receive these topics based upon detection of an event, then the telemetry data storage 124 may decouple the telemetry data streams that include the detected event. For example, the detected event may include streaming videos and images that were uploaded by a law enforcement officer's camera. Given a situation where the model component 120 subscribed to receive this event upon its detection, then the model component 120 may decouple the telemetry data streams that represent the detected event from the queue 110. The decoupled telemetry data streams may include the event and associated device ID, timestamps, and a header. The decoupled telemetry data streams may be used to gather metrics from different locations, track activity of the media recording devices 102(1)-102(N), gather application logs, and/or investigate incidents in case of law enforcement operations.
The model component 120 may include an application that may perform management and transformation of the decoupled telemetry data streams from the queue 110. In one example, the model component 120 decouples the telemetry data streams from the queue 110 and stores the decoupled telemetry data streams in the telemetry data storage 124. Decoupling of telemetry data streams includes independently retrieving and processing the data streams without affecting the configuration of the source such as the queue 110.
With stored telemetry data streams in the telemetry data storage 124, the data query module 122 may be configured to query snapshots of the decoupled telemetry data streams and store the queried snapshots as models in the model module 126. For example, the queried snapshots from the decoupled telemetry data streams represent images or events that were uploaded from the first media recording device 102(1). In this example, the images or events may be stored as models in the model module 126. The images or events may be associated with the device ID of the first media recording device 102(1), timestamps of the events, and a header such as flags and data length.
In one example, the data query module 122 may separately query the decoupled telemetry data streams that represent a captured audio content from the decoupled telemetry data streams that represent video content. In this case, the model that may represent the audio content can be rendered separately from the model that represents the video content. Further, a media recording device may utilize an application that can represent the captured video content as a sequence of images, and each one of these images may be further subdivided into sections known as tiles. In this manner, the different tiles can be represented independently by different models since each tile can be queried separately from the decoupled telemetry data streams.
The controller component 130 may include an application that is tied to the model component 120. In one example, the controller component 130 utilizes different types of filters to generate different products (filtered telemetry data stream) from the same model. Each one of the generated different products may be rendered separately and independently in the view component 140. For example, a particular model in the model module 126 includes decoupled telemetry data streams that represent 10 sequences of images that were taken at different timestamps and at a different angle of views. In this example, the filtering module 132 may separate and render each one of the images on different screens based on timestamps and/or angle of views, change scale of each one of the images, change color format, and the like. In this manner, the filtering module 132 may dynamically control the views to be rendered in the display units by applying different functions on the same model. The generating of the model includes, for example, querying of stored decoupled telemetry data streams based upon a device ID, timestamp, header, key-value pairs, a configuration of the display units in the view component, or based upon user-entered criteria.
The monitor selector 134 may be configured to select the first monitor 142(1), second monitor 142(2), or a combination thereof, which can render the generated filtered telemetry data stream from the filtering module 132. In one example, a configuration of the monitor as a target destination may be utilized during the filtering of the model, for example, a model that includes a high resolution for which a single display unit (1st monitor 142(1)) cannot show the content. In this example, the image stream can be filtered to split the content for rendering at the 1st monitor 142(1) and the 2nd monitor 142(2). In another example, the display unit is using a color format that is different from the color format that is by the application in the media recording device. In this other example, the filtering of the model may include changing of the color format of the model in the model module 126.
Example Filtering of the Model in an MCV Design Pattern
In one example, the first model 126(1) may include snapshots of the decoupled telemetry data streams that will be rendered in the view component 230. A snapshot may include a data set that is associated with a timestamp or a time period. The snapshot may also include the data set that is associated with a particular media recording device ID. In an implementation, the controller 230 may define the attributes of the model to be queried from the telemetry data storage such as the telemetry data storage 124 of
In one example, multiple controller components 230 may be used to perform filtering functions on the first model 126(1). For example, the first controller component 230(1) may utilize a filtering module 232(1) to access audio data streams (not shown) in each one of the events 202. In this example, the first controller component 230(1) may render the accessed audio data streams in the first monitor 142(1). In another example, the second controller component 230(2) may utilize a filtering module 232(2) to access data streams that represent images within a range of timestamps 208. In this example, the second controller component 230(2) may dynamically render or add views or images that are associated with a range of timestamps 208 in the view component 240. Alternatively, the second controller component 230(2) may dynamically delete rendered views or images that are associated with a range of timestamps 208 in the view component 240. In another example, the controller components 230 may filter the events to be rendered based upon the device ID 206. In this example, controller components 230 may independently render the events that are based on the device ID 206. In these examples, the controller components 230 transform the same first model 126(1) to render different snapshots of this model in the view component 240 without actually controlling the continuity of the decoupled telemetry data streams in the telemetry data storage. In this case, the controller components 230 may support workstation interoperability without altering the configuration of the decoupled telemetry data streams in the telemetry data storage.
In an example embodiment, the rendering of the first model 126(1) may be based upon a configured sensor format (not shown) in each one of the media recording devices 102. A sensor format may include a configuration of the media recording device 102 for collecting telemetry data that can be processed in the NOC server 104. For example, the media recording device 102 utilizes an application that separates data streams for each one of the tiles that form a single image. Upon forming of a model (e.g., first model 126(1)) that may represent the single image, the filtering module 232 may render separately the data streams for each one of the tiles that formed the signal image. In another example, the media recording device 102 utilizes an application that captures the image using a high resolution. In a case where the configuration of a target display unit in the view component 240 is not capable of showing the high-resolution image, then the controller component may scale or filter the high-resolution image to be displayed on different display units. In these examples, the controller components 230 transform the same model that will be rendered in the view component 240 without actually controlling the continuity of the decoupled telemetry data streams in the telemetry data storage 124.
Example NOC Server
The NOC server 300 includes a communication interface 302 that facilitates communication with media recording devices such as the media recording devices 102(1)-102(N). Communication between the NOC server 300 and other electronic devices may utilize any sort of communication protocol known in the art for sending and receiving data and/or voice communications.
The NOC server 300 includes a processor 304 having electronic circuitry that executes instruction code segments by performing basic arithmetic, logical, control, memory, and input/output (I/O) operations specified by the instruction code. The processor 304 can be a product that is commercially available through companies such as Intel® or AMD®, or it can be one that is customized to work with and control a particular system. The processor 304 may be coupled to other hardware components used to carry out device operations. The other hardware components may include one or more user interface hardware components not shown individually—such as a keyboard, a mouse, a display, a microphone, a camera, and/or the like—that support user interaction with the NOC server 300.
The NOC server 300 also includes memory 320 that stores data, executable instructions, modules, components, data structures, etc. The memory 320 may be implemented using computer-readable media. Computer-readable media includes, at least, two types of computer-readable media, namely computer-readable storage media and communications media. Computer-readable storage media includes, but is not limited to, Random Access Memory (RAM), Dynamic Random Access Memory (DRAM), Read-Only Memory (ROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, Compact Disc-Read-Only Memory (CD-ROM), digital versatile disks (DVD), high-definition multimedia/data storage disks, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. As defined herein, computer-readable storage media do not consist of and are not formed exclusively by modulated data signals, such as a carrier wave. In contrast, communication media may embody computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave, or other transmission mechanisms.
A memory controller 322 is stored in the memory 320 of the NOC server 300. The memory controller 322 may include hardware, software, or a combination thereof, that enables the memory 320 to interact with the communication interface 302, processor 304, and other components of the NOC server 300. For example, the memory controller 322 receives data (e.g., audio and video contents) from the communication interface 302 and sends the received data to an MCV app 340 for further processing. In another example, the memory controller 322 may retrieve data from memory 320 where the data will be processed in the processor 304.
The memory 320 stores the MCV app 340 that, when executed, implements filtering of the same models to generate a filtered product that can be rendered in the display units without actually controlling the continuity of the decoupled telemetry data streams from the event streaming platform such as the Apache Kafka™. As shown, the MCV app 340 includes a network or web-sockets 342, a queue loader 344, a controller component 346, a filtering module 348, and a database 350. The database 350 may further include a telemetry data storage 352, a data query module 354, a model module 356, and a monitor selector 358. In one example, each component of the MCV app 340 can be realized in hardware, software, or a combination thereof. For example, the controller component 346 is a combination of hardware and software module that is tied to the database 350 to control the views that will be rendered in the display units (not shown).
The web-sockets 342 are similar to web-sockets 108(1)-108(N) of
In one example, the queue loader 344 may include an application programming interface (API) to establish a connection with the event streaming platform such as the Apache Kafka™. The event streaming platform may utilize logs to store the telemetry data streams from the media recording devices. The logs are immutable records of things or events. The logs may include topics and partitions to store the telemetry data streams. In one example, the model component of the NOC server 300 may subscribe to the topics in the event streaming platform and utilize the queue loader 344 to decouple the telemetry data streams that the NOC server 300 has subscribed to receive. The decoupled telemetry data streams are stored in the telemetry data storage 352.
The controller component 346 is similar to the controller component 130 of
The telemetry data storage 352, data query module 354, and the model module 356 are similar to the telemetry data storage 124, data query module 122, and the model module 126, respectively, of
Further functionalities of the NOC server 300 and its component features are described in greater detail, below.
Example Implementation—Rendering Telemetry Data Streams
At block 402, the queue 110 receives a plurality of telemetry data streams from the media recording devices 102. In one example, the queue 110 includes an event streaming platform that receives data packet streams encoded in JSON, XML or other structured data modeling language. The data packet streams, for example, include audio content, video content, metadata, virtual reality or augmented reality data, and information that may be captured by the media recording devices 102.
At block 404, the model component 120 decouples telemetry data streams that the model component 120 subscribed to receive from the queue 110 (e.g., event streaming platform). For example, model component 120 subscribes to detected events in the plurality of telemetry data streams in the queue 110. The detected events may include, for example, streamed videos and images that were uploaded from a particular media recording device or on a particular timestamp or date. In this example, the model component 120 may decouple the telemetry data streams that represent the detected events from the queue 110. The detected events may be associated with a device ID, timestamps, and headers.
At block 406, the telemetry data storage 124 stores the decoupled telemetry data streams in a telemetry data storage 124 of the model component 120. In some cases, the storing of the telemetry data streams may include conforming with a structure of a universal schema in the telemetry data storage 124. For example, the universal schema includes fields for device IDs, timestamp, event IDs, header, sensor format, and key-value annotations. In this example, the fields of the decoupled telemetry data streams may be identified and stored to conform with the structure of the universal schema.
At block 408, the data query module 122 queries the decoupled telemetry data streams from the telemetry data storage 124 to form a model 126. In one example, the querying is based upon a desired type of filter to be used upon a potential model. For example, the filtering module 132 may apply a device ID-based function on the model in the model module 126 to access certain video content. In this example, the querying of the decoupled telemetry data streams may be based upon the device ID that is associated with certain video content. In another example, the filtering module 132 may apply a function on the model that represents the data streams within a certain time range. In this other example, the querying of the decoupled telemetry data streams may be based upon a certain time range. In other cases, the querying is based upon user-entered parameters.
In one example, the controller 230 may define the attributes of the model to be queried from the telemetry data storage 124. In this implementation, the attributes may be based upon the timestamp 208, device ID 206, and header 204. The timestamp 208 may include a date and time of capture of the event, message, or record. The device ID 206 may include the IP address and/or device description. The header 204 may include information such as number of bytes, flags, and other similar information that is associated with the captured event. For example, the data query module 122 queries the decoupled telemetry data streams to form the first model 126(1) of
At block 410, the controller component 130 filters the formed model to generate filtered telemetry data streams. For example, the controller component 130 utilizes the filtering module 132 to separate audio data streams from video data streams. In another example, the filtering module 132 utilizes a function to access and render views that represent images that were captured within a range of timestamps 208. In this example, the controller component 130 may dynamically render or add the views or images that are associated with the range of timestamps 208. Alternatively, the controller component 130 may dynamically delete rendered views or images that are associated with a range of timestamps 208 in the view component 240. In these examples, the controller component 130 transforms the same model such as the first model 126(1) to render different snapshots of the model in the view component 240 without actually controlling the continuity of the decoupled telemetry data streams in the telemetry data storage 124.
At block 412, the controller component 130 renders the filtered telemetry data streams in a display unit. For example, the controller component 130 utilizes a monitor selector to identify the target screens or display units for the rendering of the filtered telemetry data streams. In this example, the controller component 130 provides workspace interoperability by controlling the same model and not the continuity of the decoupled telemetry data streams from the event streaming platform.
Conclusion
Although the subject matter has been described in language specific to structural features and/or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as exemplary forms of implementing the claims.
Number | Name | Date | Kind |
---|---|---|---|
6760744 | Halaas et al. | Jul 2004 | B1 |
7917888 | Chong et al. | Mar 2011 | B2 |
8452722 | Naeve et al. | May 2013 | B2 |
8606844 | Kaufman et al. | Dec 2013 | B2 |
8688320 | Faenger | Apr 2014 | B2 |
9110774 | Bonn et al. | Aug 2015 | B1 |
9264678 | Nuyttens et al. | Feb 2016 | B2 |
9449229 | Laska | Sep 2016 | B1 |
9483732 | Milakovich | Nov 2016 | B1 |
9485474 | Kim et al. | Nov 2016 | B2 |
9681104 | Billau | Jun 2017 | B2 |
9723251 | Slotky | Aug 2017 | B2 |
9738125 | Brickley et al. | Aug 2017 | B1 |
9755890 | Robertson et al. | Sep 2017 | B2 |
9832205 | Santhi et al. | Nov 2017 | B2 |
9848312 | Sundel et al. | Dec 2017 | B2 |
9852132 | Chhichhia et al. | Dec 2017 | B2 |
9886261 | Hotchkies | Feb 2018 | B1 |
10324773 | Wing | Jun 2019 | B2 |
10460014 | Lloyd et al. | Oct 2019 | B2 |
10540883 | Keil et al. | Jan 2020 | B1 |
10902955 | Federoff et al. | Jan 2021 | B1 |
11238290 | Burns | Feb 2022 | B2 |
20030081127 | Kirmuss | May 2003 | A1 |
20030095688 | Kirmuss | May 2003 | A1 |
20030163512 | Mikamo | Aug 2003 | A1 |
20030208679 | Vazquez | Nov 2003 | A1 |
20060257001 | Veen et al. | Nov 2006 | A1 |
20080147267 | Plante et al. | Jun 2008 | A1 |
20080303903 | Bentley | Dec 2008 | A1 |
20090150017 | Caminiti et al. | Jun 2009 | A1 |
20090210455 | Sarkar et al. | Aug 2009 | A1 |
20090248711 | Martinez et al. | Oct 2009 | A1 |
20090284359 | Huang et al. | Nov 2009 | A1 |
20100036560 | Wright et al. | Feb 2010 | A1 |
20100144318 | Cable | Jun 2010 | A1 |
20110205068 | Huynh et al. | Aug 2011 | A1 |
20110302151 | Abadi et al. | Dec 2011 | A1 |
20120084747 | Chakradhar et al. | Apr 2012 | A1 |
20130039542 | Guzik | Feb 2013 | A1 |
20130344856 | Silver et al. | Dec 2013 | A1 |
20130347005 | Lam et al. | Dec 2013 | A1 |
20140343796 | Abuelsaad et al. | Nov 2014 | A1 |
20150089019 | Chou | Mar 2015 | A1 |
20150341370 | Khan | Nov 2015 | A1 |
20160042767 | Araya | Feb 2016 | A1 |
20160086397 | Phillips | Mar 2016 | A1 |
20160153801 | Cho et al. | Jun 2016 | A1 |
20160190859 | Blum et al. | Jun 2016 | A1 |
20160248856 | Kao | Aug 2016 | A1 |
20160371553 | Farnham, IV et al. | Dec 2016 | A1 |
20160378607 | Kumar et al. | Dec 2016 | A1 |
20170011324 | Truong et al. | Jan 2017 | A1 |
20170048482 | Drako | Feb 2017 | A1 |
20170148027 | Yu et al. | May 2017 | A1 |
20170161323 | Simitsis et al. | Jun 2017 | A1 |
20170161409 | Martin | Jun 2017 | A1 |
20170164062 | Abramov | Jun 2017 | A1 |
20180079413 | Herrero et al. | Mar 2018 | A1 |
20180145923 | Chen et al. | May 2018 | A1 |
20180285759 | Wood et al. | Oct 2018 | A1 |
20180365909 | Cheng et al. | Dec 2018 | A1 |
20190019122 | Allen | Jan 2019 | A1 |
20190026665 | Caskey et al. | Jan 2019 | A1 |
20190043351 | Yang et al. | Feb 2019 | A1 |
20190054925 | Froeschl et al. | Feb 2019 | A1 |
20190140886 | Zywicki et al. | May 2019 | A1 |
20190325354 | Rajnayak et al. | Oct 2019 | A1 |
20200007827 | Saad et al. | Jan 2020 | A1 |
20200072637 | Guidotti et al. | Mar 2020 | A1 |
20200074156 | Janumpally | Mar 2020 | A1 |
20200081899 | Shapur et al. | Mar 2020 | A1 |
20200145620 | Alcantara | May 2020 | A1 |
20200151360 | Zavesky et al. | May 2020 | A1 |
20200172112 | Kawashima | Jun 2020 | A1 |
20200211216 | Hagio et al. | Jul 2020 | A1 |
20200304854 | Baumgartner et al. | Sep 2020 | A1 |
20200312046 | Righi | Oct 2020 | A1 |
20200351381 | Lacey et al. | Nov 2020 | A1 |
20210076002 | Peters et al. | Mar 2021 | A1 |
20210089374 | Watson et al. | Mar 2021 | A1 |
20210133808 | Chan et al. | May 2021 | A1 |
20210272702 | Hakami | Sep 2021 | A1 |
20210297929 | Frusina et al. | Sep 2021 | A1 |
20210377205 | Brown | Dec 2021 | A1 |
20220014907 | Boyd et al. | Jan 2022 | A1 |
20220169258 | Samarthyam et al. | Jun 2022 | A1 |
Number | Date | Country |
---|---|---|
109671266 | Nov 2020 | CN |
2008204219 | Sep 2008 | JP |
20130010400 | Jan 2013 | KR |
20190086134 | Jul 2019 | KR |
2010056891 | May 2010 | WO |
Entry |
---|
U.S. Appl. No. 17/107,877, Office Action dated Aug. 18, 2021, 40 pages. |
U.S. Appl. No. 17/107,891, Final Office Action dated Aug. 5, 2021, 21 pages. |
U.S. Appl. No. 17/107,891, Office Action dated Apr. 1, 2021, 22 pages. |
International Patent Application No. PCT/US2021/060890, International Search Report and Written Opinion dated Mar. 21, 2022, 11 pages. |
International Patent Application No. PCT/US2021/060892, International Search Report and Written Opinion dated Mar. 21, 2022, 10 pages. |
International Patent Application No. PCT/US2021/060893, International Search Report and Written Opinion dated Mar. 21, 2022, 9 pages. |
International Patent Application No. PCT/US2021/060894, International Search Report and Written Opinion dated Mar. 21, 2022, 9 pages. |
International Patent Application No. PCT/US2021/060895, International Search Report and Written Opinion dated Mar. 2021, 9 pages. |
International Patent Application No. PCT/US2021/060896, International Search Report and Written Opinion dated Mar. 14, 2022, 11 pages. |
Juan Rendon et al. Structural combination of neural network models. 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW). IEEE Dec. 12, 2016, pp. 406-413. Section II; and figure 2. |
Massimo Bonavita et al. Machine Learning for Model Error Inference and Correction. Journal of Advances in Modeling Earth Systems. Nov. 13, 2020, pp. 1-22. Section 2.1; and figure 1. |
Md Manjurul Ahsan et al. Deep MLP-CNN Model Using Mixed-Data to Distinguish between COVID-19 and Non-COVID-19 Patients. Symmetry 2020. Sep. 16, 2020, pp. 1-14. Section 2; and figure 3. |
U.S. Appl. No. 17/107,764, Office Action dated Dec. 8, 2021, 38 pages. |
U.S. Appl. No. 17/107,824, Office Action dated Dec. 29, 2021, 30 pages. |
U.S. Appl. No. 17/107,877, Final Office Action dated Dec. 29, 2021, 40 pages. |
U.S. Appl. No. 17/107,891, Notice of Allowance dated Nov. 2, 2021, 23 pages. |
Man Hiep Phung et al. A High-Accuracy Model Average Ensemble of Convolutional Neural Networks for Classification of Cloud Image Patches on Small Datasets. Applied Sciences 2019. Oct. 23, 2019, pp. 1-16. Section 2; and figure 3. |
Xueheng Qiu et al. Ensemble Deep Learning for Regression and Time Series Forecasting. 2014 IEEE Symposium on Computational Intelligence in Ensemble Learning (CIEL). IEEE, Dec. 9, 2014, pp. 1-6. |
U.S. Appl. No. 17/107,708, Office Action dated May 9, 2022, 57 pages. |
U.S. Appl. No. 17/107,824, Notice of Allowance dated May 2, 2022, 34 pages. |
U.S. Appl. No. 17/107,824, Notice of Allowance dated Jun. 7, 2022, 33 pages. |
U.S. Appl. No. 17/107,830, Office Action dated Jun. 7, 2022, 51 pages. |
U.S. Appl. No. 17/107,708, Final Office Action dated Aug. 19, 2022, 52 pages. |
U.S. Appl. No. 17/107,714. Office Action dated Aug. 18, 2022, 66 pages. |
U.S. Appl. No. 17/107,764, Notice of Allowance dated May 26, 2022, 26 pages. |
U.S. Appl. No. 17/107,877, Notice of Allowance dated Aug. 22, 2022, 40 pages. |
Number | Date | Country | |
---|---|---|---|
20220172604 A1 | Jun 2022 | US |