Patent Grant
11973824
The present application relates to the technical field of communication, and in particular to a method for data transmission of audio and video in an end-to-end system.
The present RTP, RTMP, WEBRTC, http and so on are audio/video frequency transmission technology which is widely used at present time, audio/video frequency transmission technology is promoted by the rise of professional video surveillance field and Online video business, which varies in development degrees in their respective field, compared with the professional surveillance field, the Online video business grew more rapidly.
RTP technology which is widely used in the product of professional video surveillance, owing to the usual deployment of the professional surveillance product in private network, and the smaller interference of private network compared with complex WAN, became the standard transfer protocol in professional surveillance field; RTMP protocol is often used in demand of WAN Video Websites, as the standard protocol of video playback of FLASH plugin (FLASH is eliminated at present time); http protocol is extraordinary obsolescence, but under the boost of the demand to online audio/video, a great amount of video data is transferred by means of file via http protocol, the desired effect is achieved by ceaselessly promoting the performance and the belt width of the server; WEBRTC protocol is the most popular video transfer protocol at present time, which features in point to point, to transfer audio/video streaming in real time.
Surveillance products are mainly used in professional surveillance flied, which consistently use technology like RTP etc. and is totally in different technical route with mainstream video website, since the surveillance products gradually enters personal consumption market, some of the surveillance products in consumer market faces to various incompatible issues, the mainstream solution is using cloud server to storage and forward the data, which satisfies the usage experience of client at present time, however as the products have been ceaselessly marketed, the high cost of video data forwarding became a mainly problem, the surveillance video products are designed to be used in one Private LAN, but the users expect to see videos anywhere. At first, the product shall be able to transfer data in WAN environment. The transfer problem of internet data can be solved by increasing the server, which brings huge cost pressure at the same time, a great amount of data are generated by the surveillance video products every time, forwarding all of them by server, may cause dramatic increase of operation costs, while large delay will be caused when the data is sent after being in storage as files.
Therefore, a solution method for end-to-end system applying to the data transmission of audio/video frequency should be designed, by a self designed software service architecture, the quantity of server is decreased in same condition, the hardware requirement of the server is decreased, so that the operation cost is decreased, the tolerance of whole software system is promoted, the effect of downtime and network anomaly to the system is decreased, total data transmission volume of whole system is reduced by a new transfer mode of message, so that a transmission cost of data is reduced.
The purpose of the present application is to overcome the shortage of prior art, a method for data transmission of audio and video in an end-to-end system, by a self designed software service architecture, the quantity of server is decreased in same condition, the hardware requirement of the server is decreased, so that the operation cost is decreased, the tolerance of whole software system is promoted, the effect of downtime and network anomaly to the system is decreased, total data transmission volume of whole system is reduced by a new transfer mode of message, so that a transfer cost of data is reduced.
To achieve above purpose, the present application provides a method for data transmission of audio and video in an end-to-end system, the end-to-end system includes a client terminal, a device terminal, and a service terminal, the service terminal comprises a management server, a register server and a transmission server, and the method includes steps of:
In some embodiment, the step S1 includes sub-steps of:
In some embodiment, the step S2 includes sub-steps of:
In some embodiment, the S3 includes sub-steps of:
In some embodiment, the S4 includes sub-steps of:
In some embodiment, the S5 includes sub-steps of:
In some embodiment, in the S6, when a forward link established by the transmission server for a client terminal and a device terminal is terminated, releasing, by the transmission server, a source; when the source of the transmission server reaches a low load threshold, sending, by the transmission server as a low load server, a state message to the management server, and transferring, by the management server, a part of device terminals to the low load transmission server.
In some embodiment, the management server adopts a standard website architecture design, which is divided into a front end, a back end and database, the front end is configured for providing a configuration management page to users, the back end is configured for processing a configuration message and a data interaction with other servers, the database is configured for storing device information.
In some embodiment, the register server is configured for providing transmission server association information of corresponding device queried by the client terminal and the device terminal.
In some embodiment, the register server is configured for providing a high concurrency query service and an update service, the high concurrency query service is configured for providing a query function support to the client terminal and the device terminal, the update service is configured for interacting with the management server by using a http protocol to obtain a device list and a transmission server list.
In some embodiment, the transmission server is configured for performing a dynamic adjustment according to number of device terminals, when the transmission server is overload or a large number of transmission servers are idle, the management server is configured for dynamic increasing and decreasing number of the transmission servers via a server operator interface.
In some embodiment, the transmission server has functions of a network penetration, a forwarding service and an update service, the function of update service is the same as that of the register server, which is configured for providing information interaction with the management server.
In some embodiment, the client terminal is configured for providing an external program call in a runtime, and completing a connection establishment and a transmission for audio and video via an API interface of the client terminal.
In some embodiment, the device terminal is configured for providing an external program call in a runtime, completing a connection establishment and a transmission for audio and video via an API interface of the device terminal.
Compared with prior art, by adopting an unique architecture design of the client terminal, the device terminal and the service terminal, the present application make the user only need to call the API interface provided by the client terminal and the device terminal to implement a data transmission of real-time audio/video frequency, as well as configure and manage related device quickly via the management server. By a configuration, the system can adaptively adjust, increase and decrease the server to adapt current business demands according to the load, which has following technical benefits that: the solution decreases the transmission cost of data, which effectively decrease the error caused by server downtime and network problem and has a greater reliability compared with prior solution; by a self-generated adjustment to the serve amount of the load in the system, the cost of idle server is decreased, which may realize a comprehensive system management to the device and reduce maintenance costs.
The present application will be further described in detail in combination with FIGS.
Referring to
The S1 includes following steps:
Referring to
The S3 includes following steps:
Referring to
Referring to
In the S6, when a forward link established for a client terminal and a device terminal by the transmission server stops, releasing a source of the transmission server, under the condition that the source reaches a low load threshold, sending a state message to the management server by the transmission server as a low load server, transferring part of the device terminals to the low load server by the management server, particularly, the part of the device terminals which are transferred are random device terminals.
The present application has high expandability, all servers can be run on one PC hard ware platform when a minimum service is provided, when the amount of the device terminal gradually increases, the servers are gradually registered, the transmission servers are deployed in sequence on the independent PC hardware platform, in combination of hardware upgrade function provided by present cloud service operators such as Amazon and Ali etc., the system can be upgraded very quickly and smoothly.
The management server adopts a standard website architecture design, which is divided into front end, back end and database, the front end provides a configuration management page to the user, the back end completes a configuration message processing and a data processing of the interaction with other server, the database is configured to store device information, such as information of device terminal. The management server runs on the host of the cloud service operator by means of stand-alone program. The data storage adapts mysql database. The manager server provides for user configuration management by means of standard http.
The register server provides information of corresponding equipment queried by the client terminal and the device terminal relating to the transmission server. The related information has large concurrent and short character string. The register server runs on the host of the cloud service operator by means of stand-alone program. The register server doesn't have database, after start, all data of register server are obtained from the management server, and the service is provided after obtaining the data from the management server.
The register server includes a high concurrency query service and an update service. The high concurrency query service provides query function support for the client terminal and device terminal, the service features on high concurrency and short data (84 bytes) of each message; an http protocol is adopted for interaction with management server in the update service, to obtain a device list and a transmission server list. Mainstream http server, compared with high concurrency query service, requires to be in TCP three way handshake for each transmission, in which actual data will increase http protocol header information, every request service should be allocated independent memory for tcp data processing, and memory will be consumed before the connection ends, so that dealing with high concurrency request by means of http service requires to establish a group of service for processing. On the contrary, an udp is adopted in present solution, the service terminal has no need to allocate independent memory for each connection, and the handshake is not needed in udp data transmission. The amount of data transmission has more than 50% reduction, and the performance requirement of server hardware has more than 50% reduction.
A plurality of register servers and management server runs independently, which improves fault tolerance of the whole system. Unless all of the register servers are in downtime, the system can still ensure its normal running Compared with traditional fault tolerance solution which is using backup server, firstly a plurality of register servers separately deployed in multiple locations can effectively decrease the connection delay cause by area difference, and then all register servers backup each other. Besides, even if traditional backup server is not in the operating state, a cost should be paid for it. All register servers in present system are deployed in different areas, which reduces the probability of abnormal running in the system caused by regional network paralysis.
The register server can implement the device information list request to the management server, the state message upload the management server, in which the state message includes information such as a load of register server, a requests-per-second, and a CPU/memory utilization, the reception of a device information list update request from the management server (after the system administrator conducts an operation of adding/deleting/modifying, the management server sends the updated device information to the register server), the reception of device query request from the client terminal and the reception of device query request from the device terminal.
For example, if only used in China, 1 server can be configured, if used in Asia, 2 servers can be configured, if a global data should be loaded, each area such as Asia, Europe, Africa, North and South America can be configured 1 server according to geography, which depends on a central node of global backbone network. More register servers can contribute to a faster response got by nearby device.
Referring to
The transmission server includes a network penetration, a forwarding service and an update service, the function of the update service is the same as the register, which is to provide an information interaction to the management server.
For example, when the device scale is 10, the management server, the register server and the transmission server are deployed on an Alibaba Cloud 1-core 1G cloud server at the same time.
For example, when the device scale is 100, the management server and the register server are deployed on a 1-core 1G cloud server, the transmission server is deployed on a 1-core 1G cloud server.
For example, when the device scale exceeds one million, the management server and the register server deploy on a multiple core large memory sever, the register servers are arranged on global network nodes, the transmission server is dynamically deployed in different areas according to the actual situation.
Referring to
Referring to
The above is only a preferred embodiment in present application, which is used for helping understand the method of present application and its core idea. The protection scope of present application is not limited to above embodiments, all technical solutions belonging to the idea of present application falls into the protection scope of present application. It should be noted that, for those skilled in the art, several improvements and modifications without departing from the principle of the present application, should be considered as the protection scope of present application.
The present application reduces the whole amount of server deployment, which greatly reduces service operation cost and data transmission cost. Compare with present solution, the error caused by server downtime and network problem is reduced, which has higher reliability. By a self-generated adjustment to the serve amount of the load in the system, the cost of idle server is decreased, which may realize a comprehensive system management to the device and reduce maintenance costs.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202111117683.5 | Sep 2021 | CN | national |
The present application is a continuation of PCT application No. PCT/CN2022/120953, filed on Sep. 23, 2022, which claims the priority benefit of a China patent application No. 202111117683.5, filed on Sep. 23, 2021. The entireties of PCT application No. PCT/CN2022/120953 and China patent application No. 202111117683.5 are incorporated herein by reference and made a part of this specification.
| Number | Name | Date | Kind |
|---|---|---|---|
| 10728121 | Chitalia | Jul 2020 | B1 |
| 11522840 | Sillankorva | Dec 2022 | B2 |
| 20030028642 | Agarwal | Feb 2003 | A1 |
| 20030108002 | Chaney | Jun 2003 | A1 |
| 20040103149 | Tanigawa | May 2004 | A1 |
| 20080274739 | Wu | Nov 2008 | A1 |
| 20090180484 | Igarashi | Jul 2009 | A1 |
| 20090232129 | Wong | Sep 2009 | A1 |
| 20090313376 | Cedervall | Dec 2009 | A1 |
| 20100005177 | Cagenius | Jan 2010 | A1 |
| 20100268783 | Mizosoe | Oct 2010 | A1 |
| 20110151864 | Byun | Jun 2011 | A1 |
| 20110191462 | Smith | Aug 2011 | A1 |
| 20130170492 | Lu | Jul 2013 | A1 |
| 20130188519 | Lu | Jul 2013 | A1 |
| 20130201987 | Lu | Aug 2013 | A1 |
| 20130208592 | Lu | Aug 2013 | A1 |
| 20130282798 | McCarthy | Oct 2013 | A1 |
| 20150128199 | Kuo | May 2015 | A1 |
| 20190141121 | Bernat | May 2019 | A1 |
| 20190158544 | Ohneck | May 2019 | A1 |
| 20190278938 | Greene | Sep 2019 | A1 |
| 20190039417 | Hikita et al. | Dec 2019 | A1 |
| 20190394174 | Sillankorva | Dec 2019 | A1 |
| 20200110753 | Silk | Apr 2020 | A1 |
| 20200133827 | Tkac | Apr 2020 | A1 |
| 20200228602 | Spoczynski | Jul 2020 | A1 |
| 20200267053 | Zheng | Aug 2020 | A1 |
| 20200296155 | McGrath | Sep 2020 | A1 |
| 20200366733 | Parvataneni | Nov 2020 | A1 |
| 20200389516 | Parekh | Dec 2020 | A1 |
| 20210194847 | Gu | Jun 2021 | A1 |
| 20210326161 | Son | Oct 2021 | A1 |
| 20220006701 | Patel | Jan 2022 | A1 |
| 20220171647 | Mutha | Jun 2022 | A1 |
| 20220210073 | Bernat | Jun 2022 | A1 |
| 20220255916 | Smith | Aug 2022 | A1 |
| 20220322058 | Etuke | Oct 2022 | A1 |
| 20220330128 | Kim | Oct 2022 | A1 |
| 20220369166 | Soryal | Nov 2022 | A1 |
| 20230144294 | Qiu | May 2023 | A1 |
| Number | Date | Country |
|---|---|---|
| 103442224 | Dec 2013 | CN |
| 103581740 | Feb 2014 | CN |
| 106331115 | Jan 2017 | CN |
| 207820121 | Sep 2018 | CN |
| 113099252 | Jul 2021 | CN |
| 113824723 | Dec 2021 | CN |
| Entry |
|---|
| International Search Report Corresponding to Related Application No. PCT/CN2022/120953; dated Dec. 12, 2022; 4 pgs. |
| Number | Date | Country | |
|---|---|---|---|
| 20230144294 A1 | May 2023 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/CN2022/020953 | Sep 2022 | US |
| Child | 18096142 | US |
