The present disclosure generally relates to the technical field of Internet and, more particularly, relates to a method and a stream-pushing client for processing a live stream in WebRTC.
With continuous development of Internet and terminal devices, real-time video communication is becoming more popular. The web real-time communication (WebRTC) protocol is widely used in application scenarios such as network-based video conferencing, video calling, and the like due to its cross-platform and high real-time characteristics, and the solution for implementing a set of live interaction at a webpage client may become extremely simple by using the WebRTC. After recording a live video stream, a stream-pushing client may push data packets of the live video stream to a resource server through a WebRTC connection; similarly, a subsequent stream-pulling client may acquire the live video stream from the resource server based on the WebRTC connection, thereby implementing the live video function with a low-latency.
However, the copyright issue of the live video stream is increasingly noticeable while the WebRTC brings convenience to users. A large quantity of live video streams may be illegal acquired by criminals during the transmission process and then played on other platforms. Therefore, there is an urgent need for a method which may reflect the copyright of the live video stream.
The purpose of the present disclosure is to provide a method and a stream-pushing client for processing a live stream in WebRTC, which may add corresponding copyright information to the live stream in WebRTC communication.
To achieve the above-mentioned purpose, one aspect of the present disclosure provides a method for processing the live stream in WebRTC which may be applied to the stream-pushing client. The method may include collecting an original live stream, and loading the original live stream into a temporary video tag, thereby playing the original live stream through the temporary video tag; creating a temporary image tag and loading a preset watermark image into the temporary image tag; creating a temporary drawing tag and drawing a current video frame in the temporary video tag and the preset watermark image in the temporary image tag into the temporary drawing tag; and acquiring a canvas video stream corresponding to the temporary drawing tag, and based on the canvas video stream and the original live stream, generating a watermark live stream with one or more added watermarks, wherein the watermark live stream is configured to be transmitted to a resource server through a WebRTC connection.
To achieve the above-mentioned purpose, another aspect of the present disclosure provides a stream-pushing client. The stream-pushing client may include an original live stream collecting unit, configured to collect an original live stream and load the original live stream into a temporary video tag, thereby playing the original live stream through the temporary video tag; a watermark image loading unit, configured to create a temporary image tag and load a preset watermark image into the temporary image tag; an image drawing unit, configured to create a temporary drawing tag and draw a current video frame in the temporary video tag and the preset watermark image in the temporary image tag into the temporary drawing tag; and a watermark live stream generating unit, configured to acquire a canvas video stream corresponding to the temporary drawing tag and generate a watermark live stream with one or more added watermarks based on the canvas video stream and the original live stream, wherein the watermark live stream may be configured to be transmitted to the resource server through a WebRTC connection.
To achieve the above-mentioned purpose, another aspect of the present disclosure provides a stream-pushing client. The stream-pushing client includes a memory and a processor, wherein the memory is configured to store a computer program, and the computer program is configured to implement above-mentioned methods when executing by the processor.
It may be seen from the above that, in the technical solutions provided in the present disclosure, the stream-pushing client may load the original live stream into the pre-created temporary video tag after collecting the original live stream, and the temporary video tag may play the original live stream automatically. In the present disclosure, the copyright of the original live stream may be displayed by the manner of adding the watermark image into the original live stream. Specifically, the preset watermark image may be loaded to the temporary image tag by creating the temporary image tag; subsequently, based on the created temporary drawing tag, the current video frame and the preset watermark image of the original video stream may be respectively acquired from the temporary video tag and the temporary image tag, and the current video frame and the preset watermark image may be sequentially drawn in the temporary drawing tag, such that for each video picture frame, the corresponding preset watermark picture may be added. In such way, the video frames with one or more added watermarks image may be continuously generated in the temporary drawing tag, and the video frames may form the canvas video stream. Finally, the watermark live stream with one or more added watermarks may be generated based on the canvas video stream and the original live stream, and the watermark live stream may be configured to be transmitted to the resource server through the WebRTC connection. In such way, the live video stream subsequently transmitted on the network may carry the preset watermark image, thereby reflecting the copyright information of the live video stream.
In order to clearly illustrate the technical solutions in the embodiments of the present disclosure, the accompanying drawings, which are required to be used in the description of disclosed embodiments, are briefly described hereinafter. Obviously, the accompanying drawings in the following description are merely certain embodiments of the present disclosure. Other accompanying drawings derived from such accompanying drawings may be acquired by those skilled in the art without creative work.
To more clearly describe the objectives, technical solutions and advantages of the present disclosure, the embodiments of the present disclosure are further illustrated in detail with reference to the accompanying drawings hereinafter.
The present disclosure provides a method for processing a live stream in WebRTC, which may be applied to a stream-pushing client. The stream-pushing client may be an electronic device having image and voice collection functions used by a video host. The electronic devices may be, for example, a smart phone, a tablet computer, a notebook computer, a personal computer, a smart wearable device (virtual reality glasses and a smart watch), and the like. Obviously, the stream-pushing client may also be software running in the above-mentioned electronic devices. For example, the stream-pushing clients may be an application program (app), a browser, a software development kit (SDK), and the like.
Referring to
Referring to
At S1, an original live stream may be collected, and the original live stream may be loaded into a temporary video tag, thereby playing the original live stream through the temporary video tag.
With reference to
After collecting the original live stream, the temporary video tag may be created in the stream-pushing client and be configured to play the original live stream. Specifically, when the temporary video tag is created, the name of the temporary video tag may be specified. For example, the name of the temporary video tag may be TV. Furthermore, various play attributes of the temporary video tag may also be configured. For example, since the processing of the original video stream is actually performed in backend, the picture and audio should not be displayed on webpage frontend and the play volume of the temporary video tag may be adjusted to 0, thereby mutely playing the original live stream. Meanwhile, the temporary video tag may also be configured to be invisible (hidden), such that a user may not directly view the playing original live stream from outside. In addition, the play mode of the temporary video tag may also be configured as auto play; once the temporary video tag is loaded with the original live stream, the original live stream may be played automatically, and there is no need for the host to trigger a play button. Obviously, in practical applications, in order to prevent the temporary video tag from displaying abnormally when playing the original live stream, the size information of the original video stream may also be read, and the read size information may be configured to the temporary video tag synchronically. The size information may be, for example, the width and height of the video picture in the original live stream, and both the width and the height may be expressed by the quantity of pixels. After the above-mentioned configuration, the collected original live stream may be mutely played in the temporary video tag synchronically.
At S3, a temporary image tag may be created, and a preset watermark image may be loaded into the temporary image tag.
In one embodiment, the copyright information of the host may be marked in the original live stream by the manner of embedding the preset watermark image. The preset watermark image may be configured by the host or may be randomly generated by the stream-pushing client according to the information of the host. The preset watermark image may have its own loading address. The loading address may be, for example, the uniform resource locator (URL) of the image or the storage path of the image in the stream-pushing client. In such way, the temporary image tag may be created in the stream-pushing client and configured to load the preset watermark image. Specifically, the loading address of the preset watermark image may be associated with the temporary image tag, which may enable the temporary image tag to load the preset watermark image according to the loading address.
When creating the temporary image tag, the name of the temporary image tag may be specified. For example, the name of the temporary image tag may be TIM. In addition, the image display attribute of the temporary image tag may be configured. For example, the display size of the preset watermark image and the position information of the preset watermark image in the video picture may be configured in the temporary image tag, where the display size may be expressed by a width and a height, and both the width and the height may be expressed by the quantity of pixels. The position information may indicate a coordinate values corresponding to the vertex of an upper left corner when the preset watermark image is displayed in the video screen. The coordinate values may be determined in the coordinate system of the video picture and include an abscissa value and an ordinate value, and both the abscissa value and the ordinate value may be expressed by the sequence number of pixels. For example, the width and the height of the video picture of the original live stream are TCW and TCH, respectively. The coordinate values of the vertex of the upper left corner of the video picture of the original live stream may be (0,0), such that the coordinate values of the vertex of the lower right corner may be (TCW-1, TCH-1). In such coordinate system, the coordinate values corresponding to the vertex of the upper left corner of the preset watermark image may be identified, such that the coordinate values may be configured as the position information of the preset watermark image in the video picture. Therefore, an area occupied by the preset watermark image may be uniquely determined in the video picture according to the position information and the display size.
At S5, a temporary drawing tag may be created, and a current video frame in the temporary video tag and the preset watermark image in the temporary image tag may be drawn in the temporary drawing tag.
In one embodiment, after the original live stream and the preset watermark image are played and displayed through respective tags, a corresponding preset watermark image may be added for each frame of the video picture of the original live stream. Specifically, the temporary drawing tag may be created in the stream-pushing client, and the role of the temporary drawing tag may be used to sequentially draw the video frame and the preset watermark image of the original live stream. In such way, the video frame of the original live stream may be located in the lower layer of the temporary drawing tag, and the preset watermark image may be located in the upper layer of the temporary drawing tag, thereby generating the visual effect of covering the video frame with the preset watermark image.
In one embodiment, in order to draw the video frame and the preset watermark image sequentially, a preset instruction for returning a drawing context in the temporary drawing tag may be called. The preset instruction may be, for example, a getContext( ) instruction which may return a drawing context used on a canvas. To use the instruction normally, a drawing type is required to be specified on the canvas. Specifically, in one embodiment, the drawing type may be a 2d planar picture, so that the drawing type may be expressed by a parameter indicating the planar picture; for example, the parameter may be ā2dā. In such way, after the parameter indicating the planar picture is passed to the preset instruction, a planar picture rendering object CanvasRenderingContext2D corresponding to the temporary video tag may be acquired. In an application example, the planar picture rendering object may be expressed as the following manner:
TCC=TC.getContext (ā2dā),
where TCC may represent the name of the planar picture rendering object, TC may represent the name of the temporary drawing tag, and 2d may represent the parameter indicating the planar picture.
In one embodiment, after the planar picture rendering object is acquired, the current video frame and the preset watermark image may be respectively acquired from the temporary video tag and the temporary image tag through the planar image rendering object, and the acquired current video frame and the acquired preset watermark image may be drawn into the temporary drawing tag. In an application example, the planar picture rendering object may sequentially draw the current video frame and the preset watermark image into the temporary drawing tag through a drawing instruction drawImage. Specifically, the image drawing may be performed by the drawing instruction of the planar picture rendering object in the following manner:
TCC.drawImage (TV, 0, 0, TCW, TCH),
TCC.drawImage (TIM, WX, WY, WW, WH),
where, TV may represent the name of the temporary video tag; (0,0) may represent the coordinate values of the vertex of the upper left corner of the current video frame; TCW and TCH may respectively represent the width and height of the current video frame; TIM may represent the name of the temporary image tag; (WX,WY) may represent the coordinate values of the upper left corner vertex of the preset watermark image in the coordinate system of the current video frame; and WW and WH may respectively represent the width and height of the preset watermark image.
It can be seen that the temporary video tag TV, the size information (TCW and TCH) of the video frame in the original video stream, and the starting position (0,0) of the video frame of the video frame in the original video stream may be marked in the planar picture rendering object, thereby acquiring the current video frame from the temporary video tag. Similarly, the temporary image tag TIM, the size information (WW and WH) of the preset watermark image, and the starting position (WX and WY) of the preset watermark image may be marked in the planar picture rendering object, thereby acquiring the preset watermark image from the temporary image tag. Finally, the acquired current video frame and the preset watermark image may be sequentially drawn into the temporary drawing tag, thereby forming the current video frame covered with the preset watermark image.
Referring to
MT.postMessage (params),
where, postMessage may represent the instruction indicating the watermark addition, and params may represent parameters carried in the instruction.
After the instruction is received, the preset script MTJS may execute the preset service logic edited in advance in response to the instruction indicating the watermark addition, and after executing the preset service logic, the preset script MTJS may return notification information to the multi-threading. The notification information may indicate that the preset script MTJS has currently executed and completed its own service logic, and the multi-threading may start the process of adding the watermark. In such way, in response to the notification information, the multi-threading MT may draw the current video frame in the temporary video tag and the preset watermark image in the temporary image tag into the temporary drawing tag.
At S7, a canvas video stream corresponding to the temporary drawing tag may be acquired, a watermark live stream with one or more added watermarks may be generated based on the canvas video stream and the original live stream, and the watermark live stream may be configured to be transmitted to the resource server through the WebRTC connection.
In one embodiment, the video frames with one or more added watermarks images may be continuously generated in the temporary drawing tag according to the above-mentioned manner, and the video frames may form the canvas video stream. The video picture may only be processed in the temporary drawing tag and the corresponding audio information may not be added. Therefore, the canvas video stream and the audio information of the original live stream may be combined to form the watermark live stream with audio and visual information. Specifically, an audio track may be extracted from the original live stream, a video track may be extracted from the canvas video stream, and then the audio track and the video track may be merged to generate the watermark live stream with one or more added watermarks. In an application example, the audio track of the original live stream may be acquired by the following instruction:
AT=OS.getAudioTracks( )[0]
where, AT may represent the extracted audio track, OS may represent the name of the original live stream, getAudioTracks( ) may represent the instruction to extract the audio track, and 0 may represent the first available audio track.
Furthermore, the video track of the canvas video stream may be extracted by the following instruction:
VT=CS.getVideoTracks( )[0]
where, VT may represent the extracted video track, CS may represent the name of the canvas video stream, getVideoTracks( ) may represent the instruction to extract the video track, and 0 may represent the first available video track.
After the video track and the audio track are extracted, the audio track and the video track may be merged into the watermark live stream using a MediaStream instruction provided by the browser according to the following manner:
WS=MediaStream ([AT, VT])
where, WS may represent the name of the merged watermark live stream.
In one embodiment, after the watermark live stream is generated, the watermark live stream may be pushed to the resource server through the WebRTC connection. Specifically, referring to
In one embodiment, after the watermark live stream is generated, the stream-pushing client may display the watermark live stream in the display region shown in
The present disclosure further provides a stream-pushing client, and the stream-pushing client may include:
an original live stream collecting unit, configured to collect an original live stream and load the original live stream into a temporary video tag, thereby playing the original live stream through the temporary video tag;
a watermark image loading unit, configured to create a temporary image tag and load a preset watermark image into the temporary image tag;
an image drawing unit, configured to create a temporary drawing tag and draw a current video frame in the temporary video tag and the preset watermark image in the temporary image tag into the temporary drawing tag; and
a watermark live stream generating unit, configured to acquire a canvas video stream corresponding to the temporary drawing tag, and generate a watermark live stream with one or more added watermarks based on the canvas video stream and the original live stream, where the watermark live stream may be configured to be transmitted to the resource server through a WebRTC connection.
In one embodiment, the watermark image loading unit may include:
an address association module, configured to associate a loading address of the preset watermark image with the temporary image tag, which enables the temporary image tag to load the preset watermark image according to the loading address.
In one embodiment, the image drawing unit may include:
a webpage work object calling module, configured to call a webpage work object provided by the browser and create a multi-threading through the webpage work object; and
an instruction passing module, configured to start the multi-threading and pass the instruction indicating the watermark addition to the preset script through the multi-threading, where after the preset script executes a preset service logic in response to the instruction indicating the watermark addition, the preset script may return notification information to the multi-threading, such that in response to the notification information, the multi-threading may be enabled to draw the current video frame in the temporary video tag and the preset watermark image in the temporary image tag into the temporary drawing tag.
In one embodiment, the image drawing unit may include:
a planar image rendering object acquiring module, configured to call a preset instruction for returning a drawing context in the temporary drawing tag and pass the parameter indicating the planar picture to the preset instruction to acquire the planar picture rendering object corresponding to the temporary drawing tag; and
a drawing module, configured to respectively acquire the current video frame and the preset watermark image from the temporary video tag and the temporary image tag through the planar image rendering object, and draw the acquired current video frame and the preset watermark image into the temporary drawing tag.
In one embodiment, the image drawing unit may include:
a video information marking module, configured to mark the temporary video tag, the size information of the video frame in the original video stream, and the starting position information of the video frame in the original video stream in the planar picture rendering object, thereby acquiring the current video frame from the temporary video tag; and
an image information marking module, configured to mark the temporary image tag, the size information of the preset watermark image, and the starting position information of the preset watermark image in the planar picture rendering object, thereby acquiring the preset watermark image from the temporary image tag.
Referring to
In one embodiment, the processor may include a central processing unit (CPU) or a GPU, and obviously, may further other units which have logic processing capability, such as a single-chip microcomputer, a logic gate circuit, an integrated circuit, or other suitable combinations. The memory in one embodiment may be a storage device for storing information. In digital systems, a device which may store binary data may be the memory; in integrated circuits, a circuit with a storage function without physical form may also be the memory, such as RAM, FIFO, and the like; and in systems, a storage device with a physical form may also be called as the memory, and the like. The memory may also be implemented in a cloud storage manner during implementation, and the specific implementation manner may not be limited in the present disclosure.
It should be noted that, the specific implementation manner of the stream-pushing client in the present disclosure may refer to the method implementation manners, which may not be described in detail herein.
It may be seen from the above that, in the technical solutions provided in the present disclosure, the stream-pushing client may load the original live stream into the pre-created temporary video tag after collecting the original live stream, and the temporary video tag may play the original live stream automatically. In the present disclosure, the copyright of the original live stream may be displayed by the manner of adding the watermark image into the original live stream. Specifically, the preset watermark image may be loaded to the temporary image tag by creating the temporary image tag; subsequently, based on the created temporary drawing tag, the current video frame and the preset watermark image of the original video stream may be respectively acquired from the temporary video tag and the temporary image tag sequentially, and the current video frame and the preset watermark image may be sequentially drawn into the temporary drawing tag, such that the corresponding preset watermark picture may be added for each video picture frame. In such way, the video frames with one or more added watermarks images may be continuously generated in the temporary drawing tag, and the video frames may form the canvas video stream. Finally, the watermark live stream with one or more added watermarks may be generated based on the canvas video stream and the original live stream, and the watermark live stream may be pushed to the resource server through the WebRTC connection. Therefore, the live video stream subsequently transmitted on the network may carry the preset watermark image, thereby reflecting the copyright information of the live video stream.
Through the description of the above embodiments, those skilled in the art may clearly understand that each embodiment may be implemented by means of software and necessary universal hardware platforms, and obviously, may also be implemented by hardware. Based on such understanding, the above-mentioned technical solutions or portions of the solutions, which contribute to the existing technology, may be embodied in the form of software products essentially, which may be stored in a computer-readable storage medium, such as a ROM/RAM, a magnetic disc, an optical disc, and the like, and may include a plurality of instructions for enabling a computer device (which may be a personal computer, a server, or a network device, and the like) to perform the methods described in various embodiments or certain portions of the embodiments.
The above-mentioned embodiments may merely be preferred embodiments of the present disclosure and may not be intended to limit the present disclosure. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present disclosure should be included in the protection scope of the present disclosure.
Number | Date | Country | Kind |
---|---|---|---|
201811385153.7 | Nov 2018 | CN | national |
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/CN2018/120099 | 12/10/2018 | WO | 00 |