Patent Application
20220334706
This application claims the priority benefit of Chinese application serial no. 202110405132.2, filed on Apr. 15, 2021. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
Technical Field
The disclosure relates to an audio visual equipment and broadcasting method, in particular to a live broadcasting recording equipment, a live broadcasting recording system and a live broadcasting recording method.
Description of Related Art
With the increasing demand for remote video services such as remote teaching, video conferences, and online speeches, how to enrich the user experience of video operations is one of the main development directions in this field. However, general remote video services can only provide simple image capturing functions, such as capturing a user's speech while standing in front of a presentation, or capturing a real-time facial image of the user facing the camera, for example. In other words, general remote video services can only provide simple and boring video content to viewer equipment. In view of this, several embodiment solutions regarding how to provide diversified and favorable user experience video effects will be put forward below.
The information disclosed in this Background section is only for enhancement of understanding of the background of the described technology and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Further, the information disclosed in the Background section does not mean that one or more problems to be resolved by one or more embodiments of the disclosure were acknowledged by a person of ordinary skill in the art.
The disclosure provides a live broadcasting recording equipment, a live broadcasting recording system, and a live broadcasting recording method, capable of providing real-time, interactive, and immersive live images.
Other objectives, features, and advantages of the disclosure will be further understood from the further technological features disclosed by the embodiments of the disclosure.
In order to achieve one or part or all of the above objectives or other objectives, the live broadcasting recording equipment of the disclosure includes a camera, a processing device, and a terminal device. The camera is configured to capture images to provide photographic data. The processing device is coupled to the camera and is configured to execute background removal processing on the photographic data to generate a person image. The terminal device communicates with the processing device and has a display. The processing device executes multi-layer processing to fuse the person image, a three-dimensional virtual reality background image, an augmented reality object image, and a presentation image, and generate a composite image. After an application gateway of the processing device recognizes a login operation of the terminal device, the processing device outputs the composite image to the terminal device, so that the display of the terminal device displays the composite image.
In order to achieve one or part or all of the above objectives or other objectives, the live broadcasting recording system of the disclosure includes a live broadcasting recording equipment and a viewer equipment. The live broadcasting recording equipment includes a camera, a processing device, and a terminal device. The camera is configured to capture images to provide photographic data. The processing device is coupled to the camera and is configured to execute background removal processing on the photographic data to generate a person image. The terminal device communicates with the processing device and has a display. The viewer equipment communicates with the live broadcasting recording equipment. The processing device executes multi-layer processing to fuse the person image, a three-dimensional virtual reality background image, an augmented reality object image, and a presentation image, and generate a composite image. After an application gateway of the processing device recognizes a login operation of the terminal device, the processing device outputs the composite image to the terminal device, so that the display of the terminal device displays the composite image. The processing device provides the composite image to the viewer equipment for display.
In order to achieve one or part or all of the above objectives or other objectives, the live broadcasting recording method of the disclosure includes the following steps. Photographic data is provided by a camera of a live broadcasting recording equipment through capturing images. A person image is generated by the processing device of the live broadcasting recording equipment through executing background removal processing on the photographic data. Multi-layer processing is executed by the processing device to fuse the person image, a three-dimensional virtual reality background image, an augmented reality object image, and a presentation image, and a composite image is generated. Communication is performed with the processing device by the terminal device, and the composite image is output to the terminal device by the processing device after an application gateway of the processing device recognizes a login operation of the terminal device, so that the display of the terminal device displays the composite image. The composite image is provided to a viewer equipment by the processing device for display.
Based on the above, the live broadcasting recording equipment, the live broadcasting recording system, and the live broadcasting recording method may instantly fuse the person image, the three-dimensional virtual reality background image, the augmented reality object image, and the presentation image into the composite image, so that the viewer equipment may obtain real-time, interactive, and immersive live broadcasting images.
To make the aforementioned more comprehensible, several embodiments accompanied with drawings are described in detail as follows.
Other objectives, features, and advantages of the disclosure will be further understood from the further technological features disclosed by the embodiments of the disclosure wherein there are shown and described preferred of this disclosure, simply by way of illustration of modes best suited to carry out the disclosure.
The accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate exemplary embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure.
It is to be understood that other embodiment may be utilized and structural changes may be made without departing from the scope of the disclosure. Also, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless limited otherwise, the terms “connected,” “Coupled,” and “mounted,” and variations thereof herein are used broadly and encompass direct and indirect connections, couplings, and mountings.
The content of the disclosure and other technical content, features, and effects will be clearly presented in the following detailed description of a preferred embodiment with reference to the accompanying drawings. The directional terms mentioned in the following embodiment, such as: up, down, left, right, front, or back, etc., are only the directions with reference to the drawings. Therefore, the directional terms used are used to illustrate and not to limit the disclosure.
In order to make the content of the disclosure more comprehensible, embodiments in which the disclosure may be implemented are listed as follows. In addition, wherever possible, elements/components/steps with the same reference numerals in the drawings and embodiments represent the same or similar components.
According to this embodiment, the processing host 111 may be, for example, a desktop computer, a personal computer (PC), or a tablet PC, etc., and the processing host 111 is a device with image synthesis function, and is not particularly limited in the disclosure. The processing device 1111 may include a central processing unit (CPU) with image data processing and computing functions, or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image processing unit (IPU), graphics processing unit (GPU), programmable controller, application specific integrated circuits (ASIC), programmable logic device (PLD), other similar processing devices or a combination thereof.
According to this embodiment, the storage device 1112 may store multiple programs and algorithms to perform image processing and live broadcasting recording operations according to embodiments of the disclosure. The programs may include, for example, a presentation program, an augmented reality (AR) program, a virtual reality (VR) program, a system setting program, a background execution program, a video playback program, a video conference program, and relevant image data, modules, and file data according to embodiments of the disclosure, but the disclosure is not limited thereto. The processing device 1111 may access and execute the relevant programs and data of the storage device 1112 to realize live broadcasting recording function according to the embodiments of the disclosure.
According to this embodiment, the terminal device 113 may be, for example, a smartphone, a tablet PC, or other portable devices, having the display 1131. The terminal device 113 may communicate with the processing host 111 in wired or wireless manner to transmit and receive image data and control commands, etc. According to this embodiment, the viewer equipment 130 may be, for example, a desktop computer, a personal computer, a smartphone or a tablet PC, having a display. The viewer equipment 130 and the processing device 1111 may execute an interactive video conference program for conference communication. The viewer equipment 130 may also upload the viewer image to the cloud equipment 120, and then transfer the viewer image to the terminal device 113 of the live broadcasting recording equipment 110, so that the terminal device 113 may provide a real-time viewer image to realize video conference interactive function.
In step S230, the processing device 1111 may execute multi-layer processing to fuse the person image 303, a three-dimensional virtual reality background image 301, an augmented reality object image 304, and a presentation image 302, and generate a composite image 305. According to this embodiment, the processing device 1111 may execute the virtual reality program, the augmented reality program, and the presentation program (such as a PowerPoint presentation program) to generate the three-dimensional virtual reality background image 301, the augmented reality object image 304, and the presentation image 302. In addition, the processing device 1111 may execute the multi-layer processing to fuse the person image 303, the three-dimensional virtual reality background image 301, the augmented reality object image 304, and the presentation image 302 to generate the composite image 305 as shown in
In step S240, the terminal device 113 may communicate with the processing device 1111, and after an application gateway of the processing device 1111 recognizes a login operation of the terminal device 113, the processing device 1111 may output the composite image 305 to the terminal device 113, so that the display 1131 of the terminal device 113 may display the composite image 305. According to this embodiment, the user may, for example, hold the terminal device 113 in hand, so that during live broadcasting recording process, current image content of the composite image 305 may be monitored, and the image content of the composite image 305 may be adjusted in real time. In step S250, the processing device 1111 may provide the composite image 305 to the viewer equipment 130 for display. According to this embodiment, the processing device 1111 and the viewer equipment 130 may, for example, execute the video conference program to conduct a video conference. In addition, the processing device 1111 may provide the composite image 305 to the viewer equipment 130, so that real-time, interactive, and immersive live broadcasting images may be viewed on the viewer equipment 130.
In addition, according to some embodiments of the disclosure, the viewer equipment 130 may obtain a real-time viewer image by the viewer camera 140. The viewer equipment 130 may return the viewer image to the processing device 1111, so that the display 1131 of the terminal device 113 may display the viewer image. In this way, the live broadcasting recording system 100 may provide the video conference function with real-time interactive effects. It should be noted that in a context of use, the composite image 305 and the viewer image may be displayed on the display 1131 of the terminal device 113 at the same time, so that the user may monitor current image content of the composite image 305 and viewing status of the viewer, and may instantly adjust the image content of the composite image 305 or share information with the viewer in real-time interaction.
In addition, according to some other embodiments of the disclosure, the cloud equipment 120 may also be implemented as a cloud service management platform, to provide such as recording, bypass live broadcasting, video on demand, content analysis, real-time communication (RTC) or edge node service (ENS) and other multimedia functions, or be configured to manage multiple video interactive platforms.
Specifically, when the user 411 operates the terminal device 412, an application gateway 422 of the processing host 413 may recognize a login operation of the terminal device 412, and execute correspondingly a presentation service module 423, an augmented reality service module 424, a background execution service module 425, and a layer service module 426 according to the operation of the user 411 on the terminal device 412. The presentation service module 423, the augmented reality service module 424, and the background execution service module 425 may respectively access the databases 427 to 429 to obtain the presentation image, the augmented reality object image, and the three-dimensional virtual reality background image. The presentation service module 423, the augmented reality service module 424, and the layer service module 426 may input the presentation image, the augmented reality object image, and the three-dimensional virtual reality background image to the presentation program 431, the augmented reality program 433, and the background execution program 434 for execution and use. The background image removal program 436 may obtain the user image provided by the camera 414, and execute the background removal processing on the user image to generate the person image. According to this embodiment, the multi-layer combination program 435 may perform the multi-layer processing to fuse the person image, the three-dimensional virtual reality background image, the augmented reality object image, and the presentation image to generate the composite image according to an image stacking sequence preset by the layer service module 426. In addition, the multi-layer combination program 435 may output the composite image to a file compression program 437 for file encryption and file compression. The file compression program 437 may output an encrypted and compressed composite image to an external cloud device 440 or a viewer equipment 450. Alternatively, according to some embodiments of the disclosure, the file compression program 437 may also store the encrypted and compressed composite image to a portable hard drive or a computer hard drive.
In addition, according to some other embodiments of the disclosure, in the processing core portion 420 of the processing host 413, command line control and configuration, split screen switching and audio source access on viewer side, system configuration, system restart, composite image resolution configuration, video upload, video storage, automatic upload to the cloud equipment, reporting service operation status, service operation status, system operation logs, broadcast services, cloud synchronization and other operations and their corresponding modules may also be executed, and are not particularly limited in the disclosure.
According to this embodiment, the terminal device 412 may output a control signal to a processing device of the processing host 413 according to at least one of operation results of a presentation operation menu, an augmented reality operation menu, a background selection menu, and a system setting menu in a user interface displayed by the display. The processing device of the processing host 413 may execute the multi-layer processing according to the control signal. According to this embodiment, the user may execute the system setting menu by the user interface of the terminal device 412, and the user interface may display a stack layout selection page, a viewer screen page, or a network setting page. However, examples of the presentation operation menu, the augmented reality operation menu, the background selection menu, and the system setting menu will be illustrated by the following multiple embodiments.
With reference to
With reference to
With reference to
In addition, in some contexts of operating of the disclosure, the user may execute network settings by the terminal device 412. The display of the terminal device 412 may display the network setting page to allow the user to operate the terminal device 412 for communication settings, or set up a communication connection with the processing host 413.
With reference to
With reference to
In summary, the live broadcasting recording equipment, the live broadcasting recording system, and the live broadcasting recording method of the disclosure may instantly utilize at least one of the person image, the three-dimensional virtual reality background image, the augmented reality object image, the presentation image, and video image to generate the composite image, and the composite image may be instantly provided to the viewer equipment for display or storage to the cloud equipment, personal hard drive or computer hard drive, etc. The live broadcasting recording equipment, the live broadcasting recording system, and the live broadcasting recording method of the disclosure may provide a portable terminal equipment for the user to operate, so that the user may easily adjust the content of the composite image. The live broadcasting recording equipment, the live broadcasting recording system, and the live broadcasting recording method may display the viewer image in real time by the display of the terminal equipment to achieve video interaction.
The foregoing description of the preferred embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form or to exemplary embodiments disclosed. Accordingly, the foregoing description should be regarded as illustrative rather than restrictive. Obviously, many modifications and variations will be apparent to practitioners skilled in this art. The embodiments are chosen and described in order to best explain the principles of the invention and its best mode practical application, thereby to enable persons skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the particular use or implementation contemplated. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents in which all terms are meant in their broadest reasonable sense unless otherwise indicated. Therefore, the term “the invention”, “the present invention” or the like does not necessarily limit the claim scope to a specific embodiment, and the reference to particularly preferred exemplary embodiments of the invention does not imply a limitation on the invention, and no such limitation is to be inferred. The invention is limited only by the spirit and scope of the appended claims. Moreover, these claims may refer to use “first”, “second”, etc. following with noun or element. Such terms should be understood as a nomenclature and should not be construed as giving the limitation on the number of the elements modified by such nomenclature unless specific number has been given. The abstract of the disclosure is provided to comply with the rules requiring an abstract, which will allow a searcher to quickly ascertain the subject matter of the technical disclosure of any patent issued from this disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Any advantages and benefits described may not apply to all embodiments of the invention. It should be appreciated that variations may be made in the embodiments described by persons skilled in the art without departing from the scope of the present invention as defined by the following claims. Moreover, no element and component in the present disclosure is intended to be dedicated to the public regardless of whether the element or component is explicitly recited in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202110405132.2 | Apr 2021 | CN | national |
