Patent Application
20230315265
This application claims the priority benefit of Chinese application no. 202210331000.4, filed on Mar. 30, 2022. The entirety of the above-mentioned patent application is hereby incorporated by reference herein and made a part of this specification.
The invention relates to an audio-visual data editing system and a method therefor. In particular, the invention relates to a multi-resource editing system and a multi-resource editing method.
With increasing demands for remote video services such as distance teaching, video conferences, and online speech, how to improve the convenience for a user to edit an image scenario before conducting the remote video is one of the main development directions in the related fields. However, a general image scenario editing system can be adapted for only editing a single file, for example, for only inputting data having a specific file format for editing. In addition, the image scenario cannot be edited independently in various scenes at different timings. Therefore, how to realize the independent and fast editing on data having different file formats, so as to achieve independent editing on various scenes and obtain a synthetic audio-visual scenario including multiple file formats to be subsequently applied to related image playback, is currently a research direction in the related fields.
The information disclosed in this Background section is only for enhancement of understanding of the background of the described technology and therefore it may contain information that does not form the prior art that is already known to a person of ordinary skill in the art. Further, the information disclosed in the Background section does not mean that one or more problems to be resolved by one or more embodiments of the invention was acknowledged by a person of ordinary skill in the art.
The invention provides a multi-resource editing system and a multi-resource editing method, in which independent editing and processing on a plurality of input data and various scenes can be realized.
Other objectives and advantages according to the embodiments of the invention may be further understood from the technical features disclosed in the embodiments of the invention.
To achieve one, some, or all of the above objectives or other objectives, a multi-resource editing system comprises an editor, a user interface and a broadcasting and recording apparatus. The editor comprises an image editing module. The user interface electrically connects the image editing module. The user interface comprises a plurality of windows. The user interface generates at least one editing signal, the at least one editing signal is generated according to an operation result of the user interface, and the image editing module generates at least one editing instruction according to the at least one editing signal and generates a synthetic image scenario according to the at least one editing signal. The image editing module outputs the synthetic image scenario to the broadcasting and recording apparatus such that the broadcasting and recording apparatus generates a broadcast synthetic image according to a recorded image and the synthetic image scenario.
In an embodiment of the invention, the image editing module comprises an input signal detection module and a control and management module. The input signal detection module is configured to receive the at least one editing signal, and the control and management module is configured to generate the at least one editing instruction according to the at least one editing signal. The editor comprises a resource input module and a control and management module. The resource input module is configured to obtain input data and decode the input data to generate a plurality of input resources, wherein the input data has at least one file format, and the image editing module is coupled to the resource input module. The image editing module is configured to edit the input resources according to the at least one editing instruction and the plurality of editing resources and generate a plurality of processed scenes, each of the scenes comprising at least one layer, wherein the image editing module is configured to perform multi-scene independent processing to combine the at least one layer, the input resources, and the editing resources in each of the scenes before being processed to form the processed scenes, and generate the synthetic image scenario according to the processed scenes.
In an embodiment of the invention, the user interface comprises a scene and resource display window, an asset library window, a monitoring window, a control window and a display window, wherein the at least one editing signal is configured to be generated according to at least one operation result of the scene and resource display window, the asset library window, the monitoring window, the control window and the display window.
In an embodiment of the invention, the scene and resource display window includes a multi-scene selection page and a multi-resource display page. The multi-scene selection page is configured to display the scenes. The multi-resource display page is configured to display the input resources. The input signal detection module is configured to receive the at least one editing signal generated by the multi-scene selection page and the multi-resource display page. The control and management module is configured to generate corresponding the at least one editing instruction, and perform the multi-scene independent processing on the corresponding scene among the scenes according to the at least one editing instruction.
In an embodiment of the invention, the asset library window includes an asset selection page, an editing tool selection page, and an asset and tool switching page. The editing resources includes a plurality of assets and a plurality of editing tools. The asset selection page is configured to display the assets. The editing tool selection page is configured to display the editing tools. The asset and tool switching page is configured to switch between the asset selection page and the editing tool selection page.
In an embodiment of the invention, the image editing module further includes a resource reading module. The resource reading module is configured to identify and obtain at least one specific function of the input resources, and transmit the at least one specific function to the control and management module. The monitoring window includes a specific function selection page configured to display the at least one specific function of the input resources.
In an embodiment of the invention, the display window includes a real-time editing screen and is configured to display one of the scenes under process. The image editing module further includes a scene control module. The scene control module is configured to adjust a quantity and a sequence of the scenes according to the at least one editing instruction.
In an embodiment of the invention, the control window includes an output page. The control and management module transmits a reminding signal to the control window according to a successful archival signal being received. The output page is configured to display a reminding screen corresponding to the reminding signal.
In an embodiment of the invention, the input resources include at least one of an image resource, a model resource, a virtual reality background resource, an augmented reality object resource, an audio-visual resource, an audio resource, and a presentation resource.
In an embodiment of the invention, the image editing module generates the corresponding at least one layer according to the input resources, such that the image editing module edits the corresponding at least one layer according to the at least one editing instruction.
In an embodiment of the invention, the broadcasting and recording apparatus electrically connects to a camera, wherein the recorded image is captured by the camera.
To achieve one, some, or all of the above objectives or other objectives, a multi-resource editing method according to an embodiment of the invention includes the following. A plurality of input data are obtained and decoded by a resource input module of an editor to generate a plurality of input resources. The input data has at least one file format. The input resources are edited according to at least one editing instruction and a plurality of editing resources and a plurality of processed scenes are generated by an image editing module of the editor. Each of the scenes includes at least one layer. The image editing module performs multi-scene independent processing to combine the at least one layer, the input resources, and the editing resources in each of the scenes before being processed to form the processed scenes. A synthetic image scenario is generated according to the processed scenes.
In an embodiment of the invention, the multi-resource editing method further includes the following. The synthetic image scenario is output to a broadcasting and recording apparatus by the editor. A broadcast synthetic image is generated by the broadcasting and recording apparatus according to a recorded image and the synthetic image scenario.
In an embodiment of the invention, the multi-resource editing method further includes the following. At least one editing signal is received by an input signal detection module from an input device. The at least one editing signal is generated according to an operation result of at least part of a scene and resource display window, an asset library window, a monitoring window, a control window, and a display window of a user interface. The at least one editing instruction is generated by a control and management module according to the at least one editing signal to perform the multi-scene independent processing.
In an embodiment of the invention, the multi-scene independent processing includes the following. A multi-scene selection page and a multi-resource display page are displayed by the user interface, and the at least one editing signal generated by the multi-scene selection page and the multi-resource display page are received and transmitted to the control and management module by the input signal detection module of the image editing module when the scene and resource display window of the user interface is executed, such that the control and management module generates corresponding the at least one editing instruction according to the at least one editing signal being received, and performs the multi-scene independent processing on the corresponding scene among the scenes according to the at least one editing instruction.
In an embodiment of the invention, the editing resources include a plurality of assets and a plurality of editing tools. The multi-scene independent processing includes the following. An asset selection page, an editing tool selection page, and an asset and tool switching page are displayed by the asset library window of the user interface when the asset library window is executed. The asset selection page is configured to display the assets. The editing tool selection page is configured to display the editing tools. The asset and tool switching page is configured to switch between the asset selection page and the editing tool selection page.
In an embodiment of the invention, the multi-resource editing method further includes the following. At least one specific function of the input resources are identified and obtained by a resource reading module. The at least one specific function is transmitted to the control and management module. The multi-scene independent processing includes the following. A specific function selection page is displayed by the monitoring window of the user interface to display the at least one specific function of the input resources when the monitoring window is executed.
In an embodiment of the invention, the multi-scene independent processing includes the following. A real-time editing screen is displayed by the display window of the user interface, and a quantity and a sequence of the scenes are adjusted by a scene control module according to the at least one editing instruction when the display window is executed.
In an embodiment of the invention, the multi-scene independent processing includes the following. An output page is displayed by the control window of the user interface, and a reminding signal is transmitted to the control window by the control and management module according to a successful archival signal being received to display a reminding screen corresponding to the reminding signal by the output page when the control window is executed.
In an embodiment of the invention, the input resources include at least one of an image resource, a model resource, a virtual reality background resource, an augmented reality object resource, an audio-visual resource, an audio resource, and a presentation resource.
In an embodiment of the invention, the image editing module generates the corresponding at least one layer according to the input resources, such that the image editing module edits the corresponding at least one layer according to the at least one editing instruction.
Based on the foregoing, in the multi-resource editing system and the multi-resource editing method according to the embodiments of the invention, the multi-scene independent processing is performed on different input data and various scenes by the resource input module and the image editing module to process a plurality of input data having different file formats at the same time and independently edit a plurality of scenes to generate the synthetic audio-visual scenario.
Other objectives, features and advantages of the present invention will be further understood from the further technological features disclosed by the embodiments of the present invention wherein there are shown and described preferred embodiments of this invention, simply by way of illustration of modes best suited to carry out the invention.
To make the aforementioned more comprehensible, several embodiments accompanied with drawings are described in detail as follows.
The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.
It is to be understood that other embodiment may be utilized and structural changes may be made without departing from the scope of the present invention. Also, it is to be understood that the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless limited otherwise, the terms “connected,” “coupled,” and “mounted,” and variations thereof herein are used broadly and encompass direct and indirect connections, couplings, and mountings.
The multi-resource editing system 100 may also include a broadcasting and recording apparatus 200 coupled to the editor 110. The editor 110 is configured to provide the synthetic image scenario 101 to the broadcasting and recording apparatus 200 in a form of streaming data. The broadcasting and recording apparatus 200 is configured to display the synthetic image scenario 101 on a display 210 of the broadcasting and recording apparatus 200. In an embodiment, the editor 110 may also include a transceiver (not shown in the figure) and an input device (not shown in the figure). The resource input module 111 may obtain input data through the input device or the transceiver. The broadcasting and recording apparatus 200 may also include a transceiver (not shown in the figure). The transceiver of the editor 110 and the transceiver of the broadcasting and recording apparatus 200 may perform wired or wireless communication, for example, and may also perform low noise amplification, impedance matching, frequency mixing, frequency up-conversion or down-conversion, filtering, amplification, and similar operations. The transceiver is a user interface for receiving input data, for example.
In this embodiment, the multi-resource editing system 100 may be embodied as a processing host, for example, a desktop computer, a PC, a tablet PC, a cloud server, or the like, and may have functions such as image processing, image computing, and image synthesis, which is not limited by the invention. In an embodiment, the editor 110 and the broadcasting and recording apparatus 200 may respectively be two processing hosts disposed independently, while in another embodiment, the editor 110 and the broadcasting and recording apparatus 200 may each be disposed in one processing host, which may be adjusted depending on the actual requirements, and the invention is not limited thereto. The editor 110 may include relevant computing chips and/or computing circuits, for example, a central processing unit (CPU) with image data processing and computing functions, or any other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), image processing unit (IPU), graphics processing unit (GPU), programmable controller, application specific integrated circuit (ASIC), programmable logic device (PLD), any other similar processing device, or a combination of the devices above. Moreover, the processing host may also include a storage medium. The storage medium is, for example, any form of fixed or removable non-volatile memory (NVM), such as random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid state drive (SSD), or similar elements or a combination of the elements above. The storage medium may store a plurality of modules (e.g., the resource input module 111 and the image editing module 112), a plurality of programs, and algorithms executable by the editor 110 to perform resource inputting, scene editing, and scenario production according to the embodiments of the invention. The programs may include, but are not limited to, a presentation program, an augmented reality (AR) program, a virtual reality (VR) program, a system setting program, a background execution program, a video player program, a video conference program, and relevant image data, a plurality of modules, and file data according to the embodiments of the invention.
With reference to
In step S220, the editor 110′ may edit the input resource 301 by the image editing module 112 according to at least one editing instruction and a plurality of editing resources. The editing resources may include an asset resource, an editing tool, a model resource, a scene template resource, and the like. The editing tool may include tools for dragging, rotating, scaling, and changing colors for pictures and text. For example, the user inputs an editing signal to the image editing module 112 through an input device (not shown in the figure) or a transceiver (not shown in the figure), and the image editing module 112 generates corresponding the at least one editing instruction according to the at least one editing signal input by the user. The at least one editing signal corresponds to, for example, recording, clicking, dragging, rotating, scaling operation, and other editing instruction. The image editing module 112 edits the corresponding input resource 301 according to the at least one editing instruction and the editing resources during multi-scene independent processing to combine at least one layer, the input resource 301, and the editing resources in each scene before being processed to form a processed scene. Specifically, during the multi-scene independent processing, each scene is independent of each other, and the at least one editing signal input by the user only acts on the corresponding scene and does not affect other scenes. In addition, each scene includes at least one layer.
In an embodiment, if the scene template is a scene template resource preset in the editing resources, the plurality of layers in the scene template corresponds to the plurality of input resources 301. For example, different input resources 301 may be respectively configured in different layers, or the same input resources 301 may be configured in different layers. Moreover, the plurality of layers are edited according to at least one editing signal to achieve independent editing on settings and information of each input resource 301 in each scene. In another embodiment, if the scene template is a scene template resource of the input resource 301, the image editing module 112 may generate the corresponding layer in each scene before being processed according to the scene template of the input resource 301 to edit the corresponding layer according to the at least one editing instruction. In this embodiment, the multi-resource editing system 100′ can edit a plurality of input resources 301 at the same time, and place the input resources 301 in different forms in different layers to achieve multi-layer independent editing. In an embodiment, since each scene may include at least two of layers corresponding to virtual stereoscopy (3D), planar pictures, text, and images, the synthetic image scenario 101 may include virtual stereoscopic (3D) animations, plane pictures, text, and images at the same time.
In step S230, the editor 110′ may generate the synthetic image scenario 101 by the image editing module 112 according to the processed scene. The synthetic image scenario 101 includes, for example, edited scenes, the sequence of scenes, and audio-visual information. Accordingly, the multi-resource editing system 100′ can process a plurality of input resources 301 and independently edit each scene.
In this embodiment, the multi-resource editing system 100′ also includes the broadcasting and recording apparatus 200. The editor 110′ outputs the synthetic image scenario 101 through the transceiver to the broadcasting and recording apparatus 200, so that the broadcasting and recording apparatus 200 generates a broadcast synthetic image 202 according to a recorded image 201 and the synthetic image scenario 101. The recorded image 201 is, for example, captured by a camera (not shown in the figure) of the broadcasting and recording apparatus 200. Accordingly, the user may edit the sequence of scenes and the settings of audio-visual playback in advance.
With reference to
In an embodiment, the user interface is, for example, configured in the image editing module 112 of the editor 110′. The user interface is configured to generate at least one editing signal. In the other embodiment, the user interface is configured to generate at least one editing signal. A user interface 401 includes a scene and resource display window 401_1, a control window 401_2, a display window 401_3, a monitoring window 401_4, and an asset library window 401_5. The at least one editing signal is generated according to an operation result of at least part of the scene and resource display window 401_1, the control window 401_2, the display window 401_3, the monitoring window 401_4, and the asset library window 401_5. For example, the input signal detection module 1122 is configured to receive the at least one editing signal input by the user, and transmit the at least one editing signal to the control and management module 1121. The control and management module 1121 generates corresponding the at least one editing instruction according to the at least one editing signal, and transmits corresponding the at least one editing instruction to the resource reading module 1123 or the scene control module 1124 corresponding to the at least one editing instruction for processing of resource reading, resource adjustment, and scene control.
In this embodiment, the scene and resource display window 401_1 includes a multi-scene selection page 401_11 and a multi-resource display page (not shown in the figure). The multi-scene selection page 401_11 is configured to display each scene. For example, the multi-scene selection page 401_11 in
In this embodiment, the asset library window 401_5 includes an asset selection page 401_51, an editing tool selection page (not shown in the figure), and an asset and tool switching page (not shown in the figure). The editing resources include, for example, assets and editing tools. The asset selection page 401_51 is configured to display each asset. The editing tool selection page is configured to display the editing tools. The asset and tool switching page is configured to switch between the asset selection page and the editing tool selection page. The editing resources may be pre-stored in the storage medium (not shown in the figure), the resource input module 111, or the image editing module 112 of the multi-resource editing system 100′, and may also be imported or loaded from a cloud server (not shown) through the transceiver into the multi-resource editing system 100′.
In this embodiment, the resource reading module 1123 is configured to identify and obtain specific functions of the plurality of input resources, and transmit the specific functions to the control and management module 1121. The monitoring window 401_4 includes a specific function selection page (not shown in the figure) and is configured to display the specific functions corresponding to the input resource 301. For example, the input signal detection module 1122 detects the operation result on the user interface, and the operation result is inspecting the specific functions of the input resource 301. The specific functions are, for example, an explosion operation and a cross-section operation of a stereoscopic model. Next, the input signal detection module 1122 transmits corresponding the at least one editing signal to the control and management module 1121. The control and management module 1121 generates the at least one editing instruction according to the at least one editing signal, and transmits the at least one editing instruction to the resource reading module 1123. Accordingly, the control and management module 1121 receives the specific functions corresponding to the input resource 301 from the resource reading module 1123, and displays the specific functions in the monitoring window 401_4 for clicking and viewing by the user. For example, when the user clicks on a model resource (the input resource 301), the monitoring window 401_4 displays the explosion operation and the cross-section operation (the specific functions) of the corresponding model.
In this embodiment, the display window 401_3 includes a real-time editing screen 401_31 and is configured to display one of the scenes under process. More specifically, the real-time editing screen 401_31 is configured to display the currently edited scene. The scene control module 1124 is configured to perform the multi-scene independent processing, such as adjustment of the quantity or the sequence of scenes, scene duplication, and scene deletion, according to the at least one editing instruction. For example, the input signal detection module 1122 transmits the at least one editing signal corresponding to the operations of the user to the control and management module 1121. Next, the control and management module 1121 edits the corresponding scene according to the at least one editing signal, and transmits the corresponding editing result signal to the scene control module 1124. Accordingly, the scene control module 1124 displays the content corresponding to the editing result signal in the real-time editing screen 401_31, so that the multi-resource editing system 100′ visualizes the operation of the user and displays the operation of the user in the display window 401_3 in real time, improving efficiency in editing by the user and improving user satisfaction for the user interface.
In this embodiment, the control window 401_2 includes a function selection page 401_21 and an output page (not shown in the figure). The control and management module 1121 transmits a reminding signal to the control window 401_2 according to a received successful archival signal. The output page is configured to display a reminding screen corresponding to the reminding signal. For example, the user clicks on the archive icon in the function selection page 401_21, and the input signal detection module 1122 transmits corresponding the at least one editing signal to the control and management module 1121. Next, the control and management module 1121 generates corresponding the at least one editing instruction to the storage medium according to the at least one editing signal. Moreover, the control and management module 1121 transmits the reminding signal to the control window 401_2 according to the successful archival signal received from the storage medium, so that the output page displays the reminding screen corresponding to the successful archival signal.
As mentioned above, the resource display module 501_1 is configured to display a resource that is input (e.g., an input resource or an editing resource). The layer display module 501_2 is configured to display the layers corresponding to each scene and the corresponding resources. The attribute display module 501_3 is configured to display attributes of the current operation resource and the editing tools corresponding to the attributes. For example, when the user operates a stereoscopic model, the attribute display module 501_3 displays the corresponding data (i.e., attributes) of the explosion operation, cross-section operation, scaling operation, and the like of the stereoscopic model. The scene display module 501_4 is configured to display the sequence, the thumbnail images, and the quantity of the scenes.
In this embodiment, the auxiliary tool module 502_1 is configured to manage the editing tools. The scene editing module 502_2 is configured to preview a scene, record an animation for a single scene, or play audio and video for a scene. The resource editing module 502_3 is configured to edit the input resource 301 (shown in
In this embodiment, the recording and processing module 504_1 is configured to process and record animations of a plurality of scenes to form the synthetic image scenario 101. The scenes includes information about the positional status of the layer at different timings or the action to be performed. More specifically, the scenes include movement information and positional information of an object or a model in a certain layer. The movement information is, for example, the moving speed and the curve change of the object or the model. For example, the movement information is about how the object or the model moves from point A to point B in the currently edited scene. The positional information includes, for example, the specific positional information of point A and point B of the object or the model in the currently edited scene. The resource processing module 504_2 is configured to generate the corresponding input resource 301 according to the input data 300. The scene management module 504_3 is configured to manage the scenes and perform operations such as scene sorting, scene duplication, and scene deletion. The scenario management module 504_4 is configured for management of the generated synthetic image scenario 101, serialization of scenarios, and deserialization of scenarios. The log management module 504_5 is configured for developers to inspect positioning errors and to store system logs. The model loading module 505_1 is configured to load the stereoscopic model. The scene construction module 505_2 is configured to establish a basic scene. For example, the scene construction module 505_2 establishes the basic scene according to the scene template stored in the resource management module 503_5. The cross-section processing module 505_3 is configured to generate a cross-sectional view of the model. The stereoscopic processing module 505_4 is configured to generate stroked curves of the stereoscopic model. The editing layer 505 may adopt third party plug-ins to perform loading and processing of the stereoscopic model. The third-party plug-ins are, for example, plug-in software such as Trilib, Giles, Cross Section Shader, and Quick Outline.
In this embodiment, the resource input module 51 includes the resource layer 503, the resource processing module 504_2, and the editing layer 505. The audio-visual editing module 52 includes the operation interface layer 501, the operation function layer 502, the recording and processing module 504_1, the scene management module 504_3, the scenario management module 504_4, and the log management module 504_5. More specifically, the resource display module 501_1, the layer display module 501_2, and the scene display module 501_4 may correspond to the user interface shown in
In summary of the foregoing, in the multi-resource editing system and the multi-resource editing method according to the embodiments of the invention, the input data in different formats may be converted into the corresponding input resources by the resource input module to process multiple input resources at the same time. In addition, in the multi-resource editing system and the multi-resource editing method according to the embodiments of the invention, each layer and each scene are independently edited by the image editing module to generate a plurality of processed planar or stereoscopic scenes and combine the scenes into the synthetic image scenario to be provided to the broadcasting and recording apparatus for subsequent use. Accordingly, the user can edit and produce audio-visual scenarios for broadcasting or remote teaching in advance. Moreover, the image editing module according to the embodiments of the invention independently processes different input resources in different layers to perform operations of inputting and editing multiple resources at the same time. The image editing module according to the embodiments of the invention can independently edit each scene to provide the multi-resource editing system and the multi-resource editing method of good convenience.
The foregoing description of the preferred embodiments of the invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form or to exemplary embodiments disclosed. Accordingly, the foregoing description should be regarded as illustrative rather than restrictive. Obviously, many modifications and variations will be apparent to practitioners skilled in this art. The embodiments are chosen and described in order to best explain the principles of the invention and its best mode practical application, thereby to enable persons skilled in the art to understand the invention for various embodiments and with various modifications as are suited to the particular use or implementation contemplated. It is intended that the scope of the invention be defined by the claims appended hereto and their equivalents in which all terms are meant in their broadest reasonable sense unless otherwise indicated. Therefore, the term “the invention”, “the present invention” or the like does not necessarily limit the claim scope to a specific embodiment, and the reference to particularly preferred exemplary embodiments of the invention does not imply a limitation on the invention, and no such limitation is to be inferred. The invention is limited only by the spirit and scope of the appended claims. Moreover, these claims may refer to use “first”, “second”, etc. following with noun or element. Such terms should be understood as a nomenclature and should not be construed as giving the limitation on the number of the elements modified by such nomenclature unless specific number has been given. The abstract of the disclosure is provided to comply with the rules requiring an abstract, which will allow a searcher to quickly ascertain the subject matter of the technical disclosure of any patent issued from this disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Any advantages and benefits described may not apply to all embodiments of the invention. It should be appreciated that variations may be made in the embodiments described by persons skilled in the art without departing from the scope of the present invention as defined by the following claims. Moreover, no element and component in the present disclosure is intended to be dedicated to the public regardless of whether the element or component is explicitly recited in the following claims.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202210331000.4 | Mar 2022 | CN | national |
