The present invention relates to an intelligent system for matching audio with video, and more particularly to a music editing system for matching audio with video by means of AI matching.
For a singer, a music professional, album production personnel, single track production personnel, a record company or a media company who are concerned with providing music information, when selecting a creative composition for a produced video, it is usually up to a music professional, a video provision authority or a music application authority to select a composition, and matching audio with video is usually completed by video editing and production personnel such as an advertisement company, a movie trailer production team, a movie company, a film production student, photographer-produced photograph audio matching personnel, a theatrical company, a dance theater company, a game company, web page design music personnel, business promotion soundtrack personnel, event background music personnel, event live performance personnel, show music personnel, exhibit music personnel, interactive design music personnel, AR/VR interactive device music personnel and multimedia personnel; alternatively, the described entities who require applications of music would commission other music application units to select a composition, or commission music production/audio matching personnel, a studio, a creator, a singer, a music professional, album production personnel, single track production personnel, a record company or a media company/unit to compose music therefor. However, the described users who require music, for example, a music application authority such as a video production entity or a theatrical creation entity, often face various issues regarding music authorization. For instance, a simple act of uploading a favorite video to YouTube could result in copyright infringement and even lead to a YouTube account being deleted. When the described music information provider intends to look for audio to be matched with a video and copyright authorization, the process is extremely time consuming and will take from 8 hours to 6 months for selecting compositions, listening to the compositions and seeking authorization in order to find decent audio to be matched with the video. For a video creative composition selection unit, it would take a music application creator approximately 5 hours to select a composition each time and approximately 5 days to commission production each time, and the copyright signing process is extremely cumbersome. For a music copyright transaction unit, it would take approximately 5 hours to look for a composition each time and approximately 6 months to sign for copyright; the allocation of royalty is often not properly done in most circumstances. Therefore, for most people who seek applications of music or video creators, an issue which requires an urgent solution is to enable a composition selection time for video creation and a music copyright purchase and authorization time to be significantly reduced for a video professional matching audio with video or a theatrical company creating a play.
An intelligent system for matching audio with video is provided for enabling a unit related to seeking music authorization, such as a video production unit, a theatrical company and the like, to bypass various issues encountered while selecting a composition for video creation.
The primary object of the present invention is to provide an intelligent system for matching audio with video, which use an AI matching module to connect to a video analysis module and a music analysis module, so as to perform adequate matching between video and musical characteristics and recommend several songs for matching; if the recommended songs are not satisfactory, new recommendations of other songs can be made for matching, so as to achieve the object of quickly selecting a composition for video creation by means of intelligent matching.
Referring to
The video analysis module 10 makes an analysis according to color tone, storyboard pace, video dialogue (such as a plot, a word of turn in speech and the like), length and category, director's special requirement and characteristic, actors expression, movement, weather, scene, buildings, spatial and temporal factors, things, creature, character, character personality;
video content analysis of the video analysis module 10 includes: a color analysis, a content analysis and a character expression analysis. Referring to
The music analysis module 20 makes an analysis according to recorded music form, sectional turn, style, genre, melody, tempo, instrument, chord accompaniment, voice type, rhythm, volume and emotional tension; a music analysis and content of the music analysis module 20 includes: a music property analysis, an emotion analysis and music characteristic information, wherein the music property analysis is related to an analysis of musical tone property, instrumental arrangement, music structure, rhythm, chord, chord progression, rhythm notes, pitch, scale progression, style, music form, section, phrase, lyrical phrase, genre and other music file information.
Referring to
Referring to
The present invention of the intelligent system for matching audio with video is characterized in: an AI matching module 30 for connecting to the video analysis module 10 and the music analysis module 20, so as to perform adequate matching between a video and a musical characteristic and recommend five songs for matching in practice; if the recommended songs are not satisfactory, new recommendations of other songs can be made for matching. The music editing module 40 is connected to the AI matching module 30, and the present invention can be used to impeccably match a time axis with an impact point between a music file and a video file by means of clip cutting and editing, music editing, music volume adjustment and sound field simulation. With regard to point-to-point matching of sound effects between the music editing module 40 and the music analysis module 20, in video data referred thereby, there can be more sound effects, so that an insertion point for a sound effect can be obtained by analyzing a waveform. The video data referred to by the AI matching module 30 trained by the present invention includes: YouTube-Movie, YouTube-movie clips and the like.
Referring to
A search for related keywords in a database page includes: a title, a genre, a style, a tempo, an instrument, a related keyword, an artist, an emotion, a cover photo and the like; an unique function of an audio signal is related to formats such as a mp3, a way format or mp3 format and the like; related authorization and an order are related to commercial behaviors such as an estimated order amount based on Loop, midi and music authorization, making an order, updating an order, downloading purchased music and the like.
An algorithm of the AI matching module 30 of the present invention includes:
a filtering and selecting mode and a scoring mode, wherein the filtering and selecting mode is within a range of standard deviation for normal distribution, so as to provide a criterion for whether to select or not, a value within a 68% confidence interval (within the error range of one standard deviation) is allowed, and a category of said filtering and selecting comprises a genre or an emotional parameter and the like. The scoring mode quantifies categories such as rhythm, instrument arrangement, chord, musical emotion (x, y), keyword emotion (x, y), director-input information, main video color tone, video content and the like, so as to calculate a score for each item for performing weighting and averaging.
In conclusion, the intelligent system for matching audio with video of the present invention, the AI matching module is mainly used to connect to the video analysis module and the music analysis module, so as to adequately match a video with a musical characteristic; after diverse logging in by a video company, selecting a video and reviewing by a director, as long as an API end point blockchain smart contract is established on the platform, a music professional, a video company and a media company are enabled to quickly complete matching audio with video.
It is of course to be understood that the embodiments described herein are merely illustrative of the principles of the invention and that a wide variety of modifications thereto may be effected by persons skilled in the art without departing from the spirit and scope of the invention as set forth in the following claims.
Number | Date | Country | Kind |
---|---|---|---|
108124933 | Jul 2019 | TW | national |
This application is a Continuation in part of application of U.S. patent application Ser. No. 16/749,195, filed on Jan. 22, 2020, currently pending.
Number | Date | Country | |
---|---|---|---|
Parent | 16749195 | Jan 2020 | US |
Child | 17951133 | US |