Patent Application
20120275770
Not applicable.
The present disclosure generally relates to processing of media data and more particularly relates to systems and methods for editing media data.
Digital video cameras have become so common that an individual may use more than video one camera to record the same event. A video editing tool may be used to combine the media data from various cameras into a single instance of media data. However, the process of selecting frames and/or segments from particular files for inclusion into the combined file is cumbersome and time consuming with conventional video editing tools.
Embodiments of the present invention provide a friendly user interface and methodology in media editing to edit and generate time-based media data, such as video files. Various embodiments are described for time-based editing of media data. One embodiment is a method which includes displaying a representation of first media data and second media data. Each of the first and second media data are displayed along a corresponding one of a plurality of lines at a corresponding one of a plurality of positions. The corresponding position is chosen in accordance with a capture time associated with the media data, such that when the media data have overlapping capture times the media data with overlapping capture times are displayed on different ones of the lines. The method further includes determining an overlap region for a first and a second of the media data having overlapping capture times. The method further includes combining a plurality of frames within the overlap region of the first media data and the second media data according to an overlap style to produce an output media data.
Another embodiment is a computing device which includes memory and a processor. The memory has executable instructions stored thereon. The processor is configured by at least a portion of the executable instructions retrieved from the memory to display a representation of a first media data and a second media data. Each of the media data is displayed along a corresponding one of a plurality of lines at a corresponding one of a plurality of positions. Each corresponding position is chosen in accordance with a capture time associated with the media data, such that when the media data have overlapping capture times the media data with overlapping capture times are displayed on different ones of the lines. When the first media data and the second media data have overlapping capture times, an overlap region for the first media data and the second media data is determined. A plurality of frames within the overlap region is combined according to an overlap style to produce an output media data. The first media data and the second media data are captured from different sources.
Still another embodiment is a system including at least one computing device and an executable in the at least one computing device. The executable includes logic that displays a representation of a first media data and a second media data. Each of the media data is displayed along a corresponding one of a plurality of lines at a corresponding one of a plurality of positions. Each corresponding position is chosen in accordance with a capture time associated with the media data, such that when the media data have overlapping capture times the media data with overlapping capture times are displayed on different ones of the lines. The executable also includes logic that determines an overlap region for the first media data and the second media data, and logic that combines a plurality of frames within the overlap region according to an overlap style to produce an output media data. The first media data and the second media data are captured from different sources.
Other systems, methods, features, and advantages of the present disclosure will be or become apparent to one with skill in the art upon examination of the following drawings and detailed description. It is intended that all such additional systems, methods, features, and advantages be included within this description, be within the scope of the present disclosure, and be protected by the accompanying claims.
Many aspects of the disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating the principles of the present disclosure.
Having summarized various aspects of the present disclosure, reference will now be made in detail to the description of the disclosure as illustrated in the drawings. While the disclosure will be described in connection with these drawings, there is no intent to limit it to the embodiment or embodiments disclosed herein. On the contrary, the intent is to cover all alternatives, modifications, and equivalents included within the spirit and scope of the disclosure as defined by the appended claims.
As described earlier, using conventional editing tools to produce a single output file from multiple files is cumbersome. Navigating through multiple instances of media data to select frames or segments in a video stream is time consuming. Similarly, navigating through multiple still image files to select a set of photos is time consuming. The present disclosure generally relates to editing media, and more particularly relates to time-based systems and methods of editing media. As used herein, the term “media stream” describes video streams as well as multimedia streams, in which video is intermixed with audio, graphics, and/or data streams. Media data describes, for example, files that include video streams, files that include multimedia streams, files that include a series of still images (i.e., a “slideshow”), and files that include digital photos.
Embodiments of the disclosed systems and methods simultaneously present multiple media data to the user. The media data instances from different devices are displayed on different rows within a timeline, aligned by the actual (clock) time and/or date of video capture. Where each instance of media data may have its own internal elapsed time, the clock time used in the disclosed systems and methods has a base or reference that is common to all the files shown. Thus, if all the instances captured the same event from 12:00 PM to 12:18 PM, the timeline would show 12:00 PM to 12:18 PM, even if the instances captured by individual cameras might show an elapsed time of 18 minutes. By displaying media data simultaneously, each one relative to the same actual capture time, the user can easily appreciate timing relationships between media data and/or segments within media data. This can be especially beneficial when the data represent the same event as captured by multiple video cameras.
Furthermore, the simultaneous display of multiple instances of media data relative to the same timeline allows the user to easily appreciate regions in which media data overlap in time, i.e., time periods which are covered by more than one media data. When the time period for one media data overlaps with the time period for another media data, such that more than one source is available for the same time period, the disclosed systems and methods may apply a particular style to frames within the overlap period. For example, the frame from one media data input captured at a particular time may be tiled with the corresponding frame from another media data input. Embodiments of the disclosed systems and methods generate a media data output containing frames from multiple media data, including the frames created by applying styles to the overlap region. In this manner, embodiments of the disclosed systems and methods allow a user to combine multiple instances of media data in a manner which takes into account the time relationships between instances of media data that capture the same event.
In some embodiments, media editing system 100 is coupled via a wired or wireless communication path to a network 130, such as the Internet or a local area network (LAN). Media editing system 100 retrieves media data 135 and allows the user to edit the media data 135 to produce a media data output which combines frames from different media data inputs. In some embodiments, media data 135 are retrieved from a local or remote storage device. Examples of local storage include a compact disc (CD) reader 140-CD and a flash drive (not shown). Examples of remote storage include a network drive 140-N, video editing system 100, or a photo sharing website 140-W that is hosted on a server 145. In some embodiments, media data 135 are provided to media editing system 100 by an image capture device 150, such as a digital camera, via an interface or connection 155. Media editing system 100 may support a number of conventional computer interfaces, such as (but not limited to) IEEE-1394 High Performance Serial Bus (Firewire), universal serial bus (USB), a serial connection, and a parallel connection. As one of ordinary skill will appreciate, media data 135 may be encoded in any number of formats, including (but not limited to) JPEG (Joint Photographic Experts Group) files, TIFF (Tagged Image File Format) files, PNG (Portable Network Graphics) files, GIF (Graphics Interchange Format) files, and BMP (bitmap) files. Having introduced example functionality that is provided by media editing system 100 introduced above, these components will now be described in more detail.
Through user interface 210, the user selects two or more media data inputs 135 to be combined into a media data output 240, in the manner described below. The inputs 135 selected for inclusion in the output 240 may be less than all the media data 135 that are displayed on timeline 310. For example, if four media data inputs 135 are displayed on timeline 310, the user may choose only the first and the third media data input 135 for inclusion in the output 240, thus excluding the second and the fourth media data input 135 from the output 240. The inputs 135 displayed on timeline 310 and selected by the user may originate from different devices. List control 340 allows the user to select an style for overlap, audio, and/or slideshow modifications. When the user clicks button 350, media editing engine 230 combines the frames in the selected media data 135 in accordance with the style to produce media data output 240. The process of applying a style to multiple media data instances 135 will be described in more detail below.
In the example of
In the example of
As can also be seen in
Time-based media editing user interface 210 may use various rules in deciding the order in which instances of media data 135 are distributed to rows 320. Thus, although the approach used by time-based media editing user interface 210 in the example of
Some embodiments of time-based media editing user interface 210 allow a user to edit timestamps in a media data instance, which may be useful if timestamps in the file are not accurate, or if the capture device does not keep accurate time. In some embodiments, timestamps are edited by dragging a selected media data along the timeline to another time. For example, in
The examples above referred to video instances, which are made up of multiple frames and thus have a inherent time duration. Still images or photos are made of single frames, and thus do not have an inherent time duration. To handle still photos, time-based media editing user interface 210 may associate a still photo with a predefined duration. The still photo duration may be specified by the user for each photo, or a default value may be used. User interface 210 treats a still photo with a duration of X minutes as a video of the same duration. Thus, the still photo spans X minutes on timeline 310. In this manner, time-based media editing user interface 210 enlarges the overlap region of a still photo to fill the duration associated with the still photo. In addition to associating a duration with a still photo, a user may also edit the capture time of a still photo. Since a still photo is displayed on timeline 310 in the same way a video is displayed, the time editing techniques described above in the context of video can also be used for still photos.
The example window 300 shows a horizontal arrangement of media files 135 and timeline 310. In other embodiments, the arrangement may be vertical instead. Furthermore, the user interface window may include other controls, buttons, menus, dialogs, etc. which are not shown in
After displaying selected media data instances 135 in conjunction with timeline 310, processing continues at block 440, where editor 125 applies a style 220, such as an overlap style, to frames in the overlapping time regions 330 (
The application of style 220 in block 440 produces new frames corresponding to the overlapping region of the media data instance 135. These newly created frames are a combination of one frame from each input media data instance 135. For example, when the style is horizontal tiling, media editing engine 230 produces a series of new frames, each of which horizontally tiles a frame from the time overlap region of the first input media data instance 135 with a corresponding frame from the time overlap region in the second media data instance 135.
At block 450, editor 125 includes an audio track to accompany the media frames/slideshow. Inclusion of this audio track may be accomplished by applying an audio style to the media data output. The audio track may be obtained in various ways. The audio track used in block 450 may be the audio track from one of the media data inputs 135, and the user may make the selection. For example, a user can select as output an audio track from either main audio or another video stream (e.g., PIP). The audio track used in block 450 may be a digital audio file identified by a user. The digital audio file may be one provided by a user, or it may be selected from a set of digital audio files included with, downloaded by, or provided by editor 125. Alternatively, the audio style may specify no audio (mute) rather than an audio track. Finally, at block 450, editor 125 generates a media data output 240 (
Method 500 begins at block 510, where time-based media editor 125 identifies which of the two media data inputs 135 has an earlier capture time, and identifies the non-overlapping region(s) of the earlier media data input 135. At box 520, time-based media editor 125 then adds the frames from the non-overlapping region of the earlier media data input 135 to media data output 240. Having handled the first region of the combined media data output 240, time-based media editor 125 begins an iteration loop to handle frames in the overlap region 330 of the two media data instances 135.
In
At block 540, time-based media editor 125 combines the frames in the current pair to produce one output frame. The combination is performed according to the style 220, such as an overlap style, and may involve scaling of the images, obscuring part of one image with another image, applying a transparency effect to one or more images, or any combination thereof. Examples of styles 220 include horizontal and/or vertical tiling and picture-in-picture. Style 220 further encompasses any mechanism for combining images from two frames into a single frame.
In another embodiments, a multiple angle style is applied at block 540 rather than an overlap style 220. The multiple angle style is useful when the same scene is captured by different devices at the same time, each device capturing from a different angle. The result is multiple frames having the same timestamp. The multiple angle style does not combine the frames taken from different angles into a single frame, but instead includes all of the multiple angle frames while adjusting the timestamps appropriately.
Processing continues at block 550, where the combination frame generated at block 540, or the series of frames generated at block 540 for a multiple angle style, is added to media data output 240. The timestamp of the combination frame is the same as the timestamp of the pair of source frames from overlap region 330. That is, the new combination frame appears at the same time position in media data output 240 as the source frames do in the media data inputs 135.
Time-based media editor 125 advances to the next pair at block 560, until all pairs have been processed. When all pairs in the overlap region have been handled by the iteration loop, time-based media editor 125 moves to block 570, where the frames from the non-overlapping region of the later media data input 135 are added to media data output 240. Method 500 is then complete.
Time-based media editor 125 produces a properly sequenced output file 240 which contains a first segment from the earliest media data instance 135, a middle segment which combines multiple media data instances 135 according to style 220, and a final segment from the latest media data instance 135. This process will now be further explained with reference to
As shown in
Various embodiments of media editing system 100 handle overlapping time regions of media data instances 135 as described above. Some embodiments of time-based media editor 125 handle missing time regions, or gaps, in a continuous time sequence by moving media data around on the timeline to remove time gaps.
Other embodiments of media editing system 100 handle missing time regions, or gaps, in a continuous time sequence by inserting frames into the time gap. The frames inserted into the time gap may represent an advertisement, a transaction, a news story or event, where these inserted frames are selected from frames included with, downloaded to, or provided by editor 125. The frames inserted into the time gap may also be frames from a media data, a photo file, or a slideshow file selected by the user. A predefined style may be applied to the inserted frames.
The components discussed above (e.g., time-based media editing user interface 210 and media editing engine 230) can be implemented in software (i.e., instructions executing on a processor), and in such an implementation, memory 820 stores these components. These components can also be implemented in specialized hardware logic. Hardware implementations include (but are not limited to) a programmable logic device (PLD), programmable gate array (PGA), field programmable gate array (FPGA), an application-specific integrated circuit (ASIC), a system on chip (SoC), and a system in package (SiP). Persons of ordinary skill should also appreciate that these components may be implemented using any combination of hardware and software.
In some embodiments of media editing system 100, the software components are stored on a computer-readable medium, which in the context of this disclosure refers to any structure which can contain, store, or embody instructions executable by a processor. The computer readable medium can be, for example but not limited to, based on electronic, magnetic, optical, electromagnetic, infrared, or semiconductor technology. Specific examples of a computer-readable medium using electronic technology would include (but are not limited to) the following: a random access memory (RAM); a read-only memory (ROM); and an erasable programmable read-only memory (EPROM or Flash memory). A specific example using magnetic technology includes (but is not limited to) a disk drive; and a portable computer diskette. Specific examples using optical technology include (but are not limited to) a compact disk read-only memory (CD-ROM) or a digital video disk read-only memory (DVD-ROM).
Any process descriptions or blocks in flowcharts would be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific functions or steps in the process. As would be understood by those of ordinary skill in the art of the software development, alternate implementations are also included within the scope of the disclosure. In these alternate implementations, functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved.
The foregoing description has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Obvious modifications or variations are possible in light of the above teachings. The implementations discussed, however, were chosen and described to illustrate the principles of the disclosure and its practical application to thereby enable one of ordinary skill in the art to utilize the disclosure in various implementations and with various modifications as are suited to the particular use contemplated. All such modifications and variation are within the scope of the disclosure as determined by the appended claims when interpreted in accordance with the breadth to which they are fairly and legally entitled.
