Patent Application
20220374139
With the explosion of the user-generated video content on various platforms on the Internet, the functionality and features of many existing video editing applications as been found to be either too complicated for non-professional video editors, or too cumbersome for the quick editing of videos in the field, and ill-suited for small screen devices, such as smartphones or the like. The user interface paradigms of current video editing solutions are better suited to large display screens and professional video editors. Thus, several technical challenges are presented by the current user interfaces and video editing functionality of existing video editing applications and programs.
Additionally, a number of technical challenges are presented by the considerable number of content items that users are seeking to manipulate within a single video editing project and combine into a single video output. The presentation of such multiple content items, the individual editing, and modification of these content items, and the combination thereof into a single project output (e.g., a video) presents several technical challenges to existing video editing applications that have yet to be addressed.
There is also increasing demand on video editors to create current, trending, and visually compelling content to differentiate their videos from the substantial number of videos that are being user-generated and published on popular platforms.
To easily identify the discussion of any particular element or act, the most significant digit or digits in a reference number refer to the figure number in which that element is first introduced.
An example video editing application seeks to help passionate creators make engaging and successful videos for viewers (e.g., “fans”) on content publishing platforms. The video editing application combines powerful mobile video editing functionality with insights into trends on one or more video publishing platforms. For example, creators can learn about the #topics, songs, and augmentations (e.g., filters and lenses) that are trending on one or more video publishing platforms, view examples of user-generated media for inspiration, and then conveniently apply such inspirations to their own video projects. The video editing application, in some examples, enables mobile video editing using a collection of tools that are simple to use; powerful, and precise; and may be tailored to a specific video publishing platform
By leveraging data generated by a media content and a messaging platform regarding trending topics, music, and augmentations (e.g., filters, lenses, stickers, captions, etc.), example video editing applications help creators tap into current trends, and address the problem of “blank canvas paralysis.”
These sections or displays may be arranged in different configurations or even split across multiple video editing interfaces. For example, the tool function bar 112 may, in some examples, be positioned above the project element carousel 110, and a tool navigation bar 118 may be positioned below the project element carousel 110. Further examples of user interface layouts are discussed below with reference to subsequent figures.
Each of these display sections may be controlled and populated by a component of a project editor system 204, examples of which are shown in
The video editing application 206 includes a sign in/sign out system 202, a project editor system 204, and a trend analysis system 208. When launching the video editing application 206 for the first time, the user may be prompted by the sign in/sign out system 202 to log in with the interaction client 2402, which presents a privacy policy and terms of service for the interaction server system 2404.
Upon activation of the video editing application 206, a home screen (not shown) is presented to a user. The home screen presents a full-screen carousel of projects that the user has worked on in the past within a project listing interface component. The user has the option to swipe horizontally to look through past projects, delete or duplicate the active projects, or tap ‘View all’ to see all projects in a more compact grid view. Tapping a Create button on the home screen creates a new, empty project in the media content items 304.
The video editing application 206 provides a variety of tools for creating and editing video projects. The video editing application 206 automatically saves projects as they are worked on by a user.
After creating a new project, a user is automatically prompted to add project elements (e.g., videos or images from a camera roll, for example, of a mobile device hosting the video editing application or from cloud storage) to a main project timeline. Added video and images are previewed in a canvas 104 as described with reference to
Various tools within the tool navigation bar 118 (e.g., a main tool bar 118 may be used to add layers, captions, lenses, and other project elements to a video editing project.
A user can tap on the canvas 104 to start playback and preview an active project. When playback starts, on-canvas interface elements (e.g., the project element carousel 110, the tool navigation bar 118, the tool function bar 112, and various buttons) are temporarily hidden so that the user can preview video in a full-screen mode of a client system 106, unobstructed by user interface elements and as it will be outputted from the video edit project. A subsequent and further tap on the canvas 104 pauses the preview playback and again reveals the temporarily hidden interface elements.
When the preview video is paused, an on-screen navigation tool, in the form of a scrubber, appears in the canvas 104, above the project element carousel 110, and the tool function bar 112. A user can horizontally drag the scrubber on the canvas 104 above the toolbar to navigate forwards and backwards in a video editing project. As the user drags the scrubber, a playhead and time code are displayed to show what part of the video editing project is previewed in the canvas 104. While playback is active, a user can also press-and-hold to pause playback temporarily and reveal the scrubber.
Each tool in the tool functions 114 is used to add different project elements to a video editing project. A user taps on a respective tool icon in the tool functions 114 to activate the corresponding tool.
A number of these tools are discussed in further detail below.
Video Project Data Structures
A project element set may correspond to a project element type, but may also include project elements of multiple types. For example, a project element category “clips” may correspond to a clip layer, which includes both video and still image media content items.
Each project element set of a video project 302 further may have an associated timeline with timing information (e.g., defining start and end times for presentation of each of the project elements) within the context of a full project duration determined by a main project timeline. To this end, each project element set may be regarded as including a timeline, which determines an ordering of project elements within the project element set and also the start and end of presentation or application of a particular project element within the context of that project element set and the overall project duration.
A video project 302 is shown in
Some sets of media content items 304 may further be organized into layers, and others according to content type (e.g., an audio layer of audio content items and a caption layer of caption content types).
Sets of media augmentation items 306 include filters, lenses, effects, captions, speed adjustments, and other augmentations that may be applied to any one of the media content items 304 to augment and modify these content items.
Media layout items 308 include several media content item layouts that may be applied to media content items 304 within the canvas 104 at playback.
Only a few examples of media items are shown in
Further, the video editing application 206 may enable a dedicated and specialized editing mode for each of a number of project element categories (e.g., for each of a number of project element sets). As an example,
Each of these specialized editing modes includes and presents a selection of editing mode tools (e.g., 312, 318, 324) and tool functions (e.g., 314, 320, 326) that are appropriate to the relevant project element sets. Certain editing mode tools and tool functions may be shared and common across different editing modes as appropriate.
Each timeline is associated with a project element set (e.g., of a particular project element category, or a layer). The timeline includes timing and sequence information for each of relevant project elements of the project element set. This timing information may include the start and end times for presentation or activation of a particular project element within the context of the relevant timeline, and/or sequence and duration information for each of the project elements. The timing information maintained within a particular timeline is further relative to a designated “main” project timeline, and/or an overall project duration.
Considering the main timeline 402 as an example, a timeline is associated with a project element set including four project elements in the form of clip elements 404, 406, 408, and 410. These project elements span the duration of a video project 302 (e.g., the playback duration of a video outputted by the video project 302).
The sequencing of the clip elements 404, 406, 408, and 410 is also maintained within the context of the main timeline 402, as well as the start and end times (or duration) of each of these clip elements. Similarly, the layer timeline 412 includes clip element 414 and clip element 416. It will be noted that these project elements do not span the entire duration of the project timeline, as defined by the main timeline 402.
A transition timeline 418 includes a single transition element 420, which may be applied to modify the transition between the clip element 404 and the clip element 406 within the main timeline 402.
The layout timeline 422 includes a single layout element 424, according to which clip elements in multiple project element sets (e.g., project elements of the main timeline 402, and auxiliary or supplemental project elements of the layer timeline 412) may be combined in an overlaid or split-screen fashion).
The effect timeline 426 includes three effect elements 428, effect element 430, effect element 432, which may apply one or more effects (or other augmentations) to project elements within the other timelines.
Main Video Editing User Interface 502
The video editing user interface 502 includes the following elements:
It should be noted that selecting a particular tool icon within the main tool bar 118 does not change the current playback state (e.g., play/pause). Further, during the playback of content within the canvas 104, the main tool bar 118 may be hidden.
A discussion of various example video editing user interface video editing user interfaces 502 that may be presented responsive to selection of different tools within the main tool bar 118 will not be provided.
Clip Tool 216
Referring back to
A tool function bar 112 presents a number of tool functions including:
In order to activate the clip tool 216, the user selects (e.g., taps) a clip tool button 604 within the main tool bar 118. The clip tool 216, in some examples, allows a user to manage and manipulate project elements, in the form of timeline media (e.g., a sequence of videos and images that comprise lowest Z-index layers in a video editing project).
Within the clip tool user interface 602, below the canvas 104, are presented a project element carousel 110, and the tool function bar 112. The project element carousel 110 presents in indicia (e.g., clip indicia) representing a sequence of project elements (e.g., clips presented in the current, active project). Within the project element carousel 110, a sequence of clip indicia can be scrolled horizontally. Specifically, clip indicia are presented within the project element carousel 110 in an order in which the corresponding clips appear within the main timeline of the video editing project, when the relevant project is previewed (e.g., within the canvas 104), or rendered as a video project output.
Each in indicium associated with the clip in the project element carousel 110 may be a thumbnail based on a determined frame within the clip (e.g., the 45th frame within a video clip). For the various clips represented within the project element carousel 110, one is active at any particular time, as determined by a current position of the playhead 120. An active clip indicium may be visually distinguished from non-active clip indicium (e.g., with a red border or with some other visualization) within the project element carousel 110.
A position of the playhead 120 may be visualized within the scrubber 504 by shading an area of the scrubber 504 that is proportionate to the right of an offset within the clip, with a black 50% capacity overlay. Further, the clips to the right of an active clip may be fully shaded with a black 50% capacity overlay.
A first type of user interaction (e.g., tapping) on the clip indicium within the set of clip indicia within the project element carousel 110 relocates (e.g., advances or retreats) and sets the current position of the playhead to a selected frame (e.g., the first frame) of the selected clip in a particular video editing project.
Performing a second type of user interaction (e.g., a long press) on a clip indicium transfers the clip tool user interface 602 into a reorder mode, which allows clips to be reordered within the project timeline. For example, a user can select a clip indicium and drag and drop that clip indicium to a different position within the project element carousel 110 in order to reorder the sequence of project elements in the project timeline.
The tool function bar 112, which is presented below the project element carousel 110, contains a number of tool function buttons associated with specific functions mentioned above, namely an add button 606, a trim button 608, a split button 610, a crop button 612, and a delete button 614.
User selection of the add button 606 invokes a media selection function (e.g., presented by the operating system of a user device on which the video editing application is executing), which allows for the selection of one or more content items as project elements. Any selected media items are then added as distinct clips, with associated clip indicium, to the project element carousel 110, following any existing clip indicium presented within the project element carousel 110.
The trim button 608 invokes a modal trimming user interface sequence for a currently selected an active clip. The trim modal user interface sequence contains a number of elements, including a full-screen preview canvas, a back navigation button, a commit button, and an abstract timeline that is caused to be displayed in conjunction with a double-ended range slider.
The preview canvas 104 provides the same playback functionality and scrubbing gestures as described above. Playback is also limited to a current clip, and playback is paused upon entering the trim modal user interface sequence.
The back navigation button discards any trimming changes that may have been made using the trim modal user interface sequence, while the commit button applies trim edits to a currently active clip in a non-destructive manner and creates a new history state for the relevant video editing project.
The user is able to drag each of the handles 806 within the abstract timeline 802 to adjust the selected portion of the active clip. Dragging a handle 806 changes the current position of the playhead 120 to equal the position of the dragged handle 806. Dragging a handle 806 also pauses playback within the canvas.
Ending a drag, or tapping on a handle 806, triggers the generation and display of ephemeral frame step previous/next buttons 808. The frame step previous/next buttons 808 advantageously provide a greater and higher resolution degree of control or to a user with respect to the movement of a particular handle 806.
Further, within the clip tool user interface 602, if there is no active clip, the trim button is disabled. Further, if the active clip is an image, the trim modal user interfaces change such that the abstract timeline 802 displays the total duration of the image within the main timeline. A set default duration for an image may be used as an initial duration (e.g., 5 seconds), but the user may manipulate the handles 806 to update the duration, down to a certain minimum duration (e.g., one second) and up to a certain maximum duration (e.g., 60 seconds).
User selection of the split button 610 invokes a split modal user interface sequence for a currently selected, active clip. The split modal user interface sequence includes a full-screen preview canvas 104, a back navigation button, a commit button, and an abstract timeline 802 displayed beneath the single-ended range slider 810. The abstract timeline 802 visualizes the full extent of the clip, and a single handle 812 is centered within the abstract timeline 802 by default. Dragging the handle 812 changes the current position of the playhead 120 to correspond to the position of the handle 812. Starting a drag operation with respect to the handle 812 causes playback. Ending a drag operation of the handle 812, or tapping the handle 812, triggers generation display of a frame step previous/next buttons 808
As with the trim modal user interface sequence, if there is no active clip, the split button 610 is disabled. Also, if the active clip is an image, the split button 610 is likewise disabled.
Returning to
The preview canvas 104 within the crop modal user interface sequence supports playback and scrubbing gestures as described above for other canvas instances. The preview canvas 104 additionally supports pinch-to-scale, twist-to-rotate, and drag-to-translate gestures that update the transform of the current clip with respect to a viewport.
The mirror button 702 and the rotate button 704 apply corresponding transforms, while the reset button 706 resets the transforms to origin. Again, if there is no active clip, the crop button 612 is disabled.
The delete button 614 removes a currently active clip from the video editing project after confirmation. If there is no active clip, the delete button 614 is similarly disabled and not displayed within the clip tool user interface 602.
Layer Tool 218
Layers may be sets of project elements (e.g., images or videos) that appear “on top” of a main timeline 402 (e.g., in the z-index) in a layer timeline 412. Unlike project elements of the main timeline 402, many layers may be visible at once within the canvas 104 of the layer tool user interface 902. The layers already in a video project 302 are visualized by a sequence of icons or indicia (e.g., thumbnails) within the project element carousel 110, and also as layer images 918 within the canvas 104. User-selection of a layer image 918 within the canvas 104 automatically selects a corresponding layer indicium 920 within the project element carousel 110, and vice versa. Within the canvas 104, an active layer image 918 is visually distinguished (e.g., by highlighting or a border color change) from non-active layer images. The active layer image 918 is further moved to the front of the z-index relative to other layer images.
Within the project element carousel 110, tapping a layer indicium 920 (e.g., a thumbnail) associated with a particular layer selects and activates the layer, and scrubs the playhead 506 to where that layer first appears chronologically in the video project 302.
As noted, the project element carousel 110 represents the layers present in the current project as a sorted list of layer indicia 920 that scrolls horizontally. Layer indicia 920 are sorted by the start time of the represented layer project element. For layer project elements that have the same start time, these are sorted internally by end time ascending, and then by the order in which the layer project elements were added to a video editing project. If there are layer indicia 920 in the carousel, then at most one is active. Whenever the layer tool 218 is opened, a first layer indicia 920 in the project element carousel 110 at the current playhead position is active. Otherwise, no edit is active.
The layer tool 218 includes a number of tool functions for which buttons are presented in a tool function bar 112 of the layer tool user interface 902, these buttons including:
Caption Tool 220
Captions, according to some examples, are created using the caption tool 220 and are styled text elements that appear on top of project elements (e.g., images and video elements) of a main timeline. As with the layer tool user interface 902, there can be many captions 1006 visible concurrently in the canvas 104.
Within the project element carousel 110, a sequence of graphic elements in the example form of caption indicia 1008 (e.g., thumbnails) representing project elements in the form of captions already in a video editing project are presented.
Tapping a caption indicium 1008 selects the associated caption and scrubs the playhead 120 in the preview video editing canvas 104 to where that caption first appears in a video editing project. Tapping a caption on the canvas 104 also selects the caption. The list of caption indicia 1008 scrolls horizontally and is sorted by the start time of the relevant caption. For captions that have the same start time, these are sorted internally by end time ascending, and then by the order in which the captions were added to the video editing project. If there are caption indicia 1008 in the project element carousel 110, then at most one is active. When the caption tool 220 is open, the first caption indicium 1008 in the project element carousel 110 at the current playhead position is active, if such an edit exists. Otherwise, no edit is active. Each caption indicium 1008 shows the initial segment of text from the corresponding caption, truncating it with an ellipsis if necessary.
A tool function bar 112 presented within the caption tool user interface 1002 includes the following function buttons:
Audio Tool 222
The audio tool user interface 1102 includes a canvas 104, a tool navigation bar 118, a project element carousel 110, and a tool function bar 112.
The tool navigation bar 118 includes the following controls: a back control 1114 and commit control 1116. The back control 1114 exits the audio tool user interface 1102 and returns to the main video editing user interface 502. An undo button (not shown) reverses the most recent change made to the project in the current editing session from within the audio tool 222. If no changes have been made to the project with the audio tool 222 in the editing session then the undo button is disabled.
The project element carousel 110 below the canvas 104 and tool navigation bar 118 shows a horizontally scrollable list of project element indicia (e.g., thumbnails), each of which represents an audio project element (e.g., an audio track or clip). The audio tool 222 allows a user to add audio tracks to a video editing project that playback along with the media in a main timeline. Audio tracks already in a video editing project, including audio tracks from timeline videos and video layers, are visualized by the sequence of representative indicium or icons (e.g., thumbnail images). Tapping a project element indicia in the project element carousel 110 selects the audio track and scrubs the playhead 120 to where that audio track first appears in a video editing project.
Within the project element carousel 110, the project element indicia are sorted by the start time of the corresponding audio track. For audio tracks that have the same start time, corresponding project element indicia are sorted by end-time ascending, and then by the order in which the audio tracks were added. An audio track that may be associated with each video clip in the timeline, as well as each video layer, is considered an audio edit, and is visualized in the Edit Carousel along with additional audio edits that have been added.
When the audio tool user interface 1102 is open, the audio track associated with first project element indicium and at the current playhead position is active. For licensed music tracks, the associated album artwork is used as the thumbnail in a first aspect. For other sources, a second aspect may be used.
The tool function bar 112 includes buttons associated with a number of tool functions of the audio tool 222, for example:
A user can add sounds from external applications to video projects 302 using the audio tool 222. While editing the video editing application, a user taps an audio icon associated with the audio tool 222 in the tool navigation bar 118, then taps Add, and then tap Sounds.
From a sounds dialog, a user can browse songs, or search for songs by name, that are available in one or more external applications. The My Sounds tab shows sounds created previously in, for example, the video editing application 206.
Effect Tool 224
An effect tool 224 allows a user to apply visual effects to project elements in the main timeline 402 or layer timelines 412 using media augmentation project elements (e.g., lenses or effects).
Effects already applied to a video editing project are visualized by a sequence of project element indicia (e.g., thumbnails) within the project element carousel 110. Tapping a thumbnail selects an effect and scrubs the playhead 120 to where that effect first appears in a video project 302. The list or sequence of thumbnails scrolls horizontally within the project element carousel 110. Effects are sorted by the start time of the effect. For effects that have the same start time, these are sorted internally by end-time ascending, and then by the order in which the edits were added. If there are effects in the project element carousel 110, then at most one is active. When the effect tool 224 is open, the first transition in the project element carousel 110 at the current playhead 120 position is active, if such an effect exists. Otherwise, no effect is active.
The tool function bar 112 of the effect tool user interface 1404 includes several function buttons, including for example:
Transition Tool 226
Transitions, according to some examples, are created using a transition tool 226, are visual effects that blend two adjacent timeline project elements together.
Transitions already applied to a video editing project are visualized by a sequence of thumbnails. Tapping a thumbnail selects the transition and scrubs the playhead to where that transition first appears in a video editing project. Transitions are applied in post-capture mode to timeline media. Transitions are applied to all visual project elements, including timeline media, layers, captions, and effects. Transitions mix together the audio tracks in the adjacent pair of clips while active. One transition can be applied to an adjacent pair of clips. One transition can be rendered at once. If two transitions overlap, the latter is rendered.
The tool function bar 112 provides buttons for a number of functions of the transition tool 226, including, for example:
Speed Tool 228
The speed tool 228 allows a user to speed up or slow the presentation, or change the direction of playback, of specific parts (e.g., project elements) of a video editing project timeline in the canvas 104. Speed changes already present in a video editing project are visualized by a sequence of project element indicia (e.g., thumbnails) in the project element carousel 110. Tapping a thumbnail selects the associated speed change and scrubs the playhead 120 to where the change first appears in a video editing project.
The tool function bar 112 includes buttons to active or apply several functions of the speed tool 228, these button including for example:
Timing Control 230
The timing control 230 is available in the layer tool 218, caption tool 220, audio tool 222, effect tool 224, and speed tool 228, and allows a user to precisely specify when selected project elements are active or presented (either in a video preview in the canvas 104 or in a video generated and stored by the video editing application 206) within the overall timeline of a video editing project.
The timing tool user interface 1702 includes the following interface elements:
The preview canvas 104 is paused by default and shows the frame of the current timestamp from the main video editing user interface 502. The same play/pause and scrubbing affordances present in other canvas implementations are present here, too.
The back navigation button 1708 dismisses the timing control 230 and returns to the preview screen without committing any of the changes temporarily made to the timing of the active edit.
The commit button 1706 applies the current timing to the active edit, creating a new history state. The commit button 1706 is disabled until a model-state change has been made—e.g., until either of the handles 1704 of the abstract timeline 1710 has been moved.
The auxiliary object type filter list 1714 determines which project elements (e.g., clips or edits) are displayed in the auxiliary object list 1712 below the timeline. The type elements correspond to the tools of the video editing application 206: Clips, Captions, Layers, Audio, Speed, Effects, Transitions etc. The objects of the auxiliary object list 1712 scroll horizontally, and one is active at any time. Clips is selected by default.
The abstract timeline 802 is overlaid with an abstract timing control that consists of a pair of handles 1704 representing the start and stop time of an active project element, relative to the total project duration Moving the left handle to the right will cause the project element to start later in the project timeline, and moving the right handle to the left will cause the project element to stop earlier in the project timeline. Moving or tapping a handle also invokes a pair of ephemeral frame step buttons that allow the handle 1704 to be moved one frame at a time in either direction for precise placement. Tapping the frame-step buttons moves the handle's position in the timeline, one frame left or right. The frame-step buttons auto-dismiss 3s after the last interaction—either the last tap of a frame-step button or after invocation if the frame-step buttons have not been tapped. Tapping anywhere in the timing tool user interface 1702 other than the frame-step buttons dismisses the frame-step buttons.
The space between the handle 1704 is visually distinguished (e.g., colored red), and the space around the handles 1704 is transparent. The stop handle (right) is to the right of the start handle (left). The media duration limits the distance between the two handles 1704 (e.g., the timing of a timing project element (or edit) cannot exceed the duration of the underlying project element (e.g., media such as an image, video clip or audio track).
Project element indicia in the auxiliary object list 1712 below the abstract timeline 1710 correspond to project elements or objects (e.g., clips or edits) already in a video editing project, and are used for alignment of the current edit. As noted above, the auxiliary object type filter list 1714 section determines which clips or edits are visualized below the abstract timeline 1710: if Clips is selected, then the project's clips are shown; if Captions is selected, then the project's captions are shown, etc.
Referring to
By default, and after switching the filter project element, no project element is selected. When an auxiliary project element is selected, the timing of that auxiliary project element is visualized in the abstract timeline 1710 with a light red region. When moving the edit handles 1704, those handles 1704 snap to the left/right edges of the auxiliary object's start/stop time in the timeline. When an auxiliary project element is selected, in some examples, two ephemeral buttons appear above the auxiliary project element's thumbnail, namely an align to object control 1804 and a jump to object timing control 1806. The align to object control 1804 changes the active project elements' start/stop time to match the timing of the auxiliary project element. The jump to object timing control 1806 opens the timing control 230 for the selected auxiliary project element, making that the active project element. The align to object control 1804 and the jump to object timing control 1806 auto-dismiss 3s after the last interaction.
Methods/Algorithms
The method 1900 commences at block 1902 with the presentation on a display of the client system 106 of a video preview display of the video editing application 206. Specifically, the preview and playback system 210 of the project editor system 204 may generate and cause the presentation of the video preview display 102 and the canvas 104. Additionally, within the canvas 104, a preview video output (e.g., of an active project element) of the video project 302 is displayed, with a playhead location (e.g., determined by the playhead 120 of the playhead location tool 116) a specific temporal location of a project timeline (e.g., the main timeline 402).
The playhead location tool 116 may additionally, within the context of the canvas 104, present information (e.g., as time information), that indicates the playhead location within the context of the project timeline.
At block 1904, in addition to presenting the video preview display 102 within the larger video editing interface, the project element display 108 is also generated and displayed. The project element display 108 includes a project element carousel 110, which, as described above, is composed of a sequence of project elements that may be arranged according to various criteria and rules associated with a particular tool of the video editing application 206.
At block 1906, the video editing application 206 further generates and causes display of a tool display (e.g., the tool navigation bar 118 and/or the tool function bar 112) that presents tool indicia. The tool indica are user-selectable to enable and activate various tools of the video editing application 206, as well as various functions of these specific tools.
At block 1908, the video editing application 206 activates the first video editing mode of the video editing application 206, thereby activating one of a number of video editing tools 214 provided by the video editing application 206. The activation of the first editing mode may, in some examples, be responsive to user selection of one or more of the tools 214 and controls of the video editing application 206. For example, a user, using the tool display (e.g., the tool navigation bar 118) may select a specific tool (e.g., the clip tool 216), which then activates a corresponding editing mode as well as the clip tool 216.
At block 1910, and responsive to the activation of the first editing mode (e.g., a clip editing mode responsive to user selection of the clip tool 216), the preview and playback system 210 of the video editing application 206 presents the first set of project element indicia (e.g., a set of video clips) within the project element display 108, and more specifically within the project element carousel 110. Each project element indicium of this first set of project element indicia represents a corresponding associated project element (e.g., a video clip) of the project elements (e.g., media content items 304) of the video project 302. The project element indicium presented at block 1910 may be thumbnail images displayed sequentially within the project element carousel 110.
Moving on to block 1912, user selection of a project element indicium of the set of project element indicium displayed within the project element carousel 110 is detected. The user selection may include a click selection via the video editing interface.
At block 1914, responsive to the detection of the user selection of the selected project element indicium within the project element carousel 110, the playhead 120 of the project timeline (e.g., the main timeline 402) is moved by the timeline and playhead system 212. The updated playback location, within the context of the project timeline, corresponding to the selected project element indicium.
At block 1914, in addition to the advancement of the playhead 120 head to the updated playback location, the project element (e.g., a video clip) corresponding to the selected project element indicium is displayed within the canvas 104 of the video preview display 102.
The updated playback location, in some examples, may be a chronological position (e.g., at a specific time) on the project timeline where a project element (e.g., video clip) associated with the selected project element indicium (e.g., a video clip thumbnail) first appears on the project timeline.
This functionality, with automatic advancement of the public location to a chronological position within the context of the broader project timeline, provides a number of advantages and benefits to the user of the video editing application 206. Particularly, when considering that a display screen of the client system 106 may be small and present limited real estate, providing full contextual information for any project elements of a video project 302 is challenging. Similarly, enabling a user to conveniently navigate the project timeline, which is composed of multiple project elements, presents multiple challenges. The automatic advancement of the playhead 120 to a location of a selected project element, within the context of the particular video editing tool or editing mode, provides a convenient mechanism for such navigation of the project timeline, and thus seeks to address a number of technical challenges that arise with respect to presenting navigation paradigms, without providing full context and review of the project timeline.
The method 2000 commences at block 2002, with the detection, by the video editing application 206, of the first editing mode of several editing modes supported by the video editing application 206. Each of the several editing modes may be associated with a respective project element set (e.g., the media content items 304, the media augmentation items 306, or the media layout items 308) of the video project 302. In some examples, each of the editing modes may be associated or include a set of tools (e.g., the tools 214214) specific to the respective editing mode. Accordingly, responsive to detection of the selection of the particular editing mode, a toolset associated with that editing mode may also be activated and presented.
At block 2004, responsive to activation of the first editing mode, the video editing application 206 identifies and causes presentation, within a project element display (e.g., the canvas 104) of the video editing application 206, of the project element set associated with the first editing mode. For example, where the selected editing mode is the clip editing mode 310 shown in
It should also be noted that, at block 2004, the first project element set, as displayed within the project element carousel 110, may be displayed to the exclusion of project elements of other project element sets. For example, where the clip editing mode 310 is activated, project element indicium associated with the media content items 304 may be displayed to the exclusion of project element indicium associated with the media augmentation items 306 or the media layout items 308.
At block 2006, responsive to detection of selection of a first project element indicium of the first project element set, the video editing application 206 causes presentation of an associated first project element in the canvas 104, and advances a playhead associated with the canvas 104 (e.g., the play head 120), to a location on a project timeline (e.g., the main timeline 402) associated with the first project element.
Consider the example within which project element indicia associated with the media content items 304 are displayed within the project element carousel 110, after the video editing application 206 activates the clip editing mode 310. Responsive to detection of selection of a project element indicium associated with one of the media content items 304, presentation of the relevant media content item 304 within the canvas 104 results. Further, the playhead 120 is advanced to a specific location on the timeline. For example, where the selected media content item is a clip element 408, as displayed within the context of the main timeline 402, the playhead 120 is advanced to the location of a specific frame (e.g., the first frame) of the clip element 408 within the context of the main timeline 402. Further, the clip element 408 is, in this example, displayed within the canvas 104.
Similarly, selection of a second project element (e.g., the clip element 410) using the project element carousel 110, causes the presentation of the clip element 410 within the canvas 104, and advances the playhead 120 to a location, within the main timeline 402, of the first frame of the clip element 410.
When in a particular editing mode (e.g., the clip editing mode 310), a user may, by performing a selection of a tool 214 using the tool navigation bar 118, activated a second editing mode (e.g., the augmentation editing mode 316). In this case, responsive to activation of the augmentation editing mode 316, the video editing application 206 causes presentation within the project element carousel 110, of a further and second project element set (e.g., the project element indicium associated with the media augmentation items 306). Similarly, responsive to the detection of one of the project element indicia associated with the media augmentation items 306, the video editing project 206 causes the presentation of the media augmentation item 306 (e.g., an effect element 428) within the context of an effect tool user interface 1404.
The method 2100 commences at block 2102, with the storage of a project timeline (e.g., the main timeline 402, the layer timeline 412, the transition timeline 418, the effect timeline 426, the layout timeline 422, etc.). The stored project timeline includes a number of media content items (e.g., media content items 304, clip elements, transition elements, effect elements, layout elements, etc.). The media content items within the project timeline are arranged in a chronological sequence, in some examples, with the project timeline having a start time and an end time. The media content items may also, in some examples, be arranged in a non-chronological sequence and according to their different criteria.
At block 2104, the video editing application 206 generates and causes the presentation of a tool interface (e.g., video editing user interface 502, clip tool user interface 602, layer tool user interface 902, caption tool user interface 1002, audio tool user interface 1102, create sound user interface 1202, effect tool user interface 1404, transition tool interface 1502, speed tool user interface 1602, timing tool user interface 1702, etc.). The displayed tool interface is customized and specific to a particular tool 214 of the video editing application 206. While the presented to tool interface may be specific to a particular tool 214, elements of the interface may be common to interfaces for other tools, as will be appreciated from the preceding descriptions.
The presented tool interface, in some examples, includes a number of components or display portions of sections. For example, the tool interface may include a video editing canvas (e.g., the canvas 104) for playback of the project timeline or of project elements included within the project timeline. The tool interface also includes indicium or controls associated with a specific set of tools associated with a specific tool, these tools enabling respective video editing functions of the tool. For example, where the project timeline is the main timeline 402, the tool interface presented at block 2104 may be the clip tool user interface 602, within which the clip tool button 604 is active and selected to enable the clip tool 216. In this example, the tool function bar 112 displays a number of indicium (e.g., the add button 606, trim button 608, split button 610, crop button 612, delete button 614) associated with tools of the clip tool 216, each of these indicium being user-selectable to activate the relevant function of the clip tool 216.
In addition to displaying indicia for the tool functions to enable respective video editing functions, at block 2104, the video editing application 206 causes the presentation of a number of project element indicia that are each associated with a respective media content item. For example, user selection and activation of the highlighted project element indicium within the clip tool user interface 602, and specifically within the project element carousel 110, may be detected.
At block 2108, responsive to detecting of the activation of the first project element indicium associated with the first media content item, playback of the project timeline (e.g., the main timeline 402) may be advanced (or moved) to a chronological location corresponding to the first media content item within the main timeline 402.
As is apparent from
In this example, the effect tool user interface 1404 also presents a number of indicia (e.g., buttons) for selection and activation tool functions specific to the effect tool 224 in the tool function bar 112, including the add button 1406, the timing button 1408 and the delete button 1410.
At block 2202, the video editing application 206 stores multiple project elements, in the example form of media content items 304, within a project data structure (e.g., the video project 302) as maintained by the video editing application 206.
At block 2204, the video editing application 206 stores a number of layout configurations, in the example form of the media layout items 308.
At block 2206, the video editing application 206 generates and causes the presentation of the media content items 304 within the media item display (e.g., within the project element carousel 110 of the clip tool user interface 602, as shown in
At block 2208, the video editing application 206 detects a user selection of one or more of the media content items 304. This selection may be a concurrent selection or a sequential series of selections over a period of time, with each selection of each media content item being a discrete selection.
At block 2210, the video editing application 206 detects the selection of a selected layout configuration from a number of layout configurations presented by the video editing application 206. This selection of the layout configuration may, in some examples, may include user selection of the layers tool button 904 using the tool navigation bar 118 as shown in
Responsive to activation of the layout tool 238 using the layers tool button 904, a set of project element indicia associated with respective layout project elements are displayed within the project element carousel 110 of the layer tool user interface 902. The user is then able to select a particular layout project element from within the project element carousel 110 which is then, at block 2212, automatically applied to the media content items, for which indicium (e.g., layer image 918) are displayed within the canvas 104. Accordingly, at block 2214, the video editing application 206 presents the media content items, arranged according to the selected layout configuration, within the canvas 104 of the layer tool user interface 902 of the video editing application 206.
Each of the layout project elements (e.g., layout configurations) associated with indicia in the project element carousel 110 includes configuration data, specifying a start time and an end time for application of the layout project element. Accordingly, the automatic application of the selected configuration may be applied between the start time and the end time, with respect to a primary project timeline.
It will also be noted that, with reference to
Of course, using the layout tool and the layer tool user interface 902, a user can apply multiple layout configurations to a common set of media content items 304 or discrete and different sets of media content items 304.
At block 2304, a first video editing mode (e.g., corresponding to one of the video editing tools 214) is activated responsive to user selection of a tool indicia from within the tool navigation bar 118.
At block 2306, and responsive to activation of the first video editing mode, the video editing application 206 generates and causes the display of a corresponding first video editing interface (e.g., such as anyone of video editing user interface 502, clip tool user interface 602, layer tool user interface 902, caption tool user interface 1002, audio tool user interface 1102, create sound user interface 1202, effect tool user interface 1404, transition tool interface 1502, speed tool user interface 1602, timing tool user interface 1702, etc.). The first video editing interface includes a project element display, in the form of the project element carousel 110, which displays and visually distinguishes the project element indicium for an active project element from within multiple project elements represented in the project element carousel 110.
At block 2308, activation of a timing control function (e.g., the timing control 230) is detected and, responsive to detecting activation of the timing control function, a number of graphical elements or control mechanisms are displayed within the context of the first tool editing interface.
Referring to the example interfaces are shown in
The timing control mechanism, in some examples, comprises a first start handle 1704 associated with the abstract timeline 1710 and represents a start time of the active project element within the context of the project duration of the video project 302. The timing control mechanism further comprises a stop handle 1704, also associated with the abstract timeline 802, and representing a stop time of the active project element within the project duration of the video project 302. The start handle 1704 and the stop handle 1704 are movable by a user to adjust the duration of the active project element, as displayed within the timing tool user interface 1702 and relative to the overall duration of the video project 302.
Networked Computing Environment
An interaction client 2402 interacts with other interaction clients 2402 and with the interaction server system 2404 via the network 2408. The data exchanged between the interaction clients 2402 (e.g., interactions 2420) and between the interaction clients 2402 and the interaction server system 2404 includes functions (e.g., commands to invoke functions) and payload data (e.g., text, audio, video, or other multimedia data).
The interaction server system 2404 provides server-side functionality via the network 2408 to the interaction clients 2402. While certain functions of the interaction system 2400 are described herein as being performed by either an interaction client 2402 or by the interaction server system 2404, the location of certain functionality either within the interaction client 2402 or the interaction server system 2404 may be a design choice. For example, it may be technically preferable to initially deploy a particular technology and functionality within the interaction server system 2404 but to later migrate this technology and functionality to the interaction client 2402 where a client system 106 has sufficient processing capacity.
The interaction server system 2404 supports various services and operations that are provided to the interaction clients 2402. Such operations include transmitting data to, receiving data from, and processing data generated by the interaction clients 2402. This data may include message content, client device information, geolocation information, media augmentation and overlays, message content persistence conditions, social network information, and live event information. Data exchanges within the interaction system 2400 are invoked and controlled through functions available via user interfaces (UIs) of the interaction clients 2402.
Turning now specifically to the interaction server system 2404, an Application Program Interface (API) server 2410 is coupled to and provides a programmatic interface to interaction servers 2414. The interaction servers 2414 are communicatively coupled to a database server 2412, facilitating access to a database 2416 that stores data associated with interactions processed by the interaction servers 2414. Similarly, a web server 2418 is coupled to the interaction servers 2414 and provides web-based interfaces to the interaction servers 2414. To this end, the web server 2418 processes incoming network requests over the Hypertext Transfer Protocol (HTTP) and several other related protocols.
The Application Program Interface (API) server 2410 receives and transmits interaction data (e.g., commands and message payloads) between the client system 106 and the interaction servers 2414. Specifically, the Application Program Interface (API) server 2410 provides a set of interfaces (e.g., routines and protocols) that can be called or queried by the interaction client 2402 to invoke the functionality of the interaction servers 2414. The Application Program Interface (API) server 2410 exposes various functions supported by the interaction servers 2414, including account registration, login functionality, the sending of interaction data via the interaction servers 2414, from a particular interaction client 2402 to another interaction client 2402, the communication of media files (e.g., images or video) from an interaction client 2402 to the interaction servers 2414, the settings of a collection of media data (e.g., a story), the retrieval of a list of friends of a user of a client system 106, the retrieval of messages and content, the addition and deletion of entities (e.g., friends) to an entity graph (e.g., a social graph), the location of friends within a social graph, and opening an application event (e.g., relating to the interaction client 2402).
Linked Applications
Returning to the interaction client 2402, features and functions of a linked application (e.g., a linked video editing application 206 or applet) may be made available to a user via an interface of the interaction client 2402. In this context, “external” refers to the fact that the video editing application 206 or applet is external to the interaction client 2402. The linked application may be provided by a third party but may also be provided by the creator or provider of the interaction client 2402. The interaction client 2402 receives a user selection of an option to launch or access features of such a linked application. The linked application may be the video editing application 206 installed on the client system 106 (e.g., as a “native app”), or a small-scale version of the application (e.g., as an “applet”) that is hosted on the client system 106 or remote of the client system 106 (e.g., on third-party servers 2406). The small-scale version of the application includes a subset of features and functions of the application (e.g., the full-scale, native version of the application) and is implemented using a markup-language document. In one example, the small-scale version of the application (e.g., an “applet”) is a web-based, markup-language version of the application and is embedded in the interaction client 2402. In addition to using markup-language documents (e.g., a .*ml file), an applet may incorporate a scripting language (e.g., a .*js file or a .json file) and a style sheet (e.g., a .*ss file).
In response to receiving a user selection of the option to launch or access features of the video editing application 206, the interaction client 2402 determines whether the selected linked application is a web-based linked application or is locally-installed. In some cases, video editing applications 206 that are locally installed on a client system 106 can be launched independently of and separately from the interaction client 2402, such as by selecting an icon, corresponding to the video editing application 206 on a home screen of the client system 106. The video editing application 206 can be launched or accessed via the interaction client 2402 and, in some examples, no or limited portions of the video editing application 206 can be accessed outside of the interaction client 2402. The video editing application 206 can be launched by the interaction client 2402 receiving from a third-party server 2406, for example, a markup-language document associated with the small-scale application and processing such a document.
In response to determining that the linked application is a locally-installed video editing application 206, the interaction client 2402 instructs the client system 106 to launch this resource by executing locally-stored code. In response to determining that the linked application is a web-based resource, the interaction client 2402 communicates with the third-party servers 2406 (for example) to obtain a markup-language document corresponding to the selected linked application. The interaction client 2402 then processes the obtained markup-language document to present the web-based linked application within a user interface of the interaction client 2402.
The interaction client 2402 can notify a user of the client system 106, or other users related to such a user (e.g., “friends”), of activity taking place in the video editing application 206. For example, the interaction client 2402 can provide participants in a conversation (e.g., a chat session) in the interaction client 2402 with notifications relating to the current or recent use of aa video editing application 206 by one or more members of a group of users. One or more users can be invited to join in an active linked application or to launch a recently-used but currently inactive (in the group of friends) linked application. The video editing application 206 can provide participants in a conversation, each using respective interaction clients 2402, with the ability to share an item, status, state, or location in the video editing application 206 with one or more members of a group of users into a chat session.
The interaction client 2402 can present a list of the available linked applications (e.g., video editing applications 206 or applets) to a user to launch or access a given linked application. This list can be presented in a context-sensitive menu. For example, the icons representing different ones of the video editing application 206 (or applets) can vary based on how the menu is launched by the user (e.g., from a conversation interface or from a non-conversation interface).
Machine Architecture
The machine 2500 may include processors 2504, memory 2506, and input/output I/O components 2502, which may be configured to communicate with each other via a bus 2540. In an example, the processors 2504 (e.g., a Central Processing Unit (CPU), a Reduced Instruction Set Computing (RISC) Processor, a Complex Instruction Set Computing (CISC) Processor, a Graphics Processing Unit (GPU), a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Radio-Frequency Integrated Circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 2508 and a processor 2512 that execute the instructions 2510. The term “processor” is intended to include multi-core processors that may comprise two or more independent processors (sometimes referred to as “cores”) that may execute instructions contemporaneously. Although
The memory 2506 includes a main memory 2514, a static memory 2516, and a storage unit 2518, both accessible to the processors 2504 via the bus 2540. The main memory 2506, the static memory 2516, and storage unit 2518 store the instructions 2510 embodying any one or more of the methodologies or functions described herein. The instructions 2510 may also reside, completely or partially, within the main memory 2514, within the static memory 2516, within machine-readable medium 2520 within the storage unit 2518, within at least one of the processors 2504 (e.g., within the Processor's cache memory), or any suitable combination thereof, during execution thereof by the machine 2500.
The I/O components 2502 may include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I/O components 2502 that are included in a particular machine will depend on the type of machine. For example, portable machines such as mobile phones may include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I/O components 2502 may include many other components that are not shown in
In further examples, the I/O components 2502 may include biometric components 2530, motion components 2532, environmental components 2534, or position components 2536, among a wide array of other components. For example, the biometric components 2530 include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye-tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram-based identification), and the like. The motion components 2532 include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope).
The environmental components 2534 include, for example, one or cameras (with still image/photograph and video capabilities), illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometer), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors to detection concentrations of hazardous gases for safety or to measure pollutants in the atmosphere), or other components that may provide indications, measurements, or signals corresponding to a surrounding physical environment.
With respect to cameras, the client system 106 may have a camera system comprising, for example, front cameras on a front surface of the client system 106 and rear cameras on a rear surface of the client system 106. The front cameras may, for example, be used to capture still images and video of a user of the client system 106 (e.g., “selfies”), which may then be augmented with augmentation data (e.g., filters) described above. The rear cameras may, for example, be used to capture still images and videos in a more traditional camera mode, with these images similarly being augmented with augmentation data. In addition to front and rear cameras, the client system 106 may also include a 360° camera for capturing 360° photographs and videos.
Further, the camera system of the client system 106 may include dual rear cameras (e.g., a primary camera as well as a depth-sensing camera), or even triple, quad or penta rear camera configurations on the front and rear sides of the client system 106. These multiple cameras systems may include a wide camera, an ultra-wide camera, a telephoto camera, a macro camera, and a depth sensor, for example.
The position components 2536 include location sensor components (e.g., a GPS receiver component), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude may be derived), orientation sensor components (e.g., magnetometers), and the like.
Communication may be implemented using a wide variety of technologies. The I/O components 2502 further include communication components 2538 operable to couple the machine 2500 to a network 2522 or devices 2524 via respective coupling or connections. For example, the communication components 2538 may include a network interface Component or another suitable device to interface with the network 2522. In further examples, the communication components 2538 may include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), WiFi® components, and other communication components to provide communication via other modalities. The devices 2524 may be another machine or any of a wide variety of peripheral devices (e.g., a peripheral device coupled via a USB).
Moreover, the communication components 2538 may detect identifiers or include components operable to detect identifiers. For example, the communication components 2538 may include Radio Frequency Identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one-dimensional bar codes such as Universal Product Code (UPC) bar code, multi-dimensional bar codes such as Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar code, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information may be derived via the communication components 2538, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that may indicate a particular location, and so forth.
The various memories (e.g., main memory 2514, static memory 2516, and memory of the processors 2504) and storage unit 2518 may store one or more sets of instructions and data structures (e.g., software) embodying or used by any one or more of the methodologies or functions described herein. These instructions (e.g., the instructions 2510), when executed by processors 2504, cause various operations to implement the disclosed examples.
The instructions 2510 may be transmitted or received over the network 2522, using a transmission medium, via a network interface device (e.g., a network interface component included in the communication components 2538) and using any one of several well-known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 2510 may be transmitted or received using a transmission medium via a coupling (e.g., a peer-to-peer coupling) to the devices 2524.
Software Architecture
The operating system 2612 manages hardware resources and provides common services. The operating system 2612 includes, for example, a kernel 2614, services 2616, and drivers 2622. The kernel 2614 acts as an abstraction layer between the hardware and the other software layers. For example, the kernel 2614 provides memory management, processor management (e.g., scheduling), component management, networking, and security settings, among other functionalities. The services 2616 can provide other common services for the other software layers. The drivers 2622 are responsible for controlling or interfacing with the underlying hardware. For instance, the drivers 2622 can include display drivers, camera drivers, BLUETOOTH® or BLUETOOTH® Low Energy drivers, flash memory drivers, serial communication drivers (e.g., USB drivers), WI-FI® drivers, audio drivers, power management drivers, and so forth.
The libraries 2610 provide a common low-level infrastructure used by the applications 2606. The libraries 2610 can include system libraries 2618 (e.g., C standard library) that provide functions such as memory allocation functions, string manipulation functions, mathematic functions, and the like. In addition, the libraries 2610 can include API libraries 2624 such as media libraries (e.g., libraries to support presentation and manipulation of various media formats such as Moving Picture Experts Group-4 (MPEG4), Advanced Video Coding (H.264 or AVC), Moving Picture Experts Group Layer-3 (MP3), Advanced Audio Coding (AAC), Adaptive Multi-Rate (AMR) audio codec, Joint Photographic Experts Group (JPEG or JPG), or Portable Network Graphics (PNG)), graphics libraries (e.g., an OpenGL framework used to render in two dimensions (2D) and three dimensions (3D) in a graphic content on a display), database libraries (e.g., SQLite to provide various relational database functions), web libraries (e.g., WebKit to provide web browsing functionality), and the like. The libraries 2610 can also include a wide variety of other libraries 2628 to provide many other APIs to the applications 2606.
The frameworks 2608 provide a common high-level infrastructure that is used by the applications 2606. For example, the frameworks 2608 provide various graphical user interface (GUI) functions, high-level resource management, and high-level location services. The frameworks 2608 can provide a broad spectrum of other APIs that can be used by the applications 2606, some of which may be specific to a particular operating system or platform.
In an example, the applications 2606 may include a home application 2636, a contacts application 2630, a browser application 2632, a book reader application 2634, a location application 2642, a media application 2644, a messaging application 2646, a game application 2648, and a broad assortment of other applications such as a third-party application 2640. The applications 2606 are programs that execute functions defined in the programs. Various programming languages can be employed to create one or more of the applications 2606, structured in a variety of manners, such as object-oriented programming languages (e.g., Objective-C, Java, or C++) or procedural programming languages (e.g., C or assembly language). In a specific example, the third-party application 2640 (e.g., an application developed using the ANDROID™ or IOS™ software development kit (SDK) by an entity other than the vendor of the particular platform) may be mobile software running on a mobile operating system such as IOS™, ANDROID™, WINDOWS® Phone, or another mobile operating system. In this example, the third-party application 2640 can invoke the API calls 2650 provided by the operating system 2612 to facilitate functionality described herein.
A method to enable video editing of a video project within a video editing application, the method comprising: presenting a video preview display within a user interface of the video editing application, the video preview display presenting a preview video of the video project at a playhead location of project timeline; presenting a project element display within the user interface of the video editing application; presenting a tool display within the user interface of the video editing application; activating a first video editing mode of the video editing application to activate one of a plurality of video editing tools provided by the video editing application; responsive to the activation of the first editing mode, presenting a first set of project element indicia within the project element display of the user interface, each project element indicium of the first set of project element indicia representing a respective project element of a first set of project elements of the video project; detecting user selection of a selected project element indicium of the first set of project element indicia within the project element display of the user interface; and responsive to the detection of the user selection of the selected project element indicium, moving the playhead of the project timeline to an updated playback location corresponding to the selected project element.
The method according to any one of the preceding examples, comprising, responsive to the detection of the user selection of the selected project element indicium, presenting a project element corresponding to the selected project element indicium within the video preview display of the user interface of the video editing application
The method according to any one of the preceding examples, wherein the project element indicia comprise a plurality of thumbnail images, and the selected project element indicium comprises a selected thumbnail image.
The method according to any one of the preceding examples, wherein the updated playback location is a chronological position on the project timeline where the selected project element associated with the selected project element indicium first appears.
The method according to any one of the preceding examples, wherein the first set of project elements represent a subset of project elements of a video project. The method according to any one of the preceding examples, wherein the project elements of the video project comprise at least one of a video content item, an image content item, an audio content item, a caption content item, an effect content item, an augmentation content item, a layout content item, and a transition content item.
A method to navigate a video editing project within a video editing application, the method comprising: detecting activation of a first editing mode of a plurality of editing modes, each of the editing modes being associated with a respective project element set of the video editing project; responsive to activation of the first editing mode, causing presentation, within an element display of the video editing application, of a first project element set associated with the first editing mode; and responsive to detection of selection of a first project element of the first project element set, causing presentation of the first project element in a canvas display the video editing application, and advancing a playhead associated with a canvas display to a location on a project timeline associated with the first project element.
The method according to any one of the preceding examples, wherein the first project element set is caused to be displayed within the element display to the exclusion of project elements of other project element sets of the video editing project.
The method according to any one of the preceding examples, wherein each of the editing modes includes a set of tools specific to the respective video editing mode, and responsive to detecting selection of the first editing mode, causing presentation of a first tool set associated with the first editing mode.
The method according to any one of the preceding examples, responsive to detection of the selection of a second project element of the first project element set, causing presentation of the second project element in a canvas display the video editing application, and advancing the playhead to a second location on a project timeline associated with the second project element.
The method according to any one of the preceding examples, further comprising: detecting activation of a second editing mode of the plurality of editing modes; responsive to activation of the second editing mode, causing presentation, within the element display of the video editing application, of a second project element set associated with the second editing mode; and responsive to detection of a selection of a second project element of the second project element set, causing presentation of the second project element in a canvas display the video editing application, and advancing the playhead to a second location on the project timeline associated with the second project element.
A method to navigate a video editing project within a video editing application, the method comprising: storing a project timeline comprising a plurality of media content items arranged in a chronological sequence, the project timeline having a start time and an end time; causing presentation of a first tool interface, the first tool interface comprising: a video editing canvas for playback of the project timeline; a plurality of tools to enable respective video editing functions; and a plurality of indicia representing the respective plurality of media content items; detecting activation of a first indicium associated with a first media content item of the plurality of media content items; and advancing the playback of the project timeline to a chronological location corresponding to the location of the first media content item within the project timeline.
The method according to any one of the preceding examples, further comprising: storing a supplemental timeline comprising a plurality of media supplementation items, the plurality of media supplementation items supplementing the media content items of the project timeline; causing presentation of a second tool interface, the second tool interface comprising: the video editing canvas; a second plurality of tools to enable a second video editing function; a second plurality of second indicium representing the respective plurality of media supplementation items; detecting activation of a second indicium associated with an activated media supplementation item of the plurality of media supplementation items; responsive detecting the activation of the second indicium, advancing playback of the project timeline to a second chronological location corresponding to a location of the activated media supplementation item within the supplemental timeline.
The method according to any one of the preceding examples, wherein media content items comprising at least one of digital image content items and digital video content items.
The method according to any one of the preceding examples, wherein the plurality of media supplementation items comprise at least one of: supplemental digital image content items or digital video content items, and the supplemental timeline comprises a layer timeline that is overlaid on the project timeline; caption content items, and the supplemental timeline comprises a caption timeline that is overlaid on the project timeline; audio content items, and the supplemental timeline comprises an audio timeline that is overlaid on the project timeline; effect content items, and the supplemental timeline comprises an effect timeline that is overlaid on the project timeline; transition content items, and the supplemental timeline comprises a transition timeline that is overlaid on the project timeline; playback speed adjustment content items, and the supplemental timeline comprises a speed timeline that is overlaid on the project timeline; and a layout content item, and the supplemental timeline comprises a layout timeline that is overlaid on the project timeline.
A method of applying a media item layout within a video editing application, the method comprising: storing a plurality of media items within a project data structure maintained by the video editing application; storing a plurality of layout configurations; causing presentation of the plurality of media items within a media item display of the video editing application; detecting selection of the plurality of media items within the media item display; detecting selection of a selected layout configuration of the plurality of layout configurations; automatically applying the selected layout configuration to the plurality of media items; presenting the plurality of media items, arranged according to the selected layout configuration, within a canvas display of the video editing application.
The method according to any one of the preceding examples, wherein each layout of the plurality of layout configurations comprises configuration data specifying a start time and end time, and wherein the automatic application of the selected layout configuration to the plurality of media items comprises applying the selected configuration between a start time and end time with respect to a primary project timeline.
The method according to any one of the preceding examples, further comprising the automatic applying of the selected layout configuration to the plurality of media items after the end time specified by the configuration data of the selected layout configuration.
The method according to any one of the preceding examples, wherein the plurality of media items are stored within respective content layers within the project data structure maintained by the video editing application.
The method according to any one of the preceding examples, including presenting a plurality of tool functions in conjunction with the presentation of the plurality of media items, each of the plurality of tool functions being selectable to invoke a respective video editing tool to edit a media item of the plurality of media items.
The method according to any one of the preceding examples, wherein at least one layout configuration of the plurality of layout configurations comprises layout data specifying respective locations of the plurality of media items relative a plurality of primary content items included in a primary content timeline.
The method of according to any one of the preceding examples, wherein at least one of the plurality of layout configurations comprises layout data specifying a picture in picture layout, wherein the plurality of media items are displayed as respective pictures overlaid on a primary media content item.
The method of according to any one of the preceding examples, wherein at least one of the plurality of layout configurations comprises layout data specifying a split screen layout, wherein the plurality of media items is displayed in a split screen fashion with a primary media content item.
The method according to any one of the preceding examples, wherein each layout configuration of the plurality of layout configurations comprises layout data specifying respective locations of the plurality of primary content items relative to each other.
The method according to any one of the preceding examples comprising: detecting selection of a further selected layout configuration of the plurality of layout configurations; automatically the automatic applying of the selected layout configuration and applying the further selected layout configuration to the plurality of media items; and causing presentation of the plurality of media items, arranged according to the further selected layout configuration, within the canvas display of the video editing application.
The method according to any one of the preceding examples, wherein the plurality of tools functions include add, trim, split, timing, and delete functions.
A method comprising: storing a video project data structure comprising a plurality of project elements of a video editing project, the video editing project having a project duration; within a video editing application, activating a first editing mode; responsive to the activation of the first editing mode, causing display of a first tool video editing interface including a project element display displaying an active project element of the plurality of project elements; responsive to activation of a timing control function within the first tool video editing interface, causing display of: a duration graphical element depicting a project element duration of the active project element relative to the project duration of the video editing project, and a timing control mechanism, manipulable by a user of the video editing application, to modify the project element duration of the active project element, relative to the project duration of the video editing project.
The method according to any one of the preceding examples, wherein the duration graphical element comprise a duration bar, and the timing control mechanism comprises: a start handle of the duration bar representing a start time of the active project element within the project duration of the video editing project, a stop handle of the duration bar representing a stop time of the active project element within the project duration of the video editing project, the start handle and the stop handle being movable by the user to adjust the duration bar and the project element duration of the active project element displayed in the first tool video editing interface.
The method according to any one of the preceding examples, comprising: detecting an interaction by the user with a selected handle, the selected handle comprising at least one of the start handle and the stop handle within the first tool video editing interface; and responsive to the detecting of the interaction with the selected handle, causing display of a frame-step graphical element associated with the selected handle, the frame-step graphical element being manipulable by the user to move the selected handles a predetermined number of video image frames of the active project element.
The method according to any one of the preceding examples, comprising: detecting an interaction by the user with a selected handle, the selected handle comprising at least one of the start handle and the stop handle within the first tool video editing interface; and responsive to the detecting of the interaction with the selected handle, adjusting the display of the active project element within the project element display to a location with the active project element associated with the selected handle.
The method according to any one of the preceding examples, further comprising: responsive to the activation of the timing control function, causing the presentation of a set of project element indicia representing a set of project elements; detecting selection of a selected project element indicium of the set of project element indicia responsive to detecting selection of the selected project element indicium, causing a project element represented by the selected project element indicium to become a further active project element; causing display of a further duration graphical element depicting a further project element duration of the further active project element relative to the project duration of the video editing project.
The method according to any one of the preceding examples, wherein the further duration graphical element comprises a rectangular duration bar having a first edge representing a start time of the further active project element and a second edge representing a stop time of the further active project element, the method further comprising: detecting an interaction by the user with a selected handle, the selected handle comprising at least one of the start handle and the stop handle within the first tool video editing interface; and responsive to the detecting of the interaction with the selected handle, automatically aligning the selected handle at least one of the first edge and the second edge of the rectangular duration bar to align timing of the active project element and the further active project element.
The method according to any one of the preceding examples, wherein the display of the further duration graphical element comprising superimposing the further duration graphical element over the duration graphical element.
The method according to any one of the preceding examples, wherein the plurality of project elements are each categorized according to a plurality of project element categories, and the set of project elements are selected by application of a category filter to comprise project elements of a first category of the plurality of project element categories.
The method according to any one of the preceding examples, comprising visually indicating the selected project element indicium, within the set of project element indicia, as being associated with the further active project element.
A computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: present a video preview display within a user interface of the video editing application, the video preview display presenting a preview video of the video project at a playhead location of project timeline; present a project element display within the user interface of the video editing application; present a tool display within the user interface of the video editing application; activate a first video editing mode of the video editing application to activate one of a plurality of video editing tools provided by the video editing application; responsive to the activation of the first editing mode, present a first set of project element indicia within the project element display of the user interface, each project element indicium of the first set of project element indicia representing a respective project element of a first set of project elements of the video project; detect user selection of a selected project element indicium of the first set of project element indicia within the project element display of the user interface; and responsive to the detection of the user selection of the selected project element indicium, move the playhead of the project timeline to an updated playback location corresponding to the selected project element.
The computing apparatus according to any one of the preceding examples, comprising, responsive to the detection of the user selection of the selected project element indicium, presenting a project element corresponding to the selected project element indicium within the video preview display of the user interface of the video editing application
The computing apparatus according to any one of the preceding examples, wherein the project element indicia comprise a plurality of thumbnail images, and the selected project element indicium comprises a selected thumbnail image.
The computing apparatus according to any one of the preceding examples, wherein the updated playback location is a chronological position on the project timeline where the selected project element associated with the selected project element indicium first appears.
The computing apparatus according to any one of the preceding examples, wherein the first set of project elements represent a subset of project elements of a video project.
The computing apparatus according to any one of the preceding examples, wherein the project elements of the video project comprise at least one of a video content item, an image content item, an audio content item, a caption content item, an effect content item, an augmentation content item, a layout content item, and a transition content item.
A computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: detect activation of a first editing mode of a plurality of editing modes, each of the editing modes being associated with a respective project element set of the video editing project; responsive to activation of the first editing mode, cause presentation, within an element display of the video editing application, of a first project element set associated with the first editing mode; and responsive to detection of selection of a first project element of the first project element set, cause presentation of the first project element in a canvas display the video editing application, and advance a playhead associated with a canvas display to a location on a project timeline associated with the first project element.
The computing apparatus according to any one of the preceding examples, wherein the first project element set is caused to be displayed within the element display to the exclusion of project elements of other project element sets of the video editing project.
The computing apparatus according to any one of the preceding examples, wherein each of the editing modes includes a set of tools specific to the respective video edit mode, and responsive to detecting selection of the first editing mode, causing presentation of a first tool set associated with the first editing mode.
The computing apparatus according to any one of the preceding examples, responsive to detection of the selection of a second project element of the first project element set, causing presentation of the second project element in a canvas display the video editing application, and advance the playhead to a second location on a project timeline associated with the second project element.
The computing apparatus according to any one of the preceding examples, wherein the instructions further configure the apparatus to: detect activation of a second editing mode of the plurality of editing modes; responsive to activation of the second editing mode, causing presentation, within the element display of the video editing application, of a second project element set associated with the second editing mode; and responsive to detection of a selection of a second project element of the second project element set, cause presentation of the second project element in a canvas display the video editing application, and advancing the playhead to a second location on the project timeline associated with the second project element.
A computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: store a project timeline comprising a plurality of media content items arranged in a chronological sequence, the project timeline having a start time and an end time; cause presentation of a first tool interface, the first tool interface comprising: a video editing canvas for playback of the project timeline; a plurality of tools to enable respective video edit functions; and a plurality of indicia representing the respective plurality of media content items; detect activation of a first indicium associated with a first media content item of the plurality of media content items; and advance the playback of the project timeline to a chronological location corresponding to the location of the first media content item within the project timeline.
The computing apparatus according to any one of the preceding examples, wherein the instructions further configure the apparatus to: store a supplemental timeline comprising a plurality of media supplementation items, the plurality of media supplementation items supplementing the media content items of the project timeline; cause presentation of a second tool interface, the second tool interface comprising: the video editing canvas; a second plurality of tools to enable second respective video edit function; a second plurality of second indicium representing the respective plurality of media supplementation items; detect activation of a second indicium associated with an activated media supplementation item of the plurality of media supplementation items; responsive detect the activation of the second indicium, advancing playback of the project timeline to a second chronological location corresponding to a location of the activated media supplementation item within the supplemental timeline.
The computing apparatus according to any one of the preceding examples, wherein media content items comprising at least one of digital image content items and digital video content items.
The computing apparatus according to any one of the preceding examples, wherein the plurality of media supplementation items comprise at least one of: supplemental digital image content items or digital video content items, and the supplemental timeline comprises a layer timeline that is overlaid on the project timeline; caption content items, and the supplemental timeline comprises a caption timeline that is overlaid on the project timeline; audio content items, and the supplemental timeline comprises an audio timeline that is overlaid on the project timeline; effect content items, and the supplemental timeline comprises an effect timeline that is overlaid on the project timeline; transition content items, and the supplemental timeline comprises a transition timeline that is overlaid on the project timeline; playback speed adjustment content items, and the supplemental timeline comprises a speed timeline that is overlaid on the project timeline; and a layout content item, and the supplemental timeline comprises a layout timeline that is overlaid on the project timeline.
A computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: store a plurality of media items within a project data structure maintained by the video editing application; store a plurality of layout configurations; cause presentation of the plurality of media items within a media item display of the video editing application; detect selection of the plurality of media items within the media item display; detect selection of a selected layout configuration of the plurality of layout configurations; automatically apply the selected layout configuration to the plurality of media items; present the plurality of media items, arranged according to the selected layout configuration, within a canvas display of the video editing application.
The computing apparatus according to any one of the preceding examples 51, wherein each layout of the plurality of layout configurations comprises configuration data specify a start time and end time, and wherein the automatic applying of the selected layout configuration to the plurality of media items comprises applying the selected configuration between a start time and end time with respect to a primary project timeline.
The computing apparatus according to any one of the preceding examples, wherein the instructions further configure the apparatus to the automatic applying of the selected layout configuration to the plurality of media items after the end time specified by the configuration data of the selected layout configuration.
The computing apparatus according to any one of the preceding examples, wherein the plurality of media items are stored within respective content layers within the project data structure maintained by the video editing application.
The computing apparatus according to any one of the preceding examples 51, including presenting a plurality of tool functions in conjunction with the presentation of the plurality of media items, each of the plurality of tool functions being selectable to invoke a respective video editing tool to edit a media item of the plurality of media items.
The computing apparatus according to any one of the preceding examples, wherein at least one layout configuration of the plurality of layout configurations comprises layout data specify respective locations of the plurality of media items relative a plurality of primary content items included in a primary content timeline.
The computing apparatus according to any one of the preceding examples, wherein at least one of the plurality of layout configurations comprises layout data specify a picture in picture layout, wherein the plurality of media items are displayed as respective pictures overlaid on a primary media content item.
The computing apparatus according to any one of the preceding examples, wherein at least one of the plurality of layout configurations comprises layout data specify a split screen layout, wherein the plurality of media items is displayed in a split screen fashion with a primary media content item.
The computing apparatus according to any one of the preceding examples, wherein each layout configuration of the plurality of layout configurations comprises layout data specify respective locations of the plurality of primary content items relative to each other.
The computing apparatus according to any one of the preceding examples, comprising: detect selection of a further selected layout configuration of the plurality of layout configurations; automatically apply the selected layout configuration and applying the further selected layout configuration to the plurality of media items; and cause presentation of the plurality of media items, arranged according to the further selected layout configuration, within the canvas display of the video editing application.
The computing apparatus according to any one of the preceding examples, wherein the plurality of tools functions include add, trim, split, time, and delete functions.
A computing apparatus comprising: a processor; and a memory storing instructions that, when executed by the processor, configure the apparatus to: store a video project data structure comprising a plurality of project elements of a video editing project, the video editing project having a project duration; within a video editing application, activating a first editing mode; responsive to the activation of the first editing mode, cause display of a first tool video editing interface include a project element display displaying an active project element of the plurality of project elements; responsive to activation of a timing control function within the first tool video editing interface, cause display of: a duration graphical element depicting a project element duration of the active project element relative to the project duration of the video editing project, and a timing control mechanism, manipulable by a user of the video editing application, to modify the project element duration of the active project element, relative to the project duration of the video editing project.
The computing apparatus according to any one of the preceding examples, wherein the duration graphical element comprise a duration bar, and the timing control mechanism comprises: a start handle of the duration bar representative of a start time of the active project element within the project duration of the video editing project, a stop handle of the duration bar representative of a stop time of the active project element within the project duration of the video editing project, the start handle and the stop handle being movable by the user to adjust the duration bar and the project element duration of the active project element displayed in the first tool video editing interface.
The computing apparatus according to any one of the preceding examples, comprising: detect an interaction by the user with a selected handle, the selected handle comprising at least one of the start handle and the stop handle within the first tool video editing interface; and responsive to the detecting of the interaction with the selected handle, cause display of a frame-step graphical element associated with the selected handle, the frame-step graphical element being manipulable by the user to move the selected handles a predetermined number of video image frames of the active project element.
The computing apparatus according to any one of the preceding examples, comprising: detect an interaction by the user with a selected handle, the selected handle comprising at least one of the start handle and the stop handle within the first tool video editing interface; and responsive to the detecting of the interaction with the selected handle, adjust the display of the active project element within the project element display to a location with the active project element associated with the selected handle.
The computing apparatus according to any one of the preceding examples, wherein the instructions further configure the apparatus to: responsive to the activation of the timing control function, causing the presentation of a set of project element indicia representing a set of project elements; detect selection of a selected project element indicium of the set of project element indicia responsive to detecting selection of the selected project element indicium, cause a project element represented by the selected project element indicium to become a further active project element; cause display of a further duration graphical element depicting a further project element duration of the further active project element relative to the project duration of the video editing project.
The computing apparatus according to any one of the preceding examples, wherein the further duration graphical element comprises a rectangular duration bar having a first edge representing a start time of the further active project element and a second edge representing a stop time of the further active project element, the method wherein the instructions further configure the apparatus to: detect an interaction by the user with a selected handle, the selected handle comprising at least one of the start handle and the stop handle within the first tool video editing interface; and responsive to the detecting of the interaction with the selected handle, automatically align the selected handle at least one of the first edge and the second edge of the rectangular duration bar to align timing of the active project element and the further active project element.
The computing apparatus according to any one of the preceding examples, wherein the display of the further duration graphical element comprising superimpose the further duration graphical element over the duration graphical element.
The computing apparatus according to any one of the preceding examples, wherein the plurality of project elements are each categorized according to a plurality of project element categories, and the set of project elements are selected by application of a category filter to comprise project elements of a first category of the plurality of project element categories.
The computing apparatus according to any one of the preceding examples, comprising visually indicate the selected project element indicium, within the set of project element indicia, as being associated with the further active project element.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to: present a video preview display within a user interface of the video editing application, the video preview display presenting a preview video of the video project at a playhead location of project timeline; present a project element display within the user interface of the video editing application; present a tool display within the user interface of the video editing application; activate a first video editing mode of the video editing application to activate one of a plurality of video editing tools provided by the video editing application; responsive to the activation of the first editing mode, presenting a first set of project element indicia within the project element display of the user interface, each project element indicium of the first set of project element indicia representing a respective project element of a first set of project elements of the video project; detect user selection of a selected project element indicium of the first set of project element indicia within the project element display of the user interface; and responsive to the detection of the user selection of the selected project element indicium, moving the playhead of the project timeline to an updated playback location corresponding to the selected project element.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to: detect activation of a first editing mode of a plurality of editing modes, each of the editing modes being associated with a respective project element set of the video editing project; responsive to activation of the first editing mode, causing presentation, within an element display of the video editing application, of a first project element set associated with the first editing mode; and responsive to detection of selection of a first project element of the first project element set, causing presentation of the first project element in a canvas display the video editing application, and advance a playhead associated with a canvas display to a location on a project timeline associated with the first project element.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to: store a project timeline comprising a plurality of media content items arranged in a chronological sequence, the project timeline having a start time and an end time; cause presentation of a first tool interface, the first tool interface comprising: a video editing canvas for playback of the project timeline; a plurality of tools to enable respective video edit functions; and a plurality of indicia representing the respective plurality of media content items; detect activation of a first indicium associated with a first media content item of the plurality of media content items; and advance the playback of the project timeline to a chronological location corresponding to the location of the first media content item within the project timeline.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to: store a plurality of media items within a project data structure maintained by the video editing application; store a plurality of layout configurations; cause presentation of the plurality of media items within a media item display of the video editing application; detect selection of the plurality of media items within the media item display; detect selection of a selected layout configuration of the plurality of layout configurations; automatically applying the selected layout configuration to the plurality of media items; present the plurality of media items, arranged according to the selected layout configuration, within a canvas display of the video editing application.
A non-transitory computer-readable storage medium, the computer-readable storage medium including instructions that when executed by a computer, cause the computer to: store a video project data structure comprising a plurality of project elements of a video editing project, the video editing project having a project duration; within a video editing application, activating a first editing mode; responsive to the activation of the first editing mode, causing display of a first tool video editing interface include a project element display displaying an active project element of the plurality of project elements; responsive to activation of a timing control function within the first tool video editing interface, cause display of: a duration graphical element depicting a project element duration of the active project element relative to the project duration of the video editing project, and a timing control mechanism, manipulable by a user of the video editing application, to modify the project element duration of the active project element, relative to the project duration of the video editing project.
Glossary
“Client device” refers to any machine that interfaces to a communications network to obtain resources from one or more server systems or other client devices. A client device may be, but is not limited to, a mobile phone, desktop computer, laptop, portable digital assistants (PDAs), smartphones, tablets, ultrabooks, netbooks, laptops, multi-processor systems, microprocessor-based or programmable consumer electronics, game consoles, set-top boxes, or any other communication device that a user may use to access a network.
“Communication network” refers to one or more portions of a network that may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the Public Switched Telephone Network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, a network or a portion of a network may include a wireless or cellular network and the coupling may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile communications (GSM) connection, or other types of cellular or wireless coupling. In this example, the coupling may implement any of a variety of types of data transfer technology, such as Single Carrier Radio Transmission Technology (1×RTT), Evolution-Data Optimized (EVDO) technology, General Packet Radio Service (GPRS) technology, Enhanced Data rates for GSM Evolution (EDGE) technology, third Generation Partnership Project (3GPP) including 3G, fourth generation wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High Speed Packet Access (HSPA), Worldwide Interoperability for Microwave Access (WiMAX), Long Term Evolution (LTE) standard, others defined by various standard-setting organizations, other long-range protocols, or other data transfer technology.
“Component” refers to a device, physical entity, or logic having boundaries defined by function or subroutine calls, branch points, APIs, or other technologies that provide for the partitioning or modularization of particular processing or control functions. Components may be combined via their interfaces with other components to carry out a machine process. A component may be a packaged functional hardware unit designed for use with other components and a part of a program that usually performs a particular function of related functions. Components may constitute either software components (e.g., code embodied on a machine-readable medium) or hardware components. A “hardware component” is a tangible unit capable of performing certain operations and may be configured or arranged in a certain physical manner. In various examples, one or more computer systems (e.g., a standalone computer system, a client computer system, or a server computer system) or one or more hardware components of a computer system (e.g., a processor or a group of processors) may be configured by software (e.g., an application or application portion) as a hardware component that operates to perform certain operations as described herein. A hardware component may also be implemented mechanically, electronically, or any suitable combination thereof. For example, a hardware component may include dedicated circuitry or logic that is permanently configured to perform certain operations. A hardware component may be a special-purpose processor, such as a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). A hardware component may also include programmable logic or circuitry that is temporarily configured by software to perform certain operations. For example, a hardware component may include software executed by a general-purpose processor or other programmable processor. Once configured by such software, hardware components become specific machines (or specific components of a machine) uniquely tailored to perform the configured functions and are no longer general-purpose processors. It will be appreciated that the decision to implement a hardware component mechanically, in dedicated and permanently configured circuitry, or in temporarily configured circuitry (e.g., configured by software), may be driven by cost and time considerations. Accordingly, the phrase “hardware component” (or “hardware-implemented component”) should be understood to encompass a tangible entity, be that an entity that is physically constructed, permanently configured (e.g., hardwired), or temporarily configured (e.g., programmed) to operate in a certain manner or to perform certain operations described herein. Considering examples in which hardware components are temporarily configured (e.g., programmed), each of the hardware components need not be configured or instantiated at any one instance in time. For example, where a hardware component comprises a general-purpose processor configured by software to become a special-purpose processor, the general-purpose processor may be configured as respectively different special-purpose processors (e.g., comprising different hardware components) at different times. Software accordingly configures a particular processor or processors, for example, to constitute a particular hardware component at one instance of time and to constitute a different hardware component at a different instance of time. Hardware components can provide information to, and receive information from, other hardware components. Accordingly, the described hardware components may be regarded as being communicatively coupled. Where multiple hardware components exist contemporaneously, communications may be achieved through signal transmission (e.g., over appropriate circuits and buses) between or among two or more of the hardware components. In examples in which multiple hardware components are configured or instantiated at different times, communications between such hardware components may be achieved, for example, through the storage and retrieval of information in memory structures to which the multiple hardware components have access. For example, one hardware component may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. A further hardware component may then, at a later time, access the memory device to retrieve and process the stored output. Hardware components may also initiate communications with input or output devices, and can operate on a resource (e.g., a collection of information). The various operations of example methods described herein may be performed, at least partially, by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute processor-implemented components that operate to perform one or more operations or functions described herein. As used herein, “processor-implemented component” refers to a hardware component implemented using one or more processors. Similarly, the methods described herein may be at least partially processor-implemented, with a particular processor or processors being an example of hardware. For example, at least some of the operations of a method may be performed by one or more processors 1004 or processor-implemented components. Moreover, the one or more processors may also operate to support performance of the relevant operations in a “cloud computing” environment or as a “software as a service” (SaaS). For example, at least some of the operations may be performed by a group of computers (as examples of machines including processors), with these operations being accessible via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., an API). The performance of certain of the operations may be distributed among the processors, not only residing within a single machine, but deployed across a number of machines. In some examples, the processors or processor-implemented components may be located in a single geographic location (e.g., within a home environment, an office environment, or a server farm). In other examples, the processors or processor-implemented components may be distributed across a number of geographic locations.
This application claims the benefit of priority to U.S. Provisional Application Ser. No. 63/201,942, filed May 19, 2021; U.S. Provisional Application Ser. No. 63/260,979, filed Sep. 8, 2021; and U.S. Provisional Application Ser. No. 63/261,284, filed Sep. 16, 2021. Each of the above applications are incorporated here in by reference in their entireties.
| Number | Date | Country | |
|---|---|---|---|
| 63201942 | May 2021 | US | |
| 63260979 | Sep 2021 | US | |
| 63261284 | Sep 2021 | US |
