Events such as sporting events and artistic performances are often captured by multiple cameras to provide viewers with different visual perspectives of the event. These multiple cameras may be arranged in a structured pattern or configuration relative to a particular focal point or region to provide viewers with a simulated rotational view about the focal point or region. The particular perspective that is presented to viewers at a given instance is typically controlled by the media organization or entity that is responsible for production of the media content. This form of central control of the media production process is typical of both live and pre-recorded media content.
A video file having a plurality of video segments is obtained by a computing device. Each video segment corresponds to a different camera view of a common temporal event. The computing device initiates playback of the video file within a first video segment corresponding to a first camera view of the common temporal event. Responsive to a user input command, the computing device changes a playback position of the video file from a current frame number of the first video segment to a destination frame number of a second video segment of the video file. The destination frame number has a predefined relationship to the current frame number. The computing devices continues playback of the video file from the destination frame number within the second video segment corresponding to a second camera view of the common temporal event. The second camera view provides a different perspective from the first camera view of a subject captured in the plurality of video segments.
The video file may be created by a computing device by obtaining the plurality of video segments and combining the plurality of video segments according to one or more predefined parameters to obtain the video file. The computing device stores the video file including the plurality of video segments at a storage system. The video file may be served to client computing devices from the storage system by a server device via a communications network.
Claimed subject matter, however, is not limited by this summary as other examples may be disclosed by the following written description and associated drawings.
An interactive platform for the creation and playback control of a video file is disclosed. The video file may have a plurality of video segments that correspond to different camera views of a common temporal event. A user may view a subject captured in these camera views from a number of different perspectives by navigating between corresponding video segments within the video file. Transitions between video segments may be performed according to a transitional process in which a playback position of a destination video segment has a predefined relationship to a current playback position of the video segment presented to the user. The transitional process may support time-registration of the video segments across transitions between camera views as well as support for the presentation of intermediate camera views spatially located between the current camera view and the destination camera view. These and other aspects of the interactive platform will be described in greater detail with reference to the following written description and associated drawings.
Video capture system 100 may include any suitable number camera views provided by respective cameras located at any suitable position and/or orientation relative to a subject.
In at least some implementations, each camera view provided by a respective camera of video capture system 100 may be spaced apart from one or more of the other camera views at intervals relative to subject 190. As one example, at least some of the camera views may be spaced apart from each other at regular intervals. For example, the eight camera views depicted in
While
Additionally, video file 250 may include one or more video segments that do not correspond to the common temporal event. For example, video file 250 may include a pre-roll video segment 260. Pre-roll video segment 260 may include, for example, an advertisement and/or an introduction of video file 250.
In
Each frame may correspond to an individual image of a video file that includes a series of images that are ordered in time. Each of the video segments described herein may have any suitable frame rate (e.g., 10, 30, 60, 120 frames per second). Individual video segments of a video file may have the same or different frame rate as compared to other video segments of the video file. Frame rates may vary within some video segments. For example, a video segment may include a first portion that has a first frame rate that is followed by a second portion that has a second frame rate that is different than the first frame rate. Frame rate may be varied across some video segments responsive to or to account for relative motion of a subject captured by the camera view of the video segment. For example, frame rate may be increased for portions of the video segment where the subject is moving at a higher speed.
Referring to
Second video segment 320 includes a frame set 322 that includes a key-frame 324 that is followed by a number of other frames (i.e., non-key frames), including example frame 325. Accordingly,
In at least some implementations, each key-frame of a first video segment may be in time registration (e.g., capturing the subject at the same or substantially the same instance) with at least one corresponding key-frame of a second video segment with respect to the common temporal event. For example, key-frame 314 of first video segment 310 may be in time registration with key-frame 324 of second video segment 320. Similarly, each frame of a first video segment may be in time registration with at least one corresponding frame of a second video segment. For example, frame 315 of first video segment 310 may be in time registration with frame 325 of second video segment 320. Time registration will be described in greater detail with reference to
In at least some implementations, key-frames and/or non-key frames of a video segment may not be in time registration with key-frames and/or non-key frames of one or more other video segments. As one example, a time registration of key-frame 324 of second video segment 320 may be offset (e.g., time-shifted) from key-frame 314 of first video segment 310 by a time offset value. This time offset value may be less than an entire frame in duration or may correspond to one or more discrete frames in duration. For example, key-frame 324 of second video segment 320 may be in time registration with non-key frame 315 of first video segment 310. As another example, key-frame 318 may be time-shifted by one half of the frame rate in duration so that key-frame 318 partially overlaps in time with key-frame 314 and non-key frame 315 of first video segment 310. As yet another example, key-frames of a second video segment may be in time registration with key-frames of a first video segment, while non-key frames of the second video segment are not in time registration with non-key frames of the first video segment. For example, second video segment 320 may include a different number of non-key frames per frame set 322 (e.g., non-key frames having a longer or shorter duration) than non-key frames per frame set 312. However, the total length of time of frame set 312 may be equal to the total length of time of frame set 322 to provide time registration of key-frames across some or all of the video segments. Offsets in key-frames and/or non-key frames of a video segment may be in either time direction relative to key-frames and/or non-key frames of other video segments.
Playback may be initiated within a first video segment. A transition between video segments of the video file may include changing a playback position of the video file from a current frame number of first video segment 410 to a destination frame number of second video segment 420 of the video file. The transition may be initiated responsive to a user input command. Playback of the video file may be continued from the destination frame number within the second video segment. The destination frame number may have a predefined relationship to the current frame number as will be subsequently described in greater detail.
As one example, the current frame number may correspond to frame 413 and the destination frame number may correspond to frame 422. The predefined relationship may define the same frame number relative to a beginning frame of each video segment, for example, if each video segment has the same number of frames and frame rate. Alternatively or additionally, the current frame number (e.g., frame 413) of first video segment 410 may be in time registration with the destination frame number (e.g., frame 422) of second video segment 420 with respect to a common temporal event. This type of transition may be used to maintain the same frame number and/or same time registration across two video segments.
As another example, the current frame number may correspond to frame 413 and the destination frame number may correspond to frame 424. Here, the predefined relationship may define the destination frame number (e.g., frame 424) as being immediately subsequent to a frame number (e.g., frame 422) of second video segment 420 that is in time registration with the current frame number (e.g., frame 413) of first video segment 410 with respect to the common temporal event. This type of transition may be used to maintain a time ordered sequence of frames across two video segments.
As yet another example, transitions may include a delay imposed in response to a user input command before changing the playback position of the video file. Here, playback of the video file may be continued within the first video segment until the current frame reaches a frame having a predefined position and/or frame type (e.g., key-frame or non-key frame) within the first video segment. The frame of the first video segment may include the next key-frame or a non-key frame preceding the next key-frame. For example, responsive to a user input command during playback of frame 413, playback may continue from frame 413 to 414 before the playback position is changed from first video segment 410 to a destination frame of second video segment 420 (e.g., frame 426 or frame 428). Here, the frame having the predefined position relative to a frame of the first video segment may be defined as a key-frame (e.g., key-frame 426) or a frame subsequent to a key-frame (e.g., non-key frame 428). This type of transition enables coordination among two video segments with respect to the key-frames. For example, if the key-frames of the video segments are in time registration with each other, but the non-key frames are not in time registration with each other, then transitions between video segments with respect to the key-frames may be used to maintain the same frame number and/or same time registration across the two video segments.
In at least some implementations, the first and second transitions of
In at least some implementations, a user may change a playback position of the video file, for example, by dragging the playback position indicator 712 (e.g., also a graphical control element) along the scrub bar in a forward or reverse direction. The scrub bar may graphically indicate a plurality of individual video segments of the video file. For example, a graphical indicator 714 may correspond to previously described video segment 210 of
In at least some implementations, a user may change a playback position within an individual video segment, for example, by dragging the playback position indicator 812 along the scrub bar graphically indicated at 810 in a forward or reverse direction. GUI 800 may further include one or more graphical control elements for changing camera views within a video segment. For example, GUI 800 includes a graphical control element 820. A user may direct a user input at graphical control element 820 to change a playback position of the video file from the current playback position within a first video segment to a destination playback position (e.g., the same or similar corresponding playback position relative to the common temporal event) of a second video segment.
GUI 800 is depicted as including a left arrow, a right arrow (e.g., graphical control element 820), an up arrow, and a down arrow which may enable a user to spatially navigate among a plurality of cameras or camera views positioned at different locations and/or orientations relative to a subject. A user may view a subject from a different perspective or camera view captured, for example, by a camera located to the right of the currently presented camera view by directing a user input at the right arrow (e.g., graphical control element 820). As another example, a user may view the subject from a different perspective or camera view captured, for example, by a camera located at a higher elevation relative to the current camera view by directing a user input at the up arrow.
Graphical control element 922 may have a different appearance from other graphical control elements to indicate to the user that the current playback position of the video file is within a video segment that corresponds to that camera or camera view (e.g., camera or camera view “5”). A user may view a subject from a different perspective or camera view captured, for example, by a camera (e.g., camera “1”) by directing a user input at graphical control element 924.
At 1010, the method may include obtaining a video file having a plurality of video segments. The video file may include or may be accompanied by audio information and/or metadata. At 1012, the method may include initiating playback of the video file within a first video segment corresponding to a first camera view of the common temporal event. At 1014, the method may include, responsive to a user input command, changing a playback position of the video file from a current frame number of the first video segment to a destination frame number of a second video segment of the video file.
It should be understood that the terms “first” video segment and “second” video segment as used herein do not necessarily denote the physical location or position of the video segment within the video file. For example, the first video segment may correspond to the Nth video segment of the video file, and the second video segment may correspond to the Nth minus one or more video segments, or to the Nth plus one or more video segments of the video file relative to the first video segment. Accordingly, the terms “first” and “second” may be used herein to distinguish the two video segments.
As previously described with reference to
In at least some implementations, the destination frame may be selected so that it is as close to the current frame as possible while still occurring at the same or later absolute time relative to the current frame to provide a smooth transition between video segments. As one example, the predefined relationship may define the same frame number relative to a beginning frame of each video segment. As another example, the predefined relationship may define a different frame number relative to a beginning frame of each video segment, whereby the destination frame number is offset from the current frame number by a predefined number of frames. For example, the predefined relationship of the destination frame number to the current frame number may define the destination frame number as being immediately subsequent to a frame number of the second video segment that is in time registration (or out of time registration) with the current frame number of the first video segment with respect to the common temporal event.
At 1016, the method may include continuing playback of the video file from the destination frame number within the second video segment corresponding to a second camera view of the common temporal event to provide a different perspective of a subject captured in the plurality of video segments. As previously discussed, the method may further include delaying changing the playback position of the video file and continuing playback of the video file within the first video segment until the current frame reaches a frame having a predefined position relative to a key-frame of the first video segment.
Method 1000 may be applied to transitions between more than two video segments of a video file. For example, the method may further include, responsive to another user input command, changing a playback position of the video file from a current frame number of the second video segment to a destination frame number of a third video segment. The destination frame number of the third video segment may also have a predefined relationship to the current frame number of the second video segment.
The method may further include continuing playback of the video file from the destination frame number within the third video segment corresponding to a third camera view of the common temporal event. Here, the first camera view may be positioned closer to the second camera view than the third camera view. Alternatively or additionally, the second camera view may be positioned between the first camera view and the third camera view along an arc having a focal point that includes the subject captured in the plurality of video segments. Accordingly, the second video segment may provide one or more transitional frames between playback of the first video segment and the third video segment.
At 1110, the method may include obtaining a plurality of video segments. Each video segment may correspond to a different camera view of a common temporal event. At 1112, the method may include combining the plurality of video segments according to one or more predefined parameters to obtain a video file. As one example, the method at 1112 may include inserting a plurality of key-frame indicators into the video file. The plurality of key-frame indicators may designate a plurality of key-frames spaced apart among frames of each video segment. At least one key-frame of each video segment may correspond to a time event of the video segment that is shared with (e.g., in time registration with) corresponding key-frames of the other video segments of the video file. Time registration of video segments or key-frames within video segments may be achieved by detecting an audio event or audio information within an audio component that is common to each of the video segments. The method at 1112 may further include encoding the video file, for example, by application of a codec. In at least some implementations, the codec may be applied to create or otherwise designate key-frames and non-key frames within the video file or video segments.
At 1114, the method may include storing the video file including the plurality of video segments at a storage device or storage system. At 1116, the method may include receiving a request for the video file from a client computing device via a communications network. At 1118, the method may include sending the video file to the client device via the communications network responsive to the request. The video file may include or may be accompanied by audio information and/or metadata. The method at 1116 and 1118 may not be performed, for example, if the computing device performing method 1100 is the client computing device or a stand-alone computing device operated by a user. Alternatively, method 1100 may be performed by a client computing device at 1110, 1112, and 1114, and may be performed by a server device or server system at 1116 and 1118, for example, responsive to a request initiated by the client computing device.
In at least some implementations, the method at 1112 may further include obtaining a plurality of camera position and/or orientation indicators. Each camera position and/or orientation indicator may define a camera or camera view position and/or orientation for an individual video segment. The plurality of video segments may be combined by ordering the plurality of video segments within the video file based, at least in part, on the relative positioning of the camera or camera view position and/or orientation indicated by the plurality of camera position and/or orientation indicators. For example, if a number of cameras are arranged along a circle, ellipse, or arc surrounding or partially surrounding a subject, then the video segments corresponding to these cameras may be ordered within the video file according to the clockwise or counter-clockwise order of the cameras.
In at least some implementations, the method at 1112 may further include designating at least one frame in each video segment as a key-frame. Each key-frame may correspond to a shared time event (e.g., in time registration) across each video segment. Combining the plurality of video segments to obtain the video file may include concatenating the plurality of video segments with respect to the key-frames. For example, as previously described with reference to
Server system 1210 may include one or more server devices. Two or more server devices may take the form of a distributed server system in some implementations. Accordingly, communications between two or more server devices may include communication via communications network 1230. Server system 1210 includes a storage system 1240 holding instructions 1242 and a data store 1244. Server system 1210 includes one or more processors (e.g., processor 1246) to execute instructions (e.g., instructions 1242). Instructions 1242 to may include or take the form of one or more application programs, an operating system, firmware, and/or other suitable instruction set. As a non-limiting example, instructions 1242 may include a media management module 1260. Media management module 1260 may be configured to perform one or more of the methods, functions, and/or operations described herein with respect to a server system or server device, including methods 1000 and 1100. For example, media management module 1260 may be configured to receive information from and transmit information to a GUI, such as GUI 1300 of
Client device 1220 is a non-limiting example of a client device. It will be understood that computing system 1200 may include any suitable number of client devices. Client device 1220 includes a storage system 1250 holding instructions 1252 and a data store 1254. Client device 1220 includes one or more processors (e.g., processor 1256) to execute instructions (e.g., instructions 1252). Instructions 1252 to may include or take the form of one or more application programs, an operating system, firmware, and/or other suitable instruction set. As a non-limiting example, instructions 1252 may include a media application program 1262, a browser application program 1264, and/or a media management module 1266. Media application program 1262, browser application program 1264, and/or media management module 1266 may be configured to perform one or more of the methods, functions, and/or operations described herein with respect to a client computing device or stand-alone computing device operated by a user, including methods 1000 and 1100.
Media application program 1262 may include or take the form of a general purpose media application program or a special purpose media application program that is specifically configured to present the video files described herein that include a plurality of video segments. In some implementations, a general purpose media application program may playback the video file disclosed herein and enable navigation within the video file without the need for specialized codecs or plugins (e.g., such as Flash). This media application program may be configured to identify the current video playback position of the video file and support the ability for the user to change the current playback position of the video file to change the camera view that is presented to the user. Browser application program 1264 may include or take the form of a general purpose web browser or a general purpose file browser that includes a media player function, or may include or take the form of a special purpose web browser or special purpose file browser that is specifically configured to present the video files described herein that include a plurality of video segments. Again, a general purpose browser program may, in some implementations, playback the video file disclosed herein and enable navigation within the video file without the need for specialized codecs or plugins. As one example, media application program 1262 and/or browser application program 1264 may be configured to present a GUI, such as GUIs 700, 800, and 900 of
Media management module 1266 may be configured to receive information from and present information at a GUI, such as GUI 1300 of
Client device 1220 may include input/output devices 1258. Non-limiting examples of input/output devices 1258 may include a keyboard or keypad, a computer mouse or other suitable controller, a graphical display device, a touch-sensitive graphical display device, a microphone, an audio speaker, an optical camera or sensor, among other suitable input and/or output devices. The GUIs described herein may be presented via a graphical display device, for example.
GUI 1300 further enables a user to associate each video segment with a respective camera view by defining or specifying a position and/or orientation of each camera or camera view. For example, GUI 1300 may include a number of graphical control elements (e.g., 1312, 1322, 1332, 1342, etc.) for defining or specifying a position and/or orientation of each camera or camera view. The position and/or orientation may be in two-dimensional or three-dimensional space. GUI 1300 may further enable a user to associate audio information (e.g., audio segments) with respective video segments or camera views.
GUI 1300 may further enable a user to specify or define file format and/or presentation control parameters. For example, GUI 1300 may include a plurality of control elements for receiving file format and/or presentation control parameters. These control elements may include or take the form of one or more graphical controls and/or text fields. Non-limiting examples of these control elements include: a control element 1350 for defining a key-frame spacing, a control element 1352 for defining a frame rate of the video file, a control element 1354 for defining a file format type (e.g., .mpeg, .wmv, etc.), a control element 1356 for defining a codec type for encoding and/or decoding the video file, a control element 1360 for defining a media player type (e.g., web-browser embedded media player, special purpose media player, etc.), a control element 1362 for defining a transition type (e.g., to select from one or more of the transitions described herein), a control element 1364 for defining a user interface type (e.g., one or more of the video presentation and control GUIs described herein), and a control element 1366 for defining a pre-roll type (e.g., introduction, advertisement, etc.).
GUI 1300 may further enable a user to create a video file defined by one or more of the predefined parameters set by the user or set on behalf of the user by directing a user input at control element 1370. GUI 1300 may further enable a user to save the settings defined by the user via GUI 1300 to a user profile stored at a storage system or storage device for later implementation.
Some or all of the information depicted in
The disclosed embodiments may be used in combination with audio information to provide a multi-media experience. As one example, the video file obtained at 1010 of method 1000 (e.g., by a media application or browser application of a client device) may include one or more audio components or may be accompanied by a separate audio file. If the video file includes one or more audio components, these audio components may take the form of a plurality of audio segments each corresponding to a respective video segment of the video file, or these audio components may take the form of a combination audio segment of the two or more audio segments.
A combined audio segment that forms a component of the video file or a separate audio file that accompanies the video file may be created or otherwise obtained by application of method 1100. As one example, a plurality of audio segments may be obtained at 1110 of method 1000. The plurality of audio segments may each correspond to a respective video segment of the video file (e.g., as different camera views). At 1112 of method 1100, the plurality of audio segments may be combined to obtain either a combination audio segment that forms an audio component of the video file, or a separate audio file that may accompany the video file.
Audio segments corresponding to respective video segments of the video file may be combined in any suitable manner. As one example, the combination of audio segments may be performed at the server system by the server based media management module at the time of video file creation. As another example, the combination of audio segments may be performed at the client device by the client device based media management module at the time of video file creation, or by the media or browser application program at the client device at the time of playback or presentation of the video file and associated audio information.
As one example, an audio file may be created that includes each of the audio segments corresponding to the video segments of the video file. The media management module at the server system or client device may be responsible for creation of the audio file or combination audio segment. Alternatively or additionally, the browser or media application program at the client device may be configured to select one or more of the audio segments from the audio file for presentation at the time of playback of the video file. The one or more audio segments that are selected may correspond to the particular video segment (e.g., camera view) that is being played by the media or browser application program. As another example, some or all of the audio segments may be mixed into a single multi-channel audio segment or file.
In at least some implementations, method 1100 may further include generating metadata that creates an association between the video file and the separate audio file. The metadata may be included as part of the video file and/or the audio file, or may take the form of a separate metadata file. As previously described with reference to
Accordingly, one or more of the following audio/video combinations may be supported by the media management module, media application program, and/or browser application programs disclosed herein: (1) a video file that includes a combination audio segment, (2) a video file that includes a plurality of audio segments that may be individually selected for playback, (3) an audio file and a separate video file that includes metadata associating the video file with the audio file, (4) a video file and a separate audio file that includes metadata associating the audio file with the video file, (5) a video file, a separate audio file, and a separate metadata file that associates the video file with the audio file, or (6) combinations thereof.
Presentation of audio information may take various forms, including static audio or dynamically changing audio responsive to the selected video segment (e.g., camera view). As an example of dynamically changing audio, a single audio segment may be presented at a given time in which the single audio segment may correspond to the selected video segment of the current playback position of the video file. As another example of dynamically changing audio, a plurality of audio segments may be presented at a given time in which the plurality of audio segments may take the form of a multi-channel audio presentation providing stereo audio (e.g., 2 channel) or multi-channel (e.g., 3, 4, 5, 6 or more channel) surround sound. Here, the selection of audio segments and/or the relative mix of the audio segments may correspond to the selected video segment of the current playback position of the video file. The selection of audio segments and/or relative mix of audio segments may change as the user navigates to a different camera view. As an example of static audio, the same audio segment or combination (e.g., stereo or multi-channel surround sound) of audio segments may be presented for some (e.g., two or more different camera views) or all of the video segments of the video file. In at least some implementations, the audio file or other suitable combination of audio segments may be of shorter duration (e.g., time length) than the video file. For example, the same audio information (e.g., with the same or different relative mix) may be repeated multiple times across playback of the entire video file among the various video segments. In some implementations, the video file may include or may be accompanied by multiple audio files. For example, a shorter audio file (e.g., lopped for each video segment) may include audio information corresponding to the video segments of the video file, and a longer audio file that includes voice-over audio information that is played over the length of the video file across multiple video segments.
The disclosed embodiments may be used in combination with three-dimensional (3-D video) to enable a user to change perspective relative to a subject. For example, video segments obtained from two or more cameras may be combined to obtain a 3-D video segment. The video file disclosed herein may include a plurality of 3-D video segments, where each 3-D video segment is formed from a different combination of camera views. In some implementations, this combination of camera views for obtaining a 3-D view of the subject may be performed by pre-processing the video segments prior to or at the time of creation of the video file (e.g., at method 1112 of
While many of the disclosed embodiments were presented in the context of video segments of a common temporal event and/or subject, it will be understood that these video segments may be associated with different temporal events and/or subjects. As one example, the video segments may take the form of advertisements having related or unrelated content. The techniques described herein may similarly enable users to create video files and/or navigate among a plurality of different video segments of the video file, including advertisements or other video content.
It should be understood that the embodiments disclosed herein are illustrative and not restrictive, since the scope of the invention is defined by the following claims rather than by the description preceding them. All changes that fall within metes and bounds of the claims or equivalence of such metes and bounds thereof are therefore intended to be embraced by the claims.