Patent Application
20140237019
1. Field
The disclosed embodiments generally relate to techniques for sharing media files, such as video or audio files. More specifically, the disclosed embodiments relate to a system that facilitates sharing of media files by providing a media stream that is transcoded based on channel performance.
2. Related Art
Users of an online content management system, such as the Dropbox™ service, provided by Dropbox, Inc. of San Francisco, Calif., often desire to stream content from stored files. For example, a user may stream a video or audio (and, more generally, a type of media) to their client device so that the user can view the video or listen to the audio on their client device in real-time. Typically, such real-time playing of media can be facilitated by pre-transcoding a stored media file at a desired data rate and quality level.
However, pre-transcoding the media file can be time-consuming and expensive. In particular, the content management system may not know in advance which stored media file(s) are potentially of interest to the user at any given time. As a consequence, many files may need to be pre-transcoded, which can consume valuable resources and may significantly increase storage costs. These challenges are often compounded by uncertainty about the communication channel performance which affects the desired data rate and quality level, and as a consequence may necessitate pre-transcoding multiple versions of a stored media file.
Alternatively, a stored media file may be transcoded in real-time. However, it is often difficult for the client device to monitor the channel performance and communicate the desired data rate and quality level to the content management system. As a consequence, the content management system may transcode the stored media file based on a pre-defined data rate and quality level. This may result in a media stream which is transcoded at a quality level that does not match the channel capacity, which can degrade the user's experience.
The disclosed embodiments relate to a feature of a file-communication system which improves the user experience by allowing the performance of a channel that is used to stream media to a client from the system to be used as a factor in modifying the data rate and/or the resolution of the media. In particular, an indicator of the channel performance is determined on the server side and then, based on this indicator, one or more additional transcoding operations may be spawned. This approach allows the user to stream media from an online content management system in a way that matches channel performance.
The following description is presented to enable any person skilled in the art to make and use the present embodiments, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present embodiments. Thus, the present embodiments are not limited to the embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein.
The data structures and code described in this detailed description are typically stored on a computer-readable storage medium, which may be any device or medium that can store code and/or data for use by a computer system. The computer-readable storage medium includes, but is not limited to, volatile memory, non-volatile memory, magnetic and optical storage devices such as disk drives, magnetic tape, CDs (compact discs), DVDs (digital versatile discs or digital video discs), or other media capable of storing computer-readable media now known or later developed.
The methods and processes described in the detailed description section can be embodied as code and/or data, which can be stored in a computer-readable storage medium as described above. When a computer system reads and executes the code and/or data stored on the computer-readable storage medium, the computer system performs the methods and processes embodied as data structures and code and stored within the computer-readable storage medium. Furthermore, the methods and processes described below can be included in hardware modules. For example, the hardware modules can include, but are not limited to, application-specific integrated circuit (ASIC) chips, field-programmable gate arrays (FPGAs), and other programmable-logic devices now known or later developed. When the hardware modules are activated, the hardware modules perform the methods and processes included within the hardware modules.
Client Devices
In various embodiments, each client device 110 may selectively execute an online-service client application 122A and 122B (collectively 122) (also referred to as an “online client”), which may be used to manage files stored within a content management system 118. It is noted that, in some embodiments, synchronized copies of files 124A and 124B may be kept on content management system 118 and each respective client device 110. In some embodiments, client devices 110 may provide a file-browser type interface (not shown) for directly manipulating the files stored on content management system 118 without maintaining a local copy. Online clients 122 may also include functionality for receiving and sending messages to a notification module 114.
While only two client devices 110A and 110B are shown in
Content Management System
Content management system 118 stores files and manages access to those files via client devices 110. Content management system 118 and its components may be implemented using any appropriate hardware and software for file serving, storage, and retrieval functions. For example, content management system 118 may be implemented in the form of a single server or multiple servers.
In various embodiments, content management system 118 includes: notification module 114, interface module 128, account module 130, media module 132, and data store 134. Each of these elements of content management system 118 is discussed below.
Content Management System—Notification Module
Notification module 114 may communicate information with client devices 110 and, in particular, with online clients 122. For example, as described further below, notification module 114 may notify one of online clients 122 about a change in the quality level of a media stream from content management system 118 to one of client devices 110.
Content Management System—Interface Module
In particular embodiments, interface module 128 may facilitate file access and file storage among content management system 118 and client devices 110. Interface module 128 may receive files from and send files to client devices 110 consistent with the user's preferences for sharing files. Interface module 128 may act as the counterpart to a client-side file-explorer style user interface that allows a user to manipulate files directly stored on content management system 118. In some embodiments, software operating on client devices 110 may integrate network-stored files with the client's local file system to enable a user to manipulate network-stored files through the same user interface (UI) used to manipulate files on the local file system, e.g., via a file explorer, file finder or browser application. As an alternative or supplement to the client-side file-explorer interface, interface module 128 may provide a web interface for client devices 110 to access (e.g., via a suitable online client 122) and allow a user to manipulate files stored within content management system 118. In this way, the user can directly manipulate files stored within content management system 118.
Content Management System—Data Store
In various embodiments, data store 134 may store files such as those uploaded using client devices 110, or using any other suitable computing device. In the embodiment illustrated in
In various embodiments, data store 134 may maintain information identifying the user, information describing the user's file directory, and other information in a file journal that is maintained for each user. In some embodiments, the file journal may be maintained on content management system 118, and in other embodiments, a file journal (e.g., a ‘server-side file journal’) may be maintained on both content management system 118 and locally on each client device 110. In various embodiments, the file journal may be used to facilitate the synchronization of the various copies of a particular file that are associated with a user's account.
As a particular example, in the embodiment shown in
Content Management System—Account Module
In particular embodiments, account module 130 may track files stored in data store 134 and entries in the server-side file journal for each file. As users grant file access permissions to other users, account module 130 may update the server-side file journal associated with each relevant user in data store 134. Account module 130 may also track client devices 110 that are associated with each user's account. For example, a user may want to share all their files among their desktop computer, tablet computer, and mobile device. To make such a sharing arrangement seamless to the user, the user's single account on content management system 118 may be associated with each of the user's respective client devices. In some embodiments, an application running on each respective client device 110 may help to coordinate synchronization of files on the client device with corresponding versions of the files within the user's account in content management system 118, and also with corresponding versions of the files stored on the user's various other client devices.
Content Management System—Media Module
In various embodiments, media module 132 may use one or more transcoding devices to transcode one or more of the files that are stored in data store 134 into media 136 (such as video, audio, documents or files). (More generally, media module 132 may transcode one or more content items that are included in files or containers in data store 134.) These transcoded media 136 may have associated quality levels, which are associated with specific data rates and resolutions supported by the channel and/or the client device. For example, media 136 may provide a set of quality levels for specific media. In some embodiments, media 136 are transcoded in real-time, i.e., concurrently as interface module 128 streams at least one of media 136 to one of client devices 110. Alternatively, media 136 may, at least in part, be pre-transcoded. This pre-transcoding may be based on a use context of the one or more files. For example, if a user of one of client devices 110 uses online client 122 to access, stream or modify a file in a container or a folder, the other files (and, more generally, content or content items) in this container may be pre-transcoded, because it is more likely that the user will access these other files in the near future.
Moreover, media module 132 may also spawn one or more additional transcoding operations on one or more media files based on an indicator of channel performance (in the one or more networks 120) while streaming media to one of client devices 110. This indicator may be determined by interface module 128. For example, media module 132 may spawn the one or more additional transcoding operations to allow media 138, having a different quality level from those of media 136, to be streamed to one of client devices 110. Prior to transitioning the streaming from one of media 136 to media 138, media module 132 may instruct notification module 114 to alert the one of client devices 110 of the change in the quality.
In particular, the user of client device 110A may use online client 122A to request that content management system 118 stream media in
Alternatively or additionally, the system may perform the transcoding operations on the media prior to the streaming of the media (operation 210). Thus, media 136 in
Moreover, in response to the user's request, notification module 114 may provide information that specifies the location of the media in data store 134 to online client 122A in
Next, the system obtains an indicator of performance for the channel (operation 214). This indicator may be based on performance of the channel as determined from content management system 118. For example, the indicator may be based on the downloading of segments of the media in less time than that needed by the client to view or listen to (and, more generally, play) the media in real-time at the quality level. In the case of video, interface module 128 and/or media module 132 in
Similarly, the indicator may include an inferred available bandwidth of the channel based on the timing of the download requests and associated acknowledgments from the client. In particular, the timing of the download requests for segments and the associated acknowledgments from online client 122A in
Note that the indicator of performance may be based on a wide variety of metrics, including an estimate of: the signal-to-noise ratio, the error rate, a mean-square error relative to a target pattern, and/or a size of an opening in an eye pattern. For example, repeated download requests followed by messages that the data was not received may indicate that the current quality level is too high for the channel to support.
Based on the indicator, the system spawns an additional transcoding operation at a different quality level (operation 216) than is available in the set of quality levels. For example, media module 132 in
In some embodiments, the system optionally provides a notification to the client of a change from the quality level to the different quality level (operation 218). For example, notification module 114 in
In some embodiments, the user is able to override the change in the quality level. For example, even though a portable electronic device and a network are able to accommodate a higher quality level, the user may not want the higher quality level to be used because it may consume more of their data plan. In this case, when notification module 114 in
Next, the system streams the media at the different quality level to the client via the channel (operation 220). For example, interface module 128 and/or media module 132 in
By allowing adaptation of the transcoded media using server-side intelligence, process 200 may allow the playing quality to be optimized. In particular, the system may obtain the indicator without receiving direct feedback about the performance (such as the error rate or the signal-to-noise ratio) from the client. In this way, the system may be able to infer what the client is doing and/or what the client will do next. This streaming process may eliminate the need to pre-transcode a significant number of files, thereby conserving valuable system resources and reducing storage requirements. In addition, by providing improved media quality, process 200 enhances the user's experience.
While video is used as an illustration of the streaming process in the preceding discussion, in other embodiments additional types of files (such as audio files) and associated transcoding techniques may be used.
Additional details about process 200 are further illustrated in
Then, the user makes a download request (operation 312). After this request is received (operation 314), the transcoding may optionally commence (operation 316). (Alternatively or additionally, the media at the different quality levels may, at least in part, be pre-transcoded.)
Next, content management system 118 provides a URL that specifies the location of the transcoded media file in content management system 118. For example, a web page associated with the URL may be displayed on a browser on client device 110A. When the user activates an icon or a button in the web page, the media may be streamed (operation 318) by content management system 118. This media stream may be received and played (operation 320) by client device 110A.
Subsequently, content management system 118 obtains the indicator of the performance (operation 322) of the channel. Based on the indicator, content management system 118 may spawn an additional transcoding operation (operation 324) at a different quality level. Furthermore, a notification may optionally be provided (operation 326) from content management system 118 and may be optionally received (operation 328) at client device 110A.
Then, content management system 118 streams the media (operation 330) at the different quality level. This media stream is then received and played (operation 332) by client device 110A.
The foregoing descriptions of embodiments have been presented for purposes of illustration and description only. They are not intended to be exhaustive or to limit the present description to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art. Additionally, the above disclosure is not intended to limit the present description. The scope of the present description is defined by the appended claims.
