Patent Application
20140032718
This description relates to video uploading from mobile devices such as smartphones and tablets and a wide variety of other mobile devices.
Modern smartphones and tablets have high-resolution video cameras capable of shooting videos in high-definition (e.g., 1080 p) format. The resulting videos, which are stored on the local persistent memory of the device, can be large—often over 100 MB per minute of video.
People often want to perform the following actions with a video they produced on a mobile device: (1) share the video with friends and family; (2) archive the video using a reliable, remote Internet-connected storage service.
Both actions require transferring the video from the mobile device to a remote Internet-connected server. The remote server will store the videos and serve them to, among other things, Internet-connected devices, including smartphones, tablets, laptops, and IP-enabled TVs.
Video transfer from a smartphone or tablet to a remote server can be a complicated process. Most email applications forbid users from attaching a large file to an email: a common limit is 20MB, which equates to only a few seconds of high-quality video. Even if you could find a software tool for transmitting the video, uploading the video from a mobile device through a typical cellular data connection can take hours.
Some software products compress videos to an extremely low quality format (e.g., reduce the resolution from 1080 pixels wide to 360 pixels wide, and reduce the frame rate from 30 to 15 frames per second). Of course, this results in a much lower-quality video, which is a compromise that many users do not wish to make; typically, users wish to preserve (completely, or as much as possible) the original quality of their video.
Typically, an important criterion for sharing is to do it quickly. For example, someone might be at a child's musical performance and wish to share a video with friends or relatives, within minutes of the performance. By contrast, an important criterion for archiving is preserving the quality of the original video.
Reconciling these two criteria, sharing the video quickly and preserving the original quality of the video, is a key focus of the concepts that we describe here.
In general, in an aspect, two or more versions of at least portions of a video that has been captured at a mobile device are uploaded from the mobile device to a server. Each of the two or more versions requires a different amount of communication resources.
Implementations may include one or more of the following features. The order of the uploading of the respective versions is determined based on a size of the version. The two or more versions include two or more degrees of compression. One of the versions has not been compressed and at least one of the versions has been compressed. The order of uploading corresponds to relative degrees of compression of the two or more versions. One or more of the two or more versions are to be stored at the server for delivery to other devices. The uploading includes uploading a highly-compressed, low-quality version of the video first, and subsequently uploading one or more higher-quality larger versions of the video. The uploading includes uploading a higher-quality version of the video to replace a lower-quality version. The uploading includes uploading a higher-quality version of the video to be maintained in addition to a lower-quality version. The versions of the video are uploaded from a mobile application running on the mobile device. Information about videos that have been captured on the device is displayed to a user of the mobile device. A user may configure the uploading. A user of the mobile device can select one or more recipients with whom to share the video. Uploading is controlled based on uploading conditions. Uploading is deferred for files larger than a predetermined size until a local wireless connection having a relatively large bandwidth is available. Uploading is deferred for files larger than a predetermined size until a charge level of a battery of the device exceeds a predetermined threshold. Uploading is deferred for files larger than a predetermined size until a charge level of the battery of the device exceeds a predetermined threshold and a Wi-Fi connection is available. A user of the device is notified that the uploading is pending. A number of videos remaining to be uploaded or a number of bytes of files to be uploaded or both is indicated to a user of the mobile device. A progress of uploading of each of two or more different versions of a single video is indicated to a user of the mobile device. A user of the mobile device is enabled to control an order in which the two or more versions are uploaded or to delete a pending upload or both. The uploading includes interleaving the uploading of two or more versions of each of two or more videos so that lower-quality versions of two or more of the videos are uploaded before higher-quality versions of any of the videos are uploaded. A user of the mobile device is enabled to control how or when or both the videos are uploaded. A user is enabled to set a battery threshold below which uploading a video should be suspended. The state of conditions that relate to the favorability of uploading is displayed to the user. The conditions are monitored. The conditions include one or more of a power state or availability of a cellular connection to upload additional data within a subscription.
In general, in an aspect, at a server at a location that is accessible from mobile devices through a public network, two or more successively uploaded versions of at least portions of videos that have been captured at the mobile devices are received. At least one of the later uploaded versions is of a higher-quality than one of the previously uploaded versions for a given one of the videos.
Implementations may include one or more of the following features. One or more of the versions of a given video are delivered to one or more devices other than the one at which the video was captured. The delivering to the one or more other devices occurs almost immediately after the video has been captured. Lower quality versions of a given video are deleted from the server after higher-quality versions have been uploaded to the server. The delivering includes streaming. The delivering includes downloading of the file, stored on the recipient device for later playback.
In general, in an aspect, at a server that is accessible from mobile devices and other devices, managing (a) two or more versions of respective videos that are captured on mobile devices are uploaded, (b) two or more of the versions are stored, and (c) one or more of the versions of a given video are delivered to one or more of the mobile devices or other devices or other devices.
Implementations include one or more of the following features. Ownership of videos among users of the devices is tracked. Identities of users who are authorized to view the respective videos are tracked. Notifications are sent to users who are potential recipients of videos to indicate that the videos have been shared by the owners with those potential recipients. The managing includes causing the deletion of one or more versions of a given one of the stored videos. The managing includes posting a given video on a social network site. The managing includes sending an e-mail or text message containing a link to a given one of the videos to an authorized recipient of the video. The managing includes responding to a request from one of the devices for access to one of the videos by arranging for the device to have access from the storage. The managing includes delivering a lower quality version of a given video until a higher-quality version is available in storage.
In general, in an aspect, a mobile device includes an application to upload two or more versions of at least portions of a video that has been captured at the mobile device through a public communication network to a server for storage, the respective versions being uploaded in order determined based on a size of the version and an urgency of uploading.
Implementations may include one or more of the following features. The mobile device includes a video capture device. The application is to form one or more of the versions by compressing a version of the video. The application is to enable a user to control the uploading. The application is to indicate to the user a status of the uploading. The monitoring element is to monitor conditions of the mobile device associated with it capability to upload. The conditions include a power state and a usage level of bandwidth of a communication subscription.
These and other aspects, features, implementations, and advantages can be expressed as methods, business methods, apparatus, systems, components, means or steps for performing functions, program products, and in other ways.
These and other aspects, features, and implementations, and advantages will become apparent from the following description, and from the claims.
Here we describe a way to enable rapid sharing of a video and also allow the archiving of a high-fidelity version of that video.
Suppose a large video file resides on a mobile device and the device owner wishes to upload the video to a remote storage system in order to archive and share it.
A key insight in what we describe here is to first upload from the mobile device a highly-compressed, low-quality version of the video, to allow for quick sharing. Subsequently, we upload one or more higher-quality, larger versions of that same video. Once uploaded, the one or more higher-quality versions will replace, or be maintained in addition to, the lower-quality one.
As shown in
We now present one example of how such a system can behave.
A user shoots a video using her mobile device 112.
Then, the user launches the mobile app. As shown in
Some of the videos shown in this display may reside on the device itself. Others may reside on a remote server. The application may visually distinguish those that reside on the remote server with an icon 210. The user may also create a new video 212 from within the app.
The user selects one or more of these videos. Having selected a video, the user selects an action from a menu (302 in
For each video requiring upload from the device, the app will compress the video to a small fraction (say 1/16) of its original size, and queue the video for upload 114 to the Cloud Storage Server (CSS) 116. File upload may be using a variety of protocols, including FTP, HTTP, and proprietary protocols.
Once the upload of this video is complete, the app will then compress the original video somewhat less aggressively (say to ⅛ its original size) and queue that (higher-quality) video for upload. As shown in
Once each version of the video is uploaded, the Control Plane may delete the previous (lower-quality) video on the CSS, or allow the multiple encodings to remain side-by-side. In either case, over time, the highest-quality version of the video stored on the CSS improves.
As soon as the first (lowest-quality) video has been uploaded, the video is then available for sharing with and viewing by friends and family. The Control Plane may (on the user's behalf) post the video to Facebook, send it to Twitter, and/or send an email or text containing a link to the video. Using one or more of these services, a recipient can then access the video on the CSS. The recipient can access the video from various endpoints: a browser-based PC/laptop, a smartphone/tablet, or a set-top box with appropriate software. Other implementations may also be possible for uploading, storing, and giving access to others.
The Control Plane may notify the recipient of the shared video in many ways. One way is to send an email (
Another approach is to send a text message to the recipient's mobile phone. Another approach, if the recipient has the mobile application installed on their device, is to send a notification directly to the application itself - notifications can be implemented using such tools and protocols as Apple Push Notifications (APN), Android C2DM, and Urban Airship. There may also be other approaches for notifying the recipient of a newly-available video.
We now walk through the complete sequence of steps from the capturing to the consumption of a personal video using the product described in this invention.
If the recipient accesses the video right away, she will see the lowest-quality version of the video. If the recipient doesn't access the video until later, she may at that time see a higher-quality version of the video, because some higher-quality version(s) will have in the meantime been uploaded to the CSS. When the recipient begins playing a lower-quality video, she may (before the video starts playing) see a message: “you may watch the video now, but be aware that a higher-quality version is being uploaded . . . come back later to watch it again in higher quality!” This message may be superimposed on the video, or may precede or follow it.
Assuming the CSS has stored multiple versions of the uploaded video in various qualities, the CSS can detect the quality of the network link between it and the video consumer, and select for delivery the highest quality version of the video that can reliably be delivered over that link. The selection could be dynamic, adapting every 10 seconds based on network quality (one such technique is called HTTP Live Streaming or “HLS”).
The mobile app may elect to defer uploads of higher-quality (e.g., very large) encodings (version) until conditions are favorable. For example, one policy might be:
If a video upload is deferred, the app may notify the user that there are still video(s) pending upload. The app may also explain to the user how to resume the uploads. E.g., a message: “You have 3 remaining videos to upload. To upload the files, be sure you're in a WiFi hotspot and charging.”
The client app may present to the user (
The app may show a separate entry in the queue for each encoding of a video, e.g., 5 separate entries for a single video, each at different quality. Alternatively, the app may show a single entry in the queue for each video.
The app may allow the user to pause the uploads 802, or to reorder the uploads and to delete a pending upload.
If more than one video is queued for upload, the system may interleave the uploading order of the various encodings as follows:
More generally, if two video encodings are queued for upload, the one with the lower-quality encoding has a higher priority and is uploaded first.
This approach ensures that if the user takes multiple videos, a version of each video shows up quickly on the Cloud storage system.
If the user takes a new video (say Video 3), while some existing videos are already queued for upload, the lowest-quality encoding of Video 3 will be prioritized before the higher-quality encodings of Videos 1 and 2. This provides “immediate gratification” to the user, i.e., their newly-taken video is uploaded quickly to the CSS, while the higher-quality encodings of Video 1, 2, and 3 are still queued for upload.
There are many ways to compress a video and this invention does not presume a specific implementation. Purely as examples (but not meant to be restrictive), compression may include reducing the frame rate (e.g. 30 fps down to 15 fps), reducing the pixel dimensions of the video (e.g. 1080 pixels wide down to 720 pixels), and reducing the encoding bitrate of the accompanying audio file (e.g. from 128 kbs to 64 kbs).
The system may allow (see
More generally, the app may present to the user a diagnostic screen (
A wide variety of other implementations are possible. For example:
The app can start at a different video encoding, e.g., 1/32 size or even lower-quality.
The app can terminate at a different video encoding, e.g., ½ the original size.
The steps do not need to be powers of two.
The user can configure the various compression levels of the videos.
The user can configure when to send the notification to the recipient (after the initial encoding has been uploaded, or after some higher-quality version has been uploaded, or both).
This application claims priority to U.S. Provisional Application Ser. No. 61/675,128, filed Jul. 24, 2012, the contents of which are incorporated here by reference.
| Number | Date | Country | |
|---|---|---|---|
| 61675128 | Jul 2012 | US |
