The teachings of the present invention can be readily understood by considering the following detailed description in conjunction with the accompanying drawings, in which:
An exemplary embodiment of the invention will now be described while referring to the figures, several of which may be simultaneously referred to during the course of the following description.
Today's VoD streaming services typically use unicast communication channels as the transport mechanism and use RTSP (Real Time Streaming Protocol) as the control protocol. Thus, user interactivity features are limited to VCR-type operations supported by RTSP.
Given the typically small size of mobile phone screens, it may be difficult to display a large view with enough details for certain types of programming. In some scenarios, different viewers may prefer to view different portions of a video frame with greater detail. For example, when viewing a sports program, such as a baseball game, the camera at times captures the entire field. Some viewers, however, may want to see the facial expressions of their favorite player or players, if possible, instead of the view of the field. Each person's viewing preference is thus different, and the close up view offered by the camera may not satisfy every viewer's desire.
The present invention sets forth a novel interactive zooming feature for streaming video, where end users can issue commands in order to receive a close-up view of a portion of the current view.
Referring to
In order to operate the zooming feature of the present invention, a new suite of commands will be added to the RTSP protocol. The information in the commands sent from the end user to the video server would include the center of a new view expressed in (x,y) coordinates of the current view, and an indication of the resolution of the new view. It would be understood that other reference points besides the center coordinates and that other coordinate systems may also be used to express a reference upon which a new view will be based, such as for example a corner coordinate
The resolution indication can be expressed, for example, as +2, +4 for zooming in, and −2, −4 for zooming out. We call this number a “zoom factor”. Two versions of the zoom factor, “relative” and “absolute” will be explained, although like methodologies for expressing a desired resolution are contemplated within the scope of the invention. In accordance with the invention, an absolute zoom factor having a value of (1) specifies the default resolution. An absolute zoom factor value of (+2), for example, corresponds to a two times (2×) increase in the default resolution, whereas an absolute zoom factor value of (−2) corresponds to a one half (½×) times the default resolution. Another manner to express resolution in accordance with the invention is to use the notion of “relative zoom factor”, where a value (1) specifies the current resolution and other zoom are based off the current view. When the “relative zoom factor” is used, values that are transmitted in the RTSP protocol would need to be translated to the “absolute zoom factor” when received at the video server as would be understood by those persons skilled in the art.
In order for the video server to respond to the end user's zooming request, multiple versions of the video stream need to be stored at the server. Each version of the stream has a different resolution. When a zoom-in request is received, the video server identifies a version of the stream with higher resolution, and sends back a portion of the stream that covers the smaller requested area. When a zoom-out request is received, the video server needs to identify a version of the stream with lower resolution, and sends back a portion of the stream that covers the larger requested area.
Note that in certain embodiments of the invention, the video display client on the end device and the video server or proxy are synchronized in the sense that the client can only request zoom factors offered by the video server. In the following example, we assume the video server stores the following 5 versions of the stream: 4×resolution, 2× resolution, 1× resolution, (½)×resolution and (¼)× resolution. We also assume the client can only send requests with the following values for the zoom factor: +4, +1, −2, −4. In this case, there is a one-to-one mapping between the zoom factor and different versions of the stream. Each zoom request will be mapped to a particular version of the stream. A request with absolute zoom factor value of 4 will be served with the stream of 4× resolution, a request with absolute zoom factor value of −2 will be served with the stream of (½)×resolution, and so on. However, in the case where the client and the server are not synchronized, a zoom request may not match the available versions of the stream at the server. In order for users to view what they have requested using the zoom feature, we design the GUI on the end device such that the zoom factors available for each stream are clearly specified by the GUI. To satisfy this requirement, a mapping function is provided at both the client and the server to map the client requested zoom factor to the different versions of the stream with different resolution. Implementation of a mapping function as described would be understood by persons skilled in the art.
Another design option is to map different values of zoom factor to one version of the stream. Because we keep the size of the video display the same, this mechanism will, however, produce lower quality images for the zoom-in request, and produce higher quality image for the zoom-out request.
Because the zooming action in most cases still results in the same frame size with the same video quality, the bandwidth required for the stream does not change. In the following exemplary illustration of the present invention, an original video stream requires 100 Kbps of bandwidth, a high resolution version of the same content has 400 Kbps of bandwidth, for example. When a zoom-in command is issued on the original stream to cover only 1/4 of the area, the resulting high-resolution stream still requires 100 Kbps. Therefore, the bandwidth requirement between different zooming requests is not changed. As would be understood, the above statements regarding bandwidth are only an estimate. The exact bandwidth of different video streams vary depending on the image and the compression schemes. Accordingly, stated roughly, a stream with A times resolution will result in A times bandwidth requirement.
The present invention may also be utilized in connection with a client-proxy-server architecture. In such an environment, the zooming command may be handled by a proxy device 16 (shown in
The current RTSP protocol does not support a zooming command. The protocol can be easily updated with a new command, however, such as “Zoom”. An exemplary embodiment of the command would contain the following fields:
Zoom-in or Zoom-out
Zooming factor
(x,y) coordinate to center the new stream.
Note that if the (x,y) coordinate is near the edge of the original frame, then it is likely that the entire display window will not be filled, since there is no data beyond the boarder of the original frame regardless which version of the stream it is. Accordingly in one embodiment of the invention, the (x,y) coordinate would be automatically translated by the programming code associated with the zoom feature so as to enable display of a full screen.
Referring to
The new zooming feature of the present invention applies to video service on mobile phones. The feature offers more interaction between the end user and the video server and it provides more interesting viewing. It has the potential to become a service differentiator for video services provided by wireless service providers. Because the bandwidth required for the streams does not change, the only complexity incurred is the control message processing, and the only extra storage required is at the video server or the proxy.
Note that this zooming feature for VoD service not only applies to mobile end devices, but also applies to VoD service over the Internet for the wireline “triple play service”. Additionally, the zooming feature not only applies to VoD service for stored content, but also applies to live streams if multiple versions of the stream are generated at the content server.
The foregoing description merely illustrates the principles of the invention. It will thus be appreciated that those skilled in the art will be able to devise various arrangements, which, although not explicitly described or shown herein, embody the principles of the invention, and are included within its spirit and scope. Furthermore, all examples and conditional language recited are principally intended expressly to be only for instructive purposes to aid the reader in understanding the principles of the invention and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments of the invention, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
In the claims hereof any element expressed as a means for performing a specified function is intended to encompass any way of performing that function including, for example, a) a combination of circuit elements which performs that function or b) software in any form, including, therefore, firmware, microcode or the like, combined with appropriate circuitry for executing that software to perform the function. The invention as defined by such claims resides in the fact that the functionalities provided by the various recited means are combined and brought together in the manner which the claims call for. Applicant thus regards any means which can provide those functionalities as equivalent as those shown herein. Many other modifications and applications of the principles of the invention will be apparent to those skilled in the art and are contemplated by the teachings herein. Accordingly, the scope of the invention is limited only by the claims.