Patent Application
20070024705
This invention relates generally to video streaming, and more particularly to the delivery of video streams.
Current video streaming technology is primarily based on the design point of delivering fixed resolution and/or fixed-rate video streams for consumption by client software. In practice, this is typically accomplished using a video source (usually a camera) providing a single video stream, a video access component (usually a video stream server) providing a single video stream to a client, and client viewing software that operates on a Personal Computer (PC) with an intervening network that is used to transfer the video stream(s) and the associated control connections. Using such technology, it is possible that the video camera and stream server components may be one device.
In practice, a video source device and stream server provide a stream of a fixed resolution (e.g., 640 H×480V) at a predetermined frame and/or bit rate (e.g., 30 frames/second). This poses a significant set of problems for viewing client software due to the fact that fixed resolution and rate video sources, whether live or stored, typically do not match well with the bandwidth availability of the intervening transit network and, in some cases, the limitations of local computer resources (processing power, memory availability, etc.). In the case where the video source/s is higher resolution (greater than or equal to VGA quality), there is a low likelihood that the video stream/s may be satisfactorily viewed due to the fact that the bandwidth involved is in the 2.5 Mbps throughput range or greater. For example, a live or prerecorded video stream from a 1 Megapixel video source at 30 fps (full motion) is in the 4 Mbps range. This poses a significant problem from both a bandwidth and compute perspective since continuous bandwidth availability of most current transit networks is about 3 Mbps, and processing resources of a typical PC-based client viewing system is limited to about 640 H×480V at 30 fps while not performing other PC-related actions. This problem is exacerbated in environments where the viewer either needs or desires to view multiple video sources simultaneously, which is a common practice in the Surveillance/CCTV industry.
As an example, assume a Windows-based PC viewing client desires to simultaneously watch six camera sources across a network. Each camera source has a traditional resolution of 640 H×480V and produces a video stream at a frame rate of 30 frames/second (fps). Currently, this video stream would have a bitrate ranging from 2 Mbps to 20 Mbps due to the various video compression types. Assume for this example that a stream rate of 3 Mbps is chosen. For a PC to watch six camera sources via 3 Mbps streams consisting of 640 H×480V 30 fps video is roughly the equivalent of trying to play six conventional digital video disks (DVDs) simultaneously. Therefore, there is a significant compute burden, and Input/Output (I/O) processing burden, associated with each stream.
To enable the viewing client to simultaneously watch the six camera sources using conventional video streaming technology, it is possible to reduce the resolution (horizontal & vertical dimensions) of the video images, reduce the frame rate of the video stream, and/or to increase the compression factor used to compress the video stream into a lower bitrate. However, all of the above options diminish the observed video quality. Furthermore, increasing the compression factor does not diminish the compute burden associated with a video stream (i.e., it might alleviate network bandwidth issues but the compute issues are still present).
Compute problems are further exacerbated by the fact that the viewing space available on a typical conventional viewing client screen (monitor, LCD, etc.) does not change with respect to the characteristics of the incoming video stream, but with respect to the viewing operations being performed by the user. In short, the more cameras/scenes simultaneously viewed by a client, the smaller the dimensions of the viewing ‘window’ for that scene. For example, assuming that there is a 1024 H×768V viewing space at the client, six equally-sized simultaneous views would each occupy an individual window space of 170 H×128V viewing. Similarly, four equally-sized views would each occupy a 256 H×192V window, and eight equally-sized views would each occupy a 128 H×96V window each. However, the resolution of such viewing windows on the client application do not match the native, or incoming, resolutions from each common camera/video source. This resolution mismatch between source and viewing client requires client applications to scale incoming video streams into the desired viewing window, many times at undesirable scaling factors, which consumes more compute and memory bandwidth, and produces video quality issues that are the resultant side-effects from scaling.
Problems become more complex when the camera/video source is factored into this scenario. To provide better bandwidth and compute management, many users configure their conventional cameras/video sources to generate video in one of two basic categories: A) better resolution at lower frame rates (e.g. 640 H×480V @ 5 fps), or B) lower resolutions at higher frame rates (e.g. 320 H×240V @ 15 fps). These categories represent the trade-offs forced upon the user trying to obtain ‘useable’ video from multiple simultaneous sources that have fixed video stream characteristics.
In the past, a separate co-processor has been employed to enable viewing of a single high bandwidth high resolution stream, however, this implementation requires additional client processing hardware expense.
Systems and methods for selecting reception of video streams in an adaptive and, in one embodiment, dynamic fashion, for example, from a multi-stream video source. In one embodiment, reception of the video streams may be dynamically switched such that optimal bandwidth is selected in adaptive fashion using a set of video parameters, such as the size or dimensions of the viewing window, and environmental related parameters, such as bandwidth and processing resource usage, to ascertain the optimal stream selection. In another embodiment, a video stream of an image may be selected for viewing that is adapted to the needs of a user while at the same time maximizing efficiency of system resource usage, e.g., by adaptively selecting a video stream that meets the minimum resolution required by a user for a given viewing situation (and no more) to increase response time, reduce bandwidth requirements, and to reduce scaling artifacts.
The disclosed systems and methods may be beneficially implemented for surveillance applications or, for example, for other types of video viewing applications such as in situations where multiple video sources (e.g., video cameras) are viewed simultaneously or in situations where a user is allowed to dynamically resize a viewing window on a display device. In this regard, the disclosed systems and methods may be implemented to enable optimized simultaneous viewing of multiple video sources for each individual viewing client. This is in contrast to conventional video viewing systems in which the cumulative effect of viewing multiple scenes simultaneously produces an inordinate bandwidth and compute burden for the viewing client and the connected network, especially as the resolution of a camera source is increased. In such conventional systems, the video source is fixed (i.e., the frame rate and resolution cannot be modified), and a viewing client is incapable of adapting the video source to its environmental constraints. In this regard, the adaptation of a video stream of fixed attributes into an arbitrary viewing space (window) is a scenario that does not provide the proper balance between computer and network resources versus viewing quality and operation. Furthermore, standard single-stream camera sources, such as those employed in the Surveillance industry, are designed such that a configuration change for any of the above parameters affects all viewers irrespective of client viewing capabilities or network capacity (i.e., the behavior is static at the source).
In the practice of the disclosed systems and methods, a video delivery system may be provided that includes one or more video source components in combination with one or more client viewing applications. In such an embodiment, a video source component may be configured to produce video streams of multiple different combinations of rates and resolutions (e.g., two or more different combinations of rates and resolutions, three or more different combinations of rates and resolutions, etc.), and a client viewing application may be configured to understand the multi-stream capabilities of the aforementioned video source component. A client viewing application may be further configured in one embodiment to analyze its own viewing operations and to dynamically select the optimal video stream type/rate based on the results of the analysis. Such an analysis by the viewing client may be based on one or more stream selection parameters including, but not limited to, attributes (e.g., bitrate, frame rate, resolution, etc.) of video streams available from a video source, local viewing window resolution for the associated video stream, the number of input video streams in combination with the number of active views, computer resource status (e.g., memory availability, compute load, etc.), network bandwidth load, resource status of the video source, one or more configured policies regarding viewing operations, combinations thereof, etc.
In one embodiment, disclosed herein is an interactive video delivery system that includes a video source and/or video source and/or coupled video access component that provides multiple (greater than one) video streams of a given scene, and an intelligent viewing client that analyzes viewing operations and/or viewing modes and dynamically selects the optimal video stream/s provided by the video source in a manner that provides optimized (e.g., optimal) bandwidth and compute utilization while maintaining the appropriate video fidelity. In such an embodiment the video source, and/or a video access component coupled thereto, may be configured to advertise (e.g., using either standard or proprietary methods) information concerning the rates and resolutions of the available video streams related to a given scene/source (camera, etc.). The viewing client may be configured to select an optimized stream rate/s (e.g., optimal stream rate/s) for viewing the video data based at least in part on the information advertised by the video source and/or video access component. The viewing client may also be configured to perform this selection based further in part on one or more viewing operations selected by the user and/or by configuration. In another embodiment, a viewing client may also be configured to select an optimized stream frame rate and/or resolution by performing an analysis in which it selects the optimal stream rate/s and/or resolutions in an adaptive fashion (i.e., adapted to current video delivery operating conditions and/or currently specified video modes) for viewing the video data. This adaptive selection process may advantageously be performed in a dynamic, real-time manner.
In one respect, disclosed herein is a video selection system, including at least one viewing client configured to be coupled to receive at least two video streams over at least one network connection, and in which the viewing client is configured to select at least one of the video streams based at least in part on at least one video stream selection parameter.
In another respect, disclosed herein is a video selection system including a viewing client configured to be coupled to receive at least two video streams over at least one network connection, and in which the viewing client is configured to adaptively select at least one of the video streams based at least in part on at least one of current video delivery operating conditions, current specified viewing display modes, or a combination thereof.
In another respect, disclosed herein is a method of selecting video streams, including selecting at least one video stream from at least two video streams available over at least one network connection, in which the at least one video stream is selected from the at least two video streams based at least in part on at least one video stream selection parameter.
In another respect, disclosed herein is a method of selecting video streams, including adaptively selecting at least one video stream from at least two video streams available over at least one network connection, in which the at least one video stream is selected from the at least two video streams based at least in part on at least one of current video delivery operating conditions, current specified viewing display modes, or a combination thereof.
In another respect, disclosed herein is a video delivery system, including: one or more video source components, the one or more video source components being configured to provide at least two video streams; at least one video display component; and a viewing client coupled to receive the at least two video streams over at least one network connection, the viewing client being coupled to the video display component. The viewing client may be configured to select at least one of the at least two video streams based at least in part on at least one video stream selection parameter, and to provide the at least one selected video stream to the video display component for display.
In another respect, disclosed herein is a video delivery system, including: at least one video access component coupled to receive at least one video stream from at least one video source component; at least one video display component; and a viewing client coupled to receive at least two video streams over at least one network connection, at least one of the at least two video streams received by the viewing client being received from the at least one video access component, and the viewing client being coupled to the video display component. The viewing client may be configured to select at least one of the at least two video streams received by the viewing client based at least in part on at least one video stream selection parameter, and to provide the at least one selected video stream to the video display component for display.
In another respect, disclosed herein is a method of delivering video streams, including: providing at least two video streams over at least one network connection; selecting at least one of the at least two video streams based at least in part on at least one video stream selection parameter; and displaying the at least one selected video stream.
As shown in
In the exemplary embodiment of
A video access component may be configured, for example, to perform advertisement of stream attributes, to perform session management tasks, and to implement video stream protocols. In this regard, examples of video access components include, for example, devices that take analog input signals and convert them to digital formats and which may also encode signals using any suitable format/protocol (e.g., known video compression format/protocol), as well as devices of any configuration that are capable of converting/transcoding (e.g., frame rate adaptation and/or scaling) or forwarding video streams.
It will be understood that a video access component need not be present between a given video source/s and a viewing client, i.e., one or more video streams may be provided from a video source to a viewing client over one or more network connections in any alternative suitable manner. Therefore, for purposes of this disclosure, a video stream/s may be considered to be provided from a video source and received by a viewing client from the video source over one or more network connections whether or not the video stream/s is transferred from the video source/s to the viewing client through a video access component. Furthermore, the session management functions of a video access component may be logically implemented in any suitable configuration, whether it is as a stand alone device or system, integrated component of another device or system, or implemented by more than one device or system.
Still referring to
In the embodiment of
In an alternatively embodiment, multiple separate video source components 102a through 102n may be each coupled to deliver one or more digital video streams to video access component 206 via a computer network (not shown). In such an alternative embodiment, video source 102b may be a DVR that is configured to record and playback digital video data received from one or more other video sources 102 through such a computer network that links video source components 102a through 102n to video access component 206.
As shown in
In the embodiment of
Video access component 206 is configured to receive the input video streams on network medium 305, scale and/or transcode these streams into various rate and resolution video streams, and, is in turn coupled to communicate these multiple digital video streams (not shown separately in
It will be understood that a video access component may be optionally configured in one embodiment to receive at least one first video stream, to decompose (e.g., decode) the first video stream, and to perform scaling and/or rate adaptation tasks on the first video stream in order to provide at least one second video stream that is based on the first received video stream. In such an embodiment, the first video stream may have a first combination of resolution and frame rate, the second video stream may have a second combination of resolution and frame rate, and the first combination of resolution and frame rate may be different than the second combination of resolution and frame rate (i.e., the resolution of the first combination is different then the resolution of the second combination, the frame rate of the first combination is different than the frame rate of the second combination, or both). Therefore, it is possible in one exemplary embodiment that that a single video access component may provide to a viewing client at least two different video streams that are based on a single video stream provided by a single video source to the video access component. Alternatively a single video access component may provide to a viewing client a single video stream that is based on a single video stream provided by a single video source to the video access component. Such a single video stream may be provided to a network with other video streams, e.g., provided by other video source/s and/or video access component/s. In one embodiment, a given video access component may advertise stream attributes of video streams provided by other video access comments to the same network, e.g., in a situation where different video streams of the same scene/image are provided by different video access components.
In each of the embodiments of FIGS. 1 to 3, client viewing application 122 may be configured to select the identity of at least one received video stream 110 for display based at least in part on one or more stream selection parameters. In one embodiment, a stream selection parameter may be a dynamic parameter (i.e., a parameter subject to change during system operations), and client viewing application 122 may adapt to changing system operating conditions by monitoring one or more of such dynamic stream selection parameters that reflect these changing conditions. Such a dynamic parameter may be based, for example, on one or more characteristics of an available video stream/s 110, based on one or more characteristics of a given viewing system hardware and/or software configuration (e.g., video display component 140 usage, processor or memory usage of viewing client 120, user operations on video client 120, etc.), based on requirements of a particular viewing application, etc. Specific examples of dynamic stream selection parameters include, but are not limited to, attributes (e.g., bitrate, frame rate, resolution, etc.) of video stream/s 110 currently available from a video source/s, available current local viewing window resolution of video display component 140 for a given associated video stream 110, the current number of input video streams 110 in combination with the current number of active views on display component 140, current resource status (e.g., memory availability, compute load, etc.) of viewing client 120, current bandwidth load of network 112, current resource status (e.g., compute load, memory availability, concurrent number of active video sessions/streams, etc.) of the video source/s 102, etc.
A stream selection parameter may also be a static parameter such as a parameter based on one or more fixed characteristics (e.g., video display component 140 capability, processor or memory capabilities of viewing client 120, etc.) of a given viewing system hardware and/or software configuration, or a user-specified or pre-programmed default policy parameter, etc. Specific examples of static stream selection parameters include, but are not limited to, maximum local viewing window resolution of video display component 140, maximum resource capability (e.g., total memory, total compute capability, etc.) of viewing client 120, maximum bandwidth capability of network 112, maximum resource capability of the video source/s 102, one or more configured policies, maximum number of active video streams allowed at video client 120, maximum bandwidth allowed to be processed by video client 120, etc.
In one exemplary embodiment a static stream selection parameter may be a configured or pre-programmed static stream selection policy that acts to constrain one or more operating characteristics of a video delivery system. One example type of static stream selection policy is a policy that specifies maximum allowable total video stream bandwidth (i.e., total bandwidth of all selected video streams) to be delivered over network 112 to a viewing client 120 at any given time. Another example type of static stream selection policy is a policy that specifies maximum allowable processor (compute) resource usage of viewing client 120 for a given combination of selected video streams displayed on a video display component 140. For example, a stream selection policy may specify a maximum allowable processor usage of about 50% for a four window Standard Interchange Format (SIF)-15 display (e.g., four 352 H by 240V pixel windows displayed at 15 frames per second) on video display component 140 as shown in
Another example type of static stream selection policy is a policy that specifies selected video stream resolutions for a given viewing mode, i.e., the given configuration of one or more video windows of given spatial resolution to be displayed on video display component 140. In this regard, a policy may specify that video stream resolution/s be selected to match specified spatial resolution/s of one or more display windows to be provided for display. For example, a static stream selection policy may specify that nine equally-sized windows always be displayed at SIF-15 (e.g., nine 352 H×240V rectangular pixel or 320 H×240V square pixel windows displayed at 15 frames per second) on video display component 140b in
In another example, a static stream selection policy may be implemented to help reduce video artifacts by specifying that client viewing application 122 always scale down a video stream (rather than scale up the video stream) to fit available window space on video display component 140. In this regard, given an available window area of 240 H×180V square pixels in combination with a video stream having a SIF of 320 H×240V square pixels (QSIF of 160 H×120V square pixels), a static stream selection policy may specify that client viewing application 122 always scale the video stream down to fit the available window area. In yet another example, a static stream selection policy may specify that client viewing application 122 always select lower video resolutions for relatively smaller-sized display windows in order to save bandwidth of network 112.
It will be understood that the preceding static stream selection policies are exemplary only, and that other policies and/or combinations of such policies may be implemented. For example, another type of stream selection policy may specify that the highest frame rate available video stream/s always be selected that may be displayed (regardless of resolution) without exceeding compute resources or network bandwidth capacity of the viewing client component. Such a policy may be desirable where fast frame rate is more important than resolution, e.g., such as in a casino surveillance operation where detection of quick movements is important. Alternatively, a stream selection policy may specify that the optimal or highest resolution available video stream/s always be selected that may be displayed (regardless of frame rate) without exceeding compute resource or network bandwidth capacity, e.g., in a situation where detection of fine details is more important than detecting quick movement. In another example, a static stream selection policy may specify that the lowest resolution available video stream/s always be selected or that the lowest frame rate available video stream/s is always selected, regardless of compute resource or network bandwidth capacity. Such policies may be desirable, for example, where preserving network bandwidth and/or computer resource capacity is most important.
In one embodiment of the practice of the disclosed systems and methods, stream selection parameters may be processed by client viewing application 122 in a manner that optimizes video quality relative to system operating efficiency, or vice-versa. In this regard, a stream selection policy may be implemented that specifies that video quality (e.g., resolution, frame rate, etc.) always be maximized at the expense of system operating efficiency (e.g., network bandwidth, compute resource usage, etc.). Alternatively, a stream selection policy may be implemented that specifies that system operating efficiency always be maximized at the expense of video quality. In yet other examples, a stream selection policy may trade-off or balance between video quality and system operating efficiency under particular conditions.
Following determination of video source/s identity and video steam capability in steps 404 and 406, CVAP 122 may determine internal viewing mode for display component 140 (i.e., based on the client viewing application's feature set and viewing capabilities) in step 408. Examples of internal viewing mode information include, but are not limited to, the types of screen layouts available for viewing, the decoding and screen rendering capabilities of the application and its hardware, the types of viewing functions supported by the client viewing application, video window attributes, the presence of video graphics hardware that offloads buffering and video scaling, operating system type/version information, available system memory, hardware display type and attributes (spatial resolution, aspect ratio, color resolution), etc. In this regard, internal viewing mode information may be obtained by CVAP 122, for example, by reading application specific configuration information from an operating system registry or from a file, by retrieving system policy information, regarding allowable functions and operation from a network attached server, etc.
Following steps 402 through 408, CVAP 122 may execute video stream selection and display logic 410, in this exemplary embodiment by implementing steps 412 through 416. As shown in
CVAP 122 may also obtain video selection parameters from sources other than video sources 122 in step 412. Such other video selection parameters include, but are not limited to, those parameters previously mentioned. For example, information concerning local viewing window resolution of video display component 140 for a given video stream 110 may be obtained by reading/querying parameters associated with the dimensions and aspect ratio of each individual viewing window. The number of active views being displayed on video display component 140 may be obtained, for example, by reading/querying screen layout/geometry parameters that indicate the number of, location of, and type of video windows per screen layout along with associated input stream parameters. Video display processor resource status (e.g., memory availability, compute load, etc.) of viewing client 120 may be obtained, for example, by querying operating system functions that provide CPU and memory utilization information or by using internal processing statistics. Bandwidth load of network 112 may be obtained, for example, by querying/reading network layer statistics or by analyzing data available in the video transport protocols that indicate latencies and data/packet loss or by analyzing I/O (interrupt, scheduling, and event) rates within the system. Resource status of video source/s 102 may be obtained, for example, by querying/reading statistics from video source/s 102 or from receiving periodic real-time status updates from video source/s 102.
In addition, one or more configured video selection policies may be obtained, for example, by reading configured policy information from a system registry or file, or by mapping specific screen layouts to specific policy parameters that govern video selection criteria. As will be described further herein, such video selection policies may be, for example, any user-specified or system default rule that may be employed in combination with one or more other video selection parameters to govern the selection of particular available video streams 110 for display on video display component 140.
Next, in step 414, CVAP 122 selects particular video stream/s from the available video streams determined in step 412, e.g., based on one or more stream selection parameters obtained in step 412. This selection process may be performed using any suitable analytical or computational logic (e.g., state machine logic, if-then-else logic, switch-case statement logic, real-time computation or analytical logic, lookup table logic, etc.). In step 416, CVAP 122 then displays the selected video stream/s on video display component 140 in accordance with internal viewing display modes determined in step 408. Video stream selection and display logic 410 may then continue by repeating steps 412 through 416 during the video delivery process, as indicated by arrow 418. As described elsewhere herein, CVAP 122 may analyze a variety of dynamic stream selection parameters (e.g., parameters related to system, network, and resource states), alone or in various combinations, to determine the optimal viewing stream selected for a given video display mode. It is also possible that configuration data regarding limits, modes, etc., may also be factored into any analysis performed. In one example dynamic adaptation to changing conditions may be achieved, e.g., for a given resolution of a single viewing mode, the frame rate may be changed upon detection of a change in computer resource load or network traffic. For example, the frame rate may be dropped as necessary to maintain a given resolution upon an increase in compute resource load or increase in network bandwidth load.
As described above, state machine logic is one type of logic that may be employed in the practice of video stream selection methodology according to the disclosed systems and methods. The use of state machine logic to define the logic flow for each viewing mode is not necessary, but may be implemented in a manner that is very efficient and flexible with respect to the ability to easily add per-state/substate logic in order to handle any additional parameter analysis (i.e., memory availability, network load, I/O rates, response times, etc.) that may be deemed necessary. In this regard, state machine logic may implemented in a manner that simplifies stream selection logic by forcing the selected active, incoming video stream type to be conditionally or directly associated with default window size of each specific viewing mode, e.g., as a static association performed within each viewing mode. Thus, any user operations resulting in a change in viewing modes dynamically triggers viewing stream re-analysis. However, in other embodiments, logic that counts the number of active display windows rather than analyzing states, or that simply analyzes compute resource loading, for example, may be alternatively employed.
In one embodiment, a state machine logic approach may be based on the current viewing mode in order to simplify the analysis and processing logic while providing flexibility for more static (pre-programmed, configuration driven) or more dynamic (complex parameter analysis) driven analysis modes. In one example of such an embodiment, each of the logic paths of the state machine may be configured to always attempt to display the video stream that most closely matches the geometric dimensions of the corresponding display window in order to reduce local compute loads and network bandwidth demands, while providing the highest-quality viewing experience by minimizing, or obviating, the need to scale a video stream into the target viewing window's display dimensions.
Table 1 illustrates exemplary client viewing modes that may be obtained from, for example, basic application configuration information and/or derived by analyzing the display capabilities of a system. As previously described, CVAP 122 may determine the client viewing modes in step 408 of
Table 2 illustrates exemplary stream selection parameters in the form of characteristics of video streams, e.g., such as may be available from video source/s 102 of
Following is a description of an exemplary state machine logic that may be employed using the information of Tables 1 and 2 to implement video selection methodology according to one exemplary embodiment of the disclosed systems and methods. As previously described, a CVAP 122 may determine client viewing modes listed in Table 1 from internal application-based parameters, configuration information, and/or any other suitable method. A CVAP 122 may also contact and connect with a video source device 102 over network 112 and, using either a well-known protocol (e.g., such as RTSP/SDP (RFCs 2326/2327) or H.245) or other suitable method, the CVAP 122 may discover the available stream types and stream selection parameters (in this case, available video stream characteristics) as listed in Table 2.
Using the following state machine logic, CVAP 122 may then dynamically select video stream/s for display based on a combination of current client viewing mode and determined stream selection parameters. For example, in this case CVAP 122 may dynamically select which video stream/s (i.e., of given SIF resolution and 5, 15 or 30 frame per second frame rate) for display based on current client viewing mode (i.e., Big Mode or single window viewing mode, 4-Way Grid or four window viewing mode, 9-Way Grid or nine window viewing mode, 16-Way Grid or sixteen window viewing mode, or 25-Way Grid or twenty-five window viewing mode in this example) in combination with stream a selection parameter of compute load (i.e., computer processor resource utilization) and/or the use of network-related statistics related to network resource utilization and data reception:
Whether in multi-window viewing mode or in single window viewing mode, the disclosed systems and methods may be advantageously implemented to dynamically select video stream/s for display based on a combination of current client viewing mode and determined stream selection parameters. For example, in single window viewing mode, video stream selection may be dynamically performed according to the disclosed system and methods upon occurrence of one or more re-sizings of the single viewing window by a user.
It will be understood that the term ‘video stream’ is used herein as a logical term. In this regard, a ‘video stream’ identifies one or more video images, transferred in a logical sequence, that share the same basic attribute, for example, attributes of frame resolution, frame rate, and bit rate. However, it will also be understood that images of a video stream may also share other types of attributes, e.g., a series of video images transferred over the same network connection (‘socket’), a series of video images associated with the same source device or file/track, a series of video images that all share the same timespan, a series of video images that are all associated with the same event or set of events, a series of video images that are all within the same specific timespan from the same video source, etc. In this regard, it is not necessary that there be a direct correlation between a specific network connection or session used to transfer video data and a particular video stream.
In the practice of the disclosed systems and methods, a video source may be configured to provide multiple video streams, and the ability to switch between these video streams in a real-time manner. In this regard, video stream ‘switching’ may be performed in any suitable manner. However, in one embodiment it may be desirable that the a video source be configured to reconfigure, reselect, setup, reconnect, and re-assign video streams in a manner such that there is no detectable disruption of video data at the CVAP. Following are three exemplary manners in which a video source may accomplish video ‘switching’, it being understood that any other suitable methods are also possible.
In one exemplary embodiment, a video source may supply individual video streams on corresponding respective different individual logical network connections (e.g., different TCP/UDP/IP ‘sockets’) that are negotiated between the video source and the CVAP. In one example, Unicast RTSP/RTP protocol may be employed for this purpose. A CVAP may implement a ‘Connect/Disconnect/Reconnect’ method to communicate with a video source to switch between video streams. In this exemplary embodiment, a network connection is equivalent to an individual video stream. A signaling/management/control protocol (e.g., such as RTSP/SDP (RFCs 2326/2327), SIP (RFC 2543), H.225/H.245, etc.) to manage these processes may be necessary in some cases.
In another exemplary embodiment, a single (possibly persistent in one embodiment) network connection (e.g., a single socket) may be enabled to dynamically transfer multiple logically separate video streams. In one example, an HTTP-like or tunneling protocol may employed for this purpose. A CVAP may signal the video source when to change the video stream within the single network connection, using a signaling/management/control protocol (e.g., such as HTTP URL management/URL aliasing, RTSP Interleaved mode, etc.) and the video stream may be changed within the data (packet transport with payload identifier) transferred within the network connection. In such an embodiment, no Connect/Disconnect/Reconnect activity is required.
In yet another exemplary embodiment, various video streams may be distributed across a network on multicast connections (e.g., using multiple multicast sockets) and a CVAP, on its own, may switch to the reception of the available multicast connection/s that supports a desired or selected video stream without any negotiation required with the video source/s. In one example a RTP Multicast protocol may be employed for this purpose.
While the invention may be adaptable to various modifications and alternative forms, specific embodiments have been shown by way of example and described herein. However, it should be understood that the invention is not intended to be limited to the particular forms disclosed. Rather, the invention is to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the invention as defined by the appended claims. Moreover, the different aspects of the disclosed systems and methods may be utilized in various combinations and/or independently. Thus the invention is not limited to only those combinations shown herein, but rather may include other combinations.
