Video distribution environments have typically required clean switching, i.e., the ability to switch from one video source to another without tearing or breakup in the video. Clean switching has been accomplished historically in dedicated video signals having specific formats, from analog Black and White, through analog color and digital formats, up to the current migration to Internet streaming formats, using corresponding dedicated hardware switches.
More recent conventional solutions for achieving clean switching in the Internet Protocol (IP) realm typically require the video receiver to accept two video sources concurrently and to determine the appropriate time for the switch. One major disadvantage of this approach is that double the bandwidth is required in the routing matrix because both video sources must be sent to the receiver for a period of time. Another approach involves sending a proprietary timing signal to the IP switch that matches the frame rate of the source signals being used. Although this approach avoids the double bandwidth problem described above, it has several drawbacks, including the requirement for proprietary hardware configured to receive the proprietary timing signal, the assumption that all video sources use the same video timing format, and the requirement that any sources from external systems delivered with significant latency be re-timed to the same timing source that the switch receives.
Thus, there is a need in the art for a switching solution enabling clean video switching using standard, commercial off-the-shelf (COTS) IP switching devices.
The following description contains specific information pertaining to implementations in the present disclosure. One skilled in the art will recognize that the present disclosure may be implemented in a manner different from that specifically discussed herein. The drawings in the present application and their accompanying detailed description are directed to merely exemplary implementations. Unless noted otherwise, like or corresponding elements among the figures may be indicated by like or corresponding reference numerals. Moreover, the drawings and illustrations in the present application are generally not to scale, and are not intended to correspond to actual relative dimensions.
As noted above, video distribution environments have typically required clean switching, i.e., the ability to switch from one video source to another without tearing or breakup in the video. As further noted above, clean switching has been accomplished historically in dedicated video signals having specific formats, from analog Black and White, through analog color and digital formats, up to the current migration to Internet streaming formats, using corresponding dedicated hardware switches.
The present application discloses systems and methods for providing clean video switching (hereinafter also referred to as “seamless video switching”) for video over Internet Protocol (IP). It is noted that the seamless video switching solution disclosed by the present application addresses seamless video switching in the IP domain, but does not guarantee clean audio switching. It is noted that there are existing methods describing clean audio switching that include buffering of the audio packets and a gentle cross fade between audio sources to minimize any distortion. As those methods are known in the art, they are not described here. Nevertheless, in various implementations, the seamless video switching solution disclosed by the present application may incorporate those known methods for clean audio switching.
One significant advantage of the present switching solution is that it enables seamless video switching using readily available commercial off-the-shelf (COTS) switching devices (hereinafter “COTS devices”). It is noted that, as defined in the present application, the feature “video” may refer to video content without audio or to video content accompanied by or including audio. Moreover, it is further noted that the switching solution disclosed in the present application may advantageously be implemented as automated systems and methods.
As used in the present application, the terms “automation.” “automated.” and “automating” refer to systems and processes that do not require the participation of a human administrator. Although in some implementations the performance of the switches and switching methods disclosed herein may be reviewed or even modified by a human system or device administrator, that human involvement is optional. Thus, the methods described in the present application may be performed under the control of hardware processing components of the disclosed systems.
By way of overview, clean switching of video signals can occur when the transition point happens in the Vertical Blanking Interval, i.e., a period of the video frame in which lines are reserved for vertical retrace on displays. This switching strategy is standard for substantially all video formats, including progressive and interlace formats, regardless of line count and frame rate. IP encapsulated video signals have their line counts referenced and their location defined in IP standards such as Society of Motion Picture and Television Engineers (SMPTE) 2110, the standard for professional media over managed IP networks. It is noted that the details of this IP standard are described in SMPTE ST 2110-20:2017 document, titled “Professional Media Over Managed IP Networks: Uncompressed Active Video Standard by Society of Motion Picture and Television Engineers.” dated Sep. 18, 2017, which is hereby incorporated fully by reference into the present application.
It is further noted that many COTS IP switches on the market today often include a software layer that can be programmed like any hardware processor enabled computing device. According to the novel and inventive seamless video switching solution disclosed by the present application this programmable logic allows the COTS IP switch to be configured to inspect the contents of a video packet, to make forwarding decisions based on those contents, and even to alter the video packet data in real-time.
For example, and as disclosed herein in some use cases, when such a programmed IP switch identifies the appropriate markers in a video stream for triggering switching from one video source to another, the switch may stop forwarding packets from the original source, and begin forwarding packets from the new source. It is noted that this approach will require the video receiver to begin listening for packets from the new source synchronously with the switch.
As an alternative example, and as also disclosed herein, in some use cases the programmed IP switch may stop forwarding packets from the original source, and begin forwarding packets from the new source, and may perform Network Address Translation (NAT) by altering the forwarded packets so that they retain the same stream identifiers the receiver was already receiving from the original source, e.g., typically multicast group number and User Datagram Protocol (UDP).
Thus, the seamless video switching solution described herein discloses actions that can be programmed into a COTS IP switch to detect a switch point in a video packet header of a video packet contained in a frame, and to seamlessly switch from one media source to another in response to a switch command at the switch point. It is noted that the present approach assumes that the two media sources between which switching is performed are time synchronized with one another, which is usually the case for affiliated sources controlled by the same media entity. It is further noted that the present switching solution may advantageously be utilized for all video line and frame rates.
It is noted that, in various use cases, first video stream 104 and second video stream 106 may carry the same, or different, content. By way merely example, in some use cases, first video stream 104 may carry pre-recorded programming content, while second video stream 106 may carry a live feed, or vice versa. Alternatively, video streams 104 and 106 may carry the same content, but maintenance requirements or resource reallocation may make it advantageous or desirable to switch from first media source 102a to second media source 102b as the source of that same content. Moreover, in various implementations, first and second video streams 104 and 106 may carry any type of broadcast or transmission over IP signals.
It is noted that although
With respect to memory 124 of switch 120, it is further noted that memory 124 may take the form of any computer-readable non-transitory storage medium. The expression “computer-readable non-transitory storage medium.” as defined in the present application, refers to any medium, excluding a carrier wave or other transitory signal that provides instructions to processing hardware 122 of switch 120. Thus, a computer-readable non-transitory storage medium may correspond to various types of media, such as volatile media and non-volatile media, for example. Volatile media may include dynamic memory, such as dynamic random access memory (dynamic RAM), while non-volatile memory may include optical, magnetic, or electrostatic storage devices. Common forms of computer-readable non-transitory storage media include, for example, optical discs, RAM, programmable read-only memory (PROM), erasable PROM (EPROM), and FLASH memory.
Processing hardware 122 may include one or more hardware processing units, such as one or more central processing units, for example. By way of definition, as used in the present application, the expression “central processing unit” (CPU) has its customary meaning in the art. That is to say, a CPU includes an Arithmetic Logic Unit (ALU) for carrying out the arithmetic and logical operations of switch 120, as well as a Control Unit (CU) for retrieving programs, such as seamless switching software code 126, from memory 124. Alternatively, or in addition, processing hardware 122 may include an application-specific integrated circuit (ASIC) configured specifically to execute the seamless switching method disclosed in greater detail below by reference to
Transceiver 128 of switch 120 may be implemented as any suitable wireless communication unit. For example, transceiver 128 may be implemented as a fourth generation (4G) wireless transceiver, or as a 5G wireless transceiver. In addition, or alternatively, transceiver 128 may be configured for communications using one or more of Wireless Fidelity (Wi-Fi), Worldwide Interoperability for Microwave Access (WiMAX). Bluetooth, Bluetooth low energy, ZigBee, radio-frequency identification (RFID), near-field communication (NFC), and 60 GHz wireless communications methods.
In various implementations, one or both of first media source 102a and second media source 102b may be a resource of a media production or broadcast facility. Furthermore it is noted that first video stream 104 provided by first media source 102a and second video stream 106 provided by second media source 102b may include video in a variety of formats including high-definition HD 720p or 1080p video, Quad HD 2K video, or ultra HD (UHD) 4K or 8K video in either of progressive format or interlaced format.
It is also noted that, that the content carried by first video stream 104 and second video stream 106 may include content of widely varying types. Examples of the types of content carried by first video stream 104 and second video stream 106 may include audio-video content having both audio and video components, or video unaccompanied by audio, as noted above. In addition, or alternatively, in some implementations, the type of content carried by first video stream 104 and second video stream 106 may be or include digital representations of persons, fictional characters, locations, objects, and identifiers such as brands and logos, for example, which populate a virtual reality (VR), augmented reality (AR), or mixed reality (MR) environment. Such content may depict virtual worlds that can be experienced by any number of users synchronously and persistently, while providing continuity of data such as personal identity, user history, entitlements, possessions, payments, and the like. In some implementations, the content carried by first video stream 104 and second video stream 106 may be a hybrid of traditional audio-video and fully immersive VR/AR/MR experiences, such as interactive video.
Switch 220 including processing hardware 222, transceiver 228, memory 224, and seamless switching software code 226 corresponds in general to switch 120 including processing hardware 122, transceiver 128, memory 124, and seamless switching software code 126, in
In addition, first media source 202a, first video stream 204, second media source 202b, second video stream 206, switch command source 215, switch command 218, and video receiver 216, in
In some implementations, one or more of first media source 202a, second media source 202b, switch command source 215, or video receiver 216 may correspond to one or more web servers accessible over communication network 210 in the form of a packet-switched network such as the Internet, for example. Alternatively, communication network 210 may be implemented as a wide area network (WAN), a local area network (LAN), or another type of private or limited distribution network. Furthermore, in some implementations, one or more of first media source 202a, second media source 202b, switch command source 215, or video receiver 216 may be implemented virtually, such as in a data center. For example, in some implementations, one or more of first media source 202a, second media source 202b, switch command source 215, or video receiver 216 may be implemented in software, or as a virtual machine.
In various implementations, one or both of first media source 202a and second media source 202b may be a resource of a media production or broadcast facility, a node of a content delivery network CDN, or a source of a remote live feed, to name a few examples. Video distribution environment 200, in
According to the exemplary implementation shown in
According to the exemplary implementation shown in
The functionality of switch 120/220, in
Referring to
As noted above, in various implementations, first media source 102a/202a may be a resource of a media production or broadcast facility, a node of a CDN, or a source of a remote live feed, to name a few examples. As further noted above, first video stream 104/204 may include video in a variety of formats including HD 720p or 1080p video, Quad HD 2K video, or UHD 4K or 8K video in either of progressive format or interlaced format. As also noted above the content carried by first video stream 104/204 and may include content of widely varying types. Examples of the types of content carried by first video stream 104/204 may include audio-video content having audio and video components, or video unaccompanied by audio, as noted above. In addition, or alternatively, in some implementations, the type of content carried by first video stream 104/204 may be or include digital representations of persons, fictional characters, locations, objects, and identifiers such as brands and logos, for example, which populate a VR, AR, or MR environment. Such content may depict virtual worlds that can be experienced by any number of users synchronously and persistently, while providing continuity of data such as personal identity, user history, entitlements, possessions, payments, and the like. In some implementations, the content carried by first video stream 104/204 may be a hybrid of traditional audio-video and fully immersive VR/AR/MR experiences, such as interactive video.
Referring to
It is noted that although flowchart 590 lists action 592 as following action 591, that representation is provided merely by way of example. In some implementations, switch 120/220 may forward some video packets included in first video stream 104/204 to video receiver 116/216 while receiving other video packets of first video stream 104/204 from first media source 102a/202a. Thus, in various implementations, action 592 may follow action 591, or may be performed in parallel with, i.e., contemporaneously with action 591.
Continuing to refer to
As noted above, like first media source 102a/202a, second media source 102b/202b may be a resource of a media production or broadcast facility, a node of a CDN, or a source of a remote live feed, to name a few examples. As further noted above, like first video stream 104/204, second video stream 106/206 may include video in a variety of formats including HD 720p or 1080p video. Quad HD 2K video, or UHD 4K or 8K video in either of progressive format or interlaced format. As also noted above the content carried by second video stream 106/206 and may include content of widely varying types. Examples of the types of content carried by second video stream 106/206 may include audio-video content having audio and video components, or video unaccompanied by audio, as noted above. In addition, or alternatively, in some implementations, the type of content carried by second video stream 106/206 may be or include digital representations of persons, fictional characters, locations, objects, and identifiers such as brands and logos, for example, which populate a VR, AR, or MR environment. Such content may depict virtual worlds that can be experienced by any number of users synchronously and persistently, while providing continuity of data such as personal identity, user history, entitlements, possessions, payments, and the like. In some implementations, the content carried by second video stream 106/206 may be a hybrid of traditional audio-video and fully immersive VR/AR/MR experiences, such as interactive video.
It is noted that although flowchart 590 lists action 593 as following actions 591 and 592, that representation is provided merely by way of example. In some implementations switch 120/220 may receive second video stream 106/206 from second media source 102b/202b before forwarding first video stream 104/204 to video receiver 116/216. Moreover, in some implementations, switch 120/220 may forward some video packets included in first video stream 104/204 to video receiver 116/216 while receiving other video packets of first video stream 104/204 from first media source 102a/202a and also receiving packets of second video stream 106/206 from second media source 102b/202b. Thus, in various implementations, action 593 may follow actions 591 and 592, may precede action 592, or may be performed in parallel with, i.e., contemporaneously with action 592 or actions 591 and 592.
Continuing to refer to
It is noted that although flowchart 590 lists action 594 as following action 593, that representation is provided merely by way of example. In various implementations, action 594 may precede action 593, may follow action 593, or may be performed in parallel with, i.e., contemporaneously with action 593.
Referring to
In some implementations in which switch point 341 is detected in video packet header 350/450, for example, switch point 341 may be at a predetermined line number, such as a predetermined one of SRD line numbers 476 in video packet header 350/450. In those implementations, for instance, switch point 341 may be at any one of lines zero through the maximum line number “N” (0-N) included by the video format of frame 330. It is noted that, in some use cases, it may be advantageous or desirable for switch point 341 to be within the Vertical Blanking Interval of the video frame, i.e., a period of the video frame in which lines are reserved for vertical retrace on displays. In the case of HDTV 720P format, for example, that interval includes the first twenty-six lines of the frame. Thus, in use cases in which the format of frame 330 is HDTV 720P, it may be advantageous or desirable for switch point 341 to be at one of SRD line numbers 0-25, such as at line nine (9), merely by way of example.
As noted above, in some implementations, video packet 340/440 may be an RTP packet and video packet header 350/450 may be an RTP header. In some of those implementations, switch 120/220 may be programmed or otherwise configured to use RTP marker bit 460 of the RTP packet as switch point 341. Thus, in some implementations in which switch point 341 is detected in video packet header 350/450, switch point 341 may be at RTP marker bit 460 of the RTP packet. Alternatively, in implementations in which switch point 341 is detected in video packet payload header 370/470 in the form of an RTP payload header, for example, switch 120/220 may be programmed or otherwise configured to detect switch point 341 at Field Identification bit 480 of the RTP packet. Detection of switch point 341 in action 595 may be performed by seamless switching software code 126/226 of switch 120/220, executed by processing hardware 122/222.
It is noted that although flowchart 590 lists action 595 as following actions 591, 592, 593, and 594, that representation is provided merely by way of example. In many or most implementations, action 595 may be performed in parallel, i.e., contemporaneously with action 592. Moreover, as noted above, in some implementations, action 593 may be performed in parallel with actions 591 and 592. Thus, in various implementations, action 595 may follow actions 591, 592, 593, and 594, may precede action 594, or may be performed in parallel with, i.e., contemporaneously with actions 592 and 593 or actions 591, 592, and 593.
Referring to
With respect to execution of actions 596 and 597, in some implementations, when switch 120/220 receives switch command 118/218, switch 120/220 may stop forwarding first video stream 104/204 from first media source 102a/202a to video receiver 116/216, and begin forwarding second video stream 106/206 from second media source 102b/202b to video receiver 116/216. It is noted that this approach requires video receiver 116/216 to begin listening for second video stream 104/204 from second media source 102b/202b synchronously with switch 120/220.
Alternatively, in some implementations switch 120/220 may stop forwarding first video stream 104/204 from first media source 102a/202a to video receiver 116/216, and begin forwarding second video stream 106/206 from second media source 102b/202b to video receiver 116/216 while performing Network Address Translation (NAT) by altering the video packets included in second video stream 106/206 so that they retain the same stream identifiers that video receiver 116/216 was already receiving from first media source 102a/202a, e.g., typically multicast group number and UDP. With respect to the method outlined by flowchart 590, it is emphasized that actions 591, 592, 593, 594, 595, 596, and 597 may be performed as an automated method from which human participation may be omitted.
Thus, as described above, the present application discloses systems and methods providing seamless video switching. As also described above, the switching solution disclosed herein advantageously enables seamless video switching using widely available commercial off-the-shelf (COTS) switching devices, and is suitable for use all video line and frame rates.
From the above description it is manifest that various techniques can be used for implementing the concepts described in the present application without departing from the scope of those concepts. Moreover, while the concepts have been described with specific reference to certain implementations, a person of ordinary skill in the art would recognize that changes can be made in form and detail without departing from the scope of those concepts. As such, the described implementations are to be considered in all respects as illustrative and not restrictive. It should also be understood that the present application is not limited to the particular implementations described herein, but many rearrangements, modifications, and substitutions are possible without departing from the scope of the present disclosure.