The present disclosure is generally related to video transmissions.
Transmission of video over a wireless data network may involve a process of encoding video data, grouping the video data into video packets, and packetizing the video packets into wireless transmission packets, before eventually transmitting the wireless transmission packets. At a receiver, the reverse process is performed to recover the video data. However, overhead requirements during video packetization and wireless packetization might often cause a significant power burden on the system during transmission and subsequent processing.
Many aspects of the present disclosure can be better understood with reference to the following drawings. The components in the drawings are not necessarily to scale, emphasis instead being placed upon clearly illustrating principles of the present disclosure. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the several views.
This disclosure pertains to various embodiments, including systems, methods, computer useable mediums, and circuitry programmed to efficiently group static regions within encoded video data that is being transmitted over a transmission medium, such as a wireless network. In an illustrative and non-limiting example, blocks or regions of a video frame may be static (e.g., may not be moving for a period of time) with respect to a previous block or frame. Accordingly, a video transmission system may take advantage of this behavior by sending an indication for each non-changing (static) region in the encoded video stream.
As an illustrative example, a user may establish a video conference, where the user is sharing his or her desktop screen with remote users. Accordingly, background parts of a video of the desktop screen may not be moving for extended periods of time from frame to frame. As discussed above, some video transmission systems take advantage of this behavior by sending a static indication for each non-changing (static) region in the screen.
However, if the encoded video stream is packetized for wireless transmission and the skip indications are sent in separate wireless packets, this can be a burden when there are many skip indications being sent within the encoded video stream. For example, sending the small packets of static indications over the air and then waiting for acknowledgements (ACK) from a receiver and subsequently processing the ACKs can be a power burden.
One way to reduce the power burden is to send a representation of the skip indications with a moving region coded data to increase the wireless transmission efficiency. In accordance with one embodiment, among others, of the present disclosure, the skip indication representation may be an indication of a number of skip indications that occur in a series before the moving region or block is reached in encoding the video data. As mentioned above, the skip indication representation may also be combined with the moving region data in its own wireless transmission packet.
Therefore, embodiments of the present disclosure provide for a set of static indications to be aggregated into a single static indication representation, and this static indication representation in turn may be aggregated with the next moving video block. This can greatly reduce the overhead of the ACK portion of a MAC (media access control) protocol using in wireless transmissions.
Referring to
The group of devices 101-106 that may interact with the data network 110 is not intended to be an exhaustive list of device that may interact with a data network, and a generic device shown as an interactive device 109 represents a generic device that includes the functionality in order to interactive with the data network 110 and/or the other devices within the data network 110. Any of the devices 101-106 may be viewed generically as being the interactive device 109 without departing from the scope and spirit of the disclosure. Each of the devices 101-106 and the interactive device 109 may be viewed as being nodes of the data network 110. Any of the devices described above within the
It is also noted that the data network 110 itself includes functionality to allow interfacing with other networks as well. For example, the data network 110 may include an Internet interface 121 that allows for interfacing to the Internet 122. This Internet interface 121 may be viewed as being a base station device for the data network 110 that allows any one of the interactive devices to access the Internet 122.
It is also noted that the data network 110 may also include functionality to allow interfacing with other networks besides the Internet 122. For example, the data network 110 may include a microwave tower interface 123 that allows for interfacing to a microwave tower 124 thereby allowing communication with one or more microwave networks. Similar to the Internet interface 121, the microwave tower interface 123 may be viewed as being a base station device for the data network 110 that allows any one of the interactive devices to access the one or more microwave networks via the microwave tower 124.
Similarly, the data network 110 may include a satellite earth station interface 125 that allows for interfacing to a satellite earth station 126 thereby allowing communication with one or more satellite networks. The satellite earth station interface 125 may be viewed as being a base station device for the data network 110 that allows any one of the interactive devices to access the one or more satellite networks via the satellite earth station interface 125.
This finite listing of various network types that may interface to the data network 110 is not intended to be exhaustive. For example, any other network 128 may communicatively couple to the data network 110 via an interface 127 that includes the functionality for any one of the interactive devices to access the other network 128.
In various embodiments, both the circuitry 410 and a MAC/baseband processor 420 are Institute of Electrical and Electronic Engineers (IEEE) 802.11a/b/g/n-compliant chipsets that may be implemented in a digital composite metal oxide semiconductor (CMOS) process. This implementation involves one of the lowest possible silicon cost approaches to wireless radio and enables new product price points via chipset integration. The circuitry 410 and a MAC/baseband processor 420 are also compliant within a proposed draft of the 802.11ad standard in some embodiments, as well as the WiGig Alliance standards with WiGig Display Extension (WDE) support in some embodiments.
It is noted that the compatibility of the 802.11n functionality is backward compatible and forward compatible with the previous and subsequent versions of 802.11. As an example, 802.11 ad is a draft proposal for a next generation of the 802.11 standard; the functionality of the proposed 802.11 ad standard will compatible with the proposed drafts of 802.11ad.
In certain embodiments, the MAC/baseband processor 420 involves a direct-conversion radio architecture that is shown in a functional block that is able to support conversion functionality 412 to transform a radio frequency (RF) signal received by the antenna to a baseband (BB) frequency for use in the MAC/baseband processor 420. If desired, this conversion functionality 412 may include radio frequency (RF) to baseband (BB) direct conversion functionality.
As mentioned above, the radio circuitry 410 may be implemented to support the RF to BB direct conversion functionality. The radio circuitry 410 may be viewed as being a direct-conversion CMOS integrated circuitry (IC). In one embodiment, the radio circuitry 410 may be located on a single integrated circuit chip. This radio combines with the MAC/baseband processor 420 to deliver a complete IEEE 802.11 solution. The MAC/baseband processor 420 may be viewed as being a Wi-Fi baseband processor that is operable to support combined wireless LAN baseband processor and MAC functionality.
The MAC/baseband processor 420 is operable to support static condenser functionality 422 in aggregating static indications with moving blocks in packets of a video stream according to embodiments of the present disclosure.
Interactive device 500 includes a radio circuitry 510 and a MAC/baseband processor 520; the MAC functionality may be viewed as being designed into the baseband processor. The radio circuitry 510 is operable to receive a wireless signal via an antenna. Similar to the embodiment of the
The MAC/baseband processor 520 integrates the IEEE 802.11 MAC, 10/100/1000 Ethernet MAC, etc. interfaces thereby enabling complete portable Internet communications connectivity. The IEEE 802.11 MAC may be implemented to include the new proposed draft of the IEEE 802.11 ad, WiGig Alliance standards with WiGig Display Extension (WDE) support, and the functionality of the disclosure described herein. Specifically, this modified and improved wireless MAC includes static condenser functionality 522 supporting aggregating static indications with moving blocks in packets of a video stream. The functionality 522 will provide improvements for wireless communication of the interactive device 500.
In certain embodiments, the MAC/baseband processor 520 is operable to support IEEE 802.11 MAC and baseband functions and interfaces to the radio circuitry 510 to provide data network connectivity supporting data rates from 1 Mbps to 7 Gbps. In addition, in certain embodiments, the MAC/baseband processor 520 is operable to support Ethernet functionality.
Additional components of the interactive device 500 include bus system interface, Tx FIFO buffers, Rx FIFO buffers, Ethernet MAC, 802.11 PHY, Ethernet PHY, Led interface, PCI configuration registers, SPROM registers, and Boot ROM/GPIO interface, as shown in the figure. The interactive device 500 may also be additionally coupled to bus interface, serial EEPROM, Boot ROM/GPIO, LEDs, Ethernet connection (e.g., 10/100/1000), as shown in the figure.
Referring now to
For the video coding layer for a macroblock, a picture is split into blocks of a fixed size or macroblocks. The first picture of a sequence or a random access point is typically Intra coded without using information other than that contained in the picture itself. Each sample of a block in an Intra frame is predicted using spatially neighboring samples of previously coded blocks.
In an H.264/AVC coded sequence, a video picture is coded as one or more slices, each containing an integral number of macroblocks, where each macroblock (MB) can be coded in one of a large number of modes. These modes include three classes of Intra coding modes and four modes that use Inter prediction. In addition, if a macroblock has motion characteristics that allow its motion to be effectively predicted from the motion of neighboring macroblocks, and it contains no non-zero quantized transform coefficients, then it is flagged as skipped. This mode is identified as the Skip mode and results in no further data being transmitted after the Skip indication.
Accordingly, at the top of
Referring now to the bottom of
Next at the middle of
Accordingly, whereas the scheme used at the top of the figure sends separate data frames for each of the static indicators S1, S2, S3, the scheme at the middle contains a single static indication S that identifies that the three macroblocks that precede the macroblock for the moving region are static regions. Accordingly, instead of 3 separate data frames being sent for the separate indicators, one consolidated data frame is transmitted. Correspondingly, one acknowledgment (ACK) is expected to be returned upon receipt of the three static indicators for the three macroblocks of the video stream.
Therefore, embodiments of the present disclosure provide for a set of static indications to be aggregated in a wireless packet. This can greatly reduce the overhead of the ACK portion of a MAC (media access control) protocol using in wireless transmissions.
As shown in the example of
Electronic device 700 also includes a primary or main memory 706, such as random access memory (RAM). Main memory 706 has stored therein control logic 728A (computer software), and data.
Electronic device 700 also includes one or more secondary storage devices 710. Secondary storage devices 710 include, for example, a hard disk drive 712 and/or a removable storage device or drive 714, as well as other types of storage devices, such as memory cards and memory sticks. For instance, electronic device 700 may include an industry standard interface, such a universal serial bus (USB) interface for interfacing with devices such as a memory stick. Removable storage drive 714 represents a floppy disk drive, a magnetic tape drive, a compact disk drive, an optical storage device, tape backup, etc. As shown in
Removable storage drive 714 interacts with a removable storage unit 716. Removable storage unit 716 includes a computer useable or readable storage medium 724 having stored therein computer software 728B (control logic) and/or data. Removable storage unit 716 represents a floppy disk, magnetic tape, compact disk, DVD, optical storage disk, or any other computer data storage device. Removable storage drive 714 reads from and/or writes to removable storage unit 716 in a well-known manner.
Electronic device 700 further includes a communication or network interface 718 (e.g., radio circuitry 400, 500). Communication interface 718 enables the electronic device 700 to communicate with remote devices. For example, communication interface 718 allows electronic device 700 to communicate over communication networks or mediums 722 (representing a form of a computer useable or readable medium), such as LANs, WANs, the Internet, etc. Network interface 718 may interface with remote sites or networks via wired or wireless connections.
Control logic 728C may be transmitted to and from electronic device 700 via the communication medium 722. Any apparatus or manufacture comprising a computer useable or readable medium having control logic (software) stored therein is referred to herein as a computer program product or program storage device. This includes, but is not limited to, electronic device 700, main memory 706, secondary storage devices 710, and removable storage unit 716. Such computer program products, having control logic stored therein that, when executed by one or more data processing devices, cause such data processing devices to operate as described herein, represent embodiments of the present disclosure.
Electronic device 700 may be implemented in association with a variety of types of display devices. For instance, electronic device 700 may be one of a variety of types of media devices, such as a stand-alone display (e.g., a television display such as flat panel display, etc.), a computer, a tablet, a smart phone, a game console, a set top box, a digital video recorder (DVR), a networking device (e.g., a router, a switch, etc.), a server, or other electronic device mentioned elsewhere herein, etc. Media content that is delivered in two-dimensional or three-dimensional form according to embodiments described herein may be stored locally or received from remote locations. For instance, such media content may be locally stored for playback (replay TV, DVR), may be stored in removable memory (e.g. DVDs, memory sticks, etc.), may be received on wireless and/or wired pathways through a network such as a home network, through Internet download streaming, through a cable network, a satellite network, and/or a fiber network, etc. For instance,
Video-image camera 740 may include an image sensor device and image processor and/or additional/alternative elements. The video-image camera 740 captures video images, and generates corresponding video data that is output on a video data signal. In an embodiment, the video data signal contains the video data that is output on an image processor output signal, including processed pixel data values that correspond to images captured by the image sensor device. The video data signal may include video data captured on a frame-by-frame basis or other basis. In an embodiment, the video data signal may include video data formatted as Bayer pattern data or in another image pattern data type known in the art.
In accordance with one embodiment, the video data signal may be transmitted using the communication interface 718. Accordingly, the communication interface 718 may support static condenser functionality to aggregate static indications with moving blocks in an encoded form of the video data signal in accordance with the present disclosure. Alternatively, in an embodiment, the transcoder 720 may support static condenser functionality to aggregate static indications with moving blocks in an encoded form of the video data signal before the video signal is relayed to the communication interface 718 for wireless or wired transmission. Accordingly, use of the static condenser functionality in accordance with the present disclosure may reduce the power burden on battery 770. Further, control logic 728A may be able to activate a mode in which the static condenser functionality is activated and an alternative mode in which the static condenser functionality is deactivated. Activation or deactivation of these modes may be selected by a user of the electronic device, in one embodiment.
In one embodiment, the series of consecutive static indications are grouped in the single wireless packet as individual static indications. In an alternative embodiment, the series of consecutive static indications are grouped in the single wireless packet as a single static indication that identifies the non-moving macroblocks that were represented by the consecutive static indications in the encoded video signal.
Next,
In an alternative embodiment, a transcoder 720 may group or condense the static indications before passing the encoded video signal to the communications interface 718. For example, referring now to
In
Certain embodiments of the present disclosure, including static condenser functionalities, can be implemented in hardware, software, firmware, or a combination thereof. Some embodiments are implemented in software or firmware that is stored in a memory and that is executed by a suitable instruction execution system. An alternative embodiment can be implemented with any or a combination of the following technologies, which are all well known in the art: a discrete logic circuit(s) having logic gates for implementing logic functions upon data signals, an application specific integrated circuit (ASIC) having appropriate combinational logic gates, a programmable gate array(s) (PGA), a field programmable gate array (FPGA), etc.
Any process descriptions or blocks in flow charts should be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process, and alternate implementations are included within the scope of the present disclosure in which functions may be executed out of order from that shown or discussed, including substantially concurrently or in reverse order, depending on the functionality involved, as would be understood by those reasonably skilled in the art.
A software program corresponding to the static condenser can comprise an ordered listing of executable instructions for implementing logical functions and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this document, a “computer-readable medium” can be any means that can contain, store, communicate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer readable medium can be, for example but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device medium. More specific examples (a nonexhaustive list) of the computer-readable medium would include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM) (electronic), a read-only memory (ROM) (electronic), an erasable programmable read-only memory (EPROM or Flash memory) (electronic), an optical fiber (optical), and a portable compact disc read-only memory (CDROM) (optical). In addition, the scope of the certain embodiments includes embodying the functionality of exemplary embodiments of the present disclosure in logic embodied in hardware or software-configured mediums.
It should be emphasized that the above-described embodiments of the present disclosure are merely possible examples of implementations, merely set forth for a clear understanding of the principles of the disclosure. Many variations and modifications may be made to the above-described embodiment(s) without departing substantially from the spirit and principles of the disclosure. All such modifications and variations are intended to be included herein within the scope of this disclosure and protected by the following claims.