This invention relates to the field of digital media servers.
A digital media server is a computing device that streams digital media content onto a digital data transmission network. In the past, digital media servers have been designed using a general-purpose personal computer (PC) based architecture in which PCs provide all significant processing relating to wire packet generation. But digital media are, by their very nature, bandwidth intensive and time sensitive, a particularly difficult combination for PC-based architectures whose stored-computing techniques require repeated data copying. This repeated data copying creates bottlenecks that diminish overall system performance especially in high-bandwidth applications. And because digital media are time sensitive, any such compromise of server performance typically impacts directly on the end-user's experience when viewing the media.
Referring now to
In step 104, the application copies the data from RAM into central processing unit (CPU) registers. In step 105, the CPU performs the necessary data manipulations to convert the data from file format to wire format. In step 106, the wire-format data is copied back into application RAM from the CPU registers.
In step 107, the application submits the wire-format data to the OS for transmission on the network and the OS allocates a new memory location for storing the packet format data. In step 108, the OS writes packet-header information to the allocated packet memory from the CPU registers. In step 109, the OS copies the media data from the application RAM to the allocated packet RAM, thus completing the process of generating a wire packet. In step 110, the completed packet is transferred from the allocated packet RAM to OS RAM.
Finally, the OS sends the wire packet out to the network. In particular, in step 111, the OS reads the packet data from the OS RAM into CPU registers and, in step 112, computes a checksum for the packet. In step 113, the OS writes the checksum to OS RAM. In step 114, the OS writes network headers to the OS RAM. In step 115, the OS copies the wire packet from OS RAM to the network interface device over the shared I/O bus, using a DMA transfer. In step 116, the network interface sends the packet to the network.
As will be recognized, a general-purpose-PC architecture accomplishes the packet-generation flow illustrated in
As shown in
In a preferred embodiment, the present system and method comprise a hardware engine adapted to transfer media asset data from a media buffer to a network. The hardware engine receives media asset streaming instructions from a general-purpose PC via control blocks stored in the buffer along with the media asset data. The hardware engine eliminates the redundant copying of data and the shared I/O bus, bottlenecks typically found in a general-purpose PC that delivers digital media. By eliminating these bottlenecks, the hardware engine improves overall delivery performance and significantly reduces the cost and size associated with delivering digital media to a large number of end users.
In a preferred embodiment, the hardware engine comprises a programmable logic device (PLD) to provide significantly higher data processing speeds than a general-purpose CPU. Advantageously, such PLDs can be reprogrammed without replacing hardware components such as read-only memories. Consequently, the present system provides flexibility and future-proofing not usually found in a dedicated hardware device, while maintaining hardware-level wire-speed performance.
In addition to extending the life cycle of the hardware solution by providing the ability to incorporate additional functional components in the future, the hardware engine's wire-speed performance increases the number of unique streams that can be processed and delivered by the digital media server. This increase in stream density in a smaller physical package (compared to servers that use a general-purpose PC architecture) leads to improved scalability which can be measured by reduced space requirements and lower environmental costs, such as air conditioning and electricity. Because each server unit has a higher stream density than previous media server units, fewer servers are required, which directly relates to a smaller capital investment for deployment of streaming video services. Fewer servers also result in lower operating costs such as reducing the need for operations personnel to maintain and upgrade the servers.
In one aspect, the present invention is directed to a system under the control of a general-purpose computer for converting digital media assets into wire data packets for transmission to a client, the assets being stored on a digital media storage device comprising an input interface for retrieving digital media asset data from the storage device, a media buffer for receiving the digital media asset data from the storage interface, a programmable logic device adapted to transfer the digital media asset data from the input interface to the media buffer, process the digital media asset data from the media buffer, and generate wire data packets, a network interface coupled to the device and adapted to transmit the wire data packets to the client, and a general-purpose interface coupled to the device and adapted to receive control information from the general-purpose computer for storage in the media buffer and to enable the device to communicate with the general-purpose computer.
In another aspect of the present invention, the media buffer is further adapted to store control blocks comprising packet header formatting instructions and digital media asset payload information, and the programmable logic device is further adapted to generate packet headers from the instructions.
In another aspect of the present invention, the digital media asset payload information comprises a pointer to the digital media asset data.
In another aspect of the present invention, the digital media asset payload information comprises the digital media asset data.
In another aspect of the present invention, the programmable logic device is a field programmable gate array.
In another aspect of the present invention, the network interface comprises a Gigabit Ethernet interface.
In another aspect of the present invention, the data generation rate is greater than or equal to the data transmission rate, the programmable logic device data reception rate is greater than or equal to the data generation rate, and the media buffer data reception rate is greater than or equal to the programmable logic device data reception rate.
In another aspect of the present invention, two or more programmable logic devices cooperatively increase the data transmission rate of the system.
In another aspect of the present invention, the programmable logic device comprises an MPEG-2 stitching engine for targeted ad insertion.
In another aspect of the present invention, the programmable logic device is further adapted to encrypt the data stream thereby increasing the quality of content security.
In another aspect, the present invention is directed to a secure method of providing an upgrade package for changing the logic in a field programmable gate array used as an engine for streaming digital media, comprising encrypting the upgrade package, compressing the upgrade package, distributing the upgrade package, decompressing the upgrade package, loading the package into the field programmable gate array, supplying a key to the field programmable gate array for decrypting the upgrade package, and rebooting the field programmable gate array; thereby installing the upgrade package.
In another aspect, the present invention is directed to a method of streaming a block of a digital media asset across a digital network using a hardware engine, comprising transferring the block of the asset into a media buffer, writing wire packet generation control instructions into the media buffer, fragmenting the block into one or more data packets, generating packet headers for a packet in accordance with the instructions, calculating a checksum for the packet, transmitting the packet onto the network, and repeating the generating, calculating, and transmitting steps until all the data packets have been transmitted.
In another aspect of the present invention, the method further comprises the steps of receiving a message to process the instructions and sending a message that the block has been sent.
In another aspect, the present invention is directed to a method for designing a streaming media hardware engine, comprising: (a) identifying one or more components that comprise the hardware engine, (b) designing a last component having a fully saturated output bandwidth greater than or equal to the required bandwidth of the hardware engine (c) calculating the input bandwidth required to fully saturate the designed component, (d) designing an adjacent preceding component having a fully saturated output bandwidth greater than or equal to the input bandwidth calculated in step (c), and recursively repeating steps (c) and (d) for remaining components identified in step (a).
One preferred embodiment of a hardware engine for streaming digital media assets is shown in
As shown in
In more detail, interface objects 420-460 preferably comprise a storage peripheral component interface (PCI) interface 420, a media buffer interface 430, a gigabit Ethernet controller interface 440, a general-purpose PCI interface 450, and a security interface 460. Interface HDL objects 420-460 provide the signals required to send or receive data from the FPGA to components 320-360, respectively.
The series of FIFO queues preferably comprises five sets of FIFO queues 471-475. FIFO queue HDL objects 471-475 buffer the flow of data between the interface HDL objects and packet engine 480 in FPGA device 400.
Parser state machines 510 preferably comprise three components, a control block parser 519, a payload builder 517, and a facilitator 515. Control block parser 519 is adapted to read a control block stored in media buffer 330 and pass appropriate data from the control block to header formatting state machines 520. Under control of control block parser 519, payload builder 517 reads asset data from media buffer 330. Facilitator 515 is adapted to schedule the output from packet header formatters 520.
Packet header formatter state machines 520 preferably comprise state machines that produce packet headers which adhere to the communication protocols necessary for streaming video across an Internet Protocol data network including IP 521, UDP 522, TCP 523, RTP 524 and HTTP 525. Each packet header formatter is responsible for generating a packet header in the appropriate format for inclusion in the wire packet. The packet headers are preferably generated from control block data determined by control block parser 519.
Packet assembly state machines preferably comprise a multiplexer 531, a payload packer engine 532, a header packer 533, a checksum generator 534, and a packet writer 535. Multiplexer 531 multiplexes the output of the various header format state machines and the payload builder into packets. Payload packer engine 532 shifts and concatenates the data to eliminate empty bytes in the packet data stream. Packer 533 shifts and concatenates the packet headers to eliminate empty bytes in the packet data stream. Checksum generator 534 generates the checksum of the wire data packet. Packet writer 535 sends the wire data packet out to the gigabit Ethernet controller. It manages payload buffers included in gigabit Ethernet controller 440, inserts checksums into the packet data stream, and creates a data entry indicating that the asset has been sent.
In an alternative preferred embodiment, packet engine 480 may include additional packet generation and protocol engines that replace many of the algorithms traditionally executed on a general-purpose CPU. For example, packet engine 480 may comprise an MPEG-2 stitching engine for targeted ad insertion, or a unique stream-encryption engine for increasing the quality of content security.
Design Methodology for Hardware Engine
Each component in hardware engine 300 is designed specifically for the sustained delivery of digital media so that any given component will not restrict the flow of data and form a bottleneck in the device. Preferably, the criterion used to calculate how much input bandwidth is required for a component is determined from the full bandwidth saturation of the output interface of the component. By determining the amount of input bandwidth that will achieve a desired output bandwidth for a particular component, the output bandwidth of its upstream component can be selected so that the upstream component will supply at least the bandwidth required at the component's input to saturate its output.
This design principle is preferably applied to all components in hardware engine 300, including those that may have a higher input bandwidth than output bandwidth at full saturation. This situation may occur where some of the data supplied to a component is not transmitted by the component. Illustratively, a component that reads data storage blocks from a hard drive and processes the blocks into data packets may not use the entire contents of the block. The packet data required may be slightly larger than one block, requiring that two blocks be read into media buffer 330. Although two full blocks are read, only a small percentage of the second block is required for generating the packet. Thus, the output bandwidth for the component may be less than its input bandwidth.
This design process is illustrated in more detail in
Because the throughput of each component and bus are selected or designed to fully saturate the next component, bottlenecks within the device are eliminated and the device operates with fully saturated output connections.
Reprogramming the FPGA
In a preferred embodiment, upgrade packages may be used to reprogram the FPGA using the hardware description language (HDL). By replacing the FPGA's configuration, the HDL components included in the FPGA are changed. The process for installing an upgrade package is illustrated in
As shown in
In a preferred embodiment, security interface 560 protects the logic programmed into the FPGA from being copied. As known in the art, different security interfaces may be designed or purchased that provide varying degrees of security and implementation overhead. Those skilled in the art may balance competing desires to maximize security while minimizing implementation time and cost in selecting an appropriate security interface for the FPGA.
The flexibility achieved by reprogramming the hardware device is illustrated by the following example. Suppose that the initial hardware description language implemented in the FPGA includes packetization algorithms and protocols specific to MPEG-2 transport streams. In the future, users may require delivery of media content in other formats such as MPEG-4. Because hardware engine 300 comprises an FPGA, new algorithms for manipulating MPEG-4 formats can be added to the layout of the chip using HDL in the form of an upgrade package.
FSH Streaming Media Operation
In operation, hardware engine 300 assembles wire packets in accordance with instructions specified in a control block found in media buffer 330. In a preferred embodiment the control block is a 128-byte data structure comprising a series of control block entries (CBE) of at least eight bytes in length. Each CBE either contains data that will be part of a media packet, or a pointer to that data. The media packet can be constructed by traversing the entire control block and concatenating the data contained in each entry or data pointed at by each entry.
Engine 300 then converts the media packet payload from file format to wire format. If the media packet is larger than the maximum transmission unit (MTU), this conversion process preferably comprises fragmentation of the media packet into several wire format data packets (step 1040). In step 1050, engine 300 generates protocol format headers specified in the CBEs for insertion into the wire packet. Next, in step 1060, engine 300 assembles the packet and calculates a checksum for the wire packet. In step 1070, engine 300 sends a wire packet out thorough gigabit Ethernet interface 340. If the last wire packet has not been sent (step 1080), engine 300 updates packet headers and checksum and sends the next wire packet. After the last packet has been transmitted, engine 300 generates a message that indicates the control block has been processed.
A preferred header-formatting process is now described in more detail. In a preferred embodiment, engine 300 adds an Ethernet header to every packet unless the control block has a “pass thru” identifier. The Ethernet header control block contains a source address, destination address, and a packet type field. In a preferred embodiment, header information for the Ethernet header is included in a CBE, as shown, for example, in
In a preferred embodiment, when the packet includes an IP header, the CBE preferably includes the following fields, illustrated in
In a preferred embodiment, when the packet includes a UDP header, the CBE preferably includes fields for a source port number, destination port number, UDP length, and UDP checksum fields as shown in
In a preferred embodiment, when the packet includes an TCP header, the CBE preferably includes fields for a source port number, destination port number, a sequence number, an acknowledgment number, a header length, a reserved field, flags, a window size, a TCP checksum, and an urgent pointer, as shown in
After sending the wire packet, engine 300 preferably increments the sequence number in the TCP control block entry. If the TCP packet is segmented, the sequence number is preferably updated in every wire data packet sent, but the sequence number in the control block is incremented after the entire media packet has been processed.
In a preferred embodiment, when the packet includes an HTTP header, the CBE preferably contains a “$” character, an HDCE byte field, and a total length field, as shown in
In a preferred embodiment, when the packet includes an RTP header, the CBE preferably includes flags, a CSRC count field, a payload type field, a sequence number, a timestamp, and a SSRC identifier, as shown in
After sending the wire packet, engine 300 preferably increments the sequence number field in the RTP CBE by 1.
In a preferred embodiment, the control block contains a payload data CBE, as shown in
In an alternative preferred embodiment, multiple PLDs may be pipelined together to execute additional algorithms, or more complex algorithms, in tandem. Embodiments comprising multiple PLDs preferably comprise additional communications structures in the PLD for inter-process communications between the PLDs in order to execute parallel algorithms.
While the invention has been described in conjunction with specific embodiments, it is evident that numerous alternatives, modifications, and variations will be apparent to those skilled in the art in light of the foregoing description.
This application claims benefit of U.S. provisional patent application Ser. No. 60/374,086, filed Apr. 19, 2002, entitled “Flexible Streaming Hardware,” U.S. provisional patent application Ser. No. 60/374,090, filed Apr. 19, 2002, entitled “Hybrid Streaming Platform,” U.S. provisional patent application Ser. No. 60/374,037, filed Apr. 19, 2002, entitled “Optimized Digital Media Delivery Engine,” and U.S. patent application Ser. No. 60/373,991, filed Apr. 19, 2002, entitled “Optimized Digital Media Delivery Engine,” each of which is hereby incorporated by reference for each of its teachings and embodiments.
Number | Name | Date | Kind |
---|---|---|---|
4731783 | Fontanes et al. | Mar 1988 | A |
4800431 | Deering | Jan 1989 | A |
5333299 | Koval et al. | Jul 1994 | A |
5367636 | Colley et al. | Nov 1994 | A |
5375233 | Kimber et al. | Dec 1994 | A |
5430842 | Thompson et al. | Jul 1995 | A |
5515379 | Crisler et al. | May 1996 | A |
5566174 | Sato et al. | Oct 1996 | A |
5638516 | Duzett et al. | Jun 1997 | A |
5689709 | Corbett et al. | Nov 1997 | A |
5710908 | Man | Jan 1998 | A |
5715356 | Hirayama et al. | Feb 1998 | A |
5719786 | Nelson et al. | Feb 1998 | A |
5737525 | Picazo, Jr. et al. | Apr 1998 | A |
5751951 | Osborne et al. | May 1998 | A |
5768598 | Marisetty et al. | Jun 1998 | A |
5781227 | Goode et al. | Jul 1998 | A |
5805804 | Laursen et al. | Sep 1998 | A |
5815516 | Aaker et al. | Sep 1998 | A |
5819049 | Rietmann | Oct 1998 | A |
5848192 | Smith et al. | Dec 1998 | A |
5857109 | Taylor | Jan 1999 | A |
5892535 | Allen et al. | Apr 1999 | A |
5915094 | Kouloheris et al. | Jun 1999 | A |
5925099 | Futral et al. | Jul 1999 | A |
5948065 | Eilert et al. | Sep 1999 | A |
5966162 | Goode et al. | Oct 1999 | A |
5995974 | Anton et al. | Nov 1999 | A |
5996015 | Day et al. | Nov 1999 | A |
6023731 | Chawla | Feb 2000 | A |
6047323 | Krause | Apr 2000 | A |
6088360 | Amaral et al. | Jul 2000 | A |
6101255 | Harrison et al. | Aug 2000 | A |
6108695 | Chawla | Aug 2000 | A |
6112226 | Weaver et al. | Aug 2000 | A |
6119154 | Weaver et al. | Sep 2000 | A |
6122670 | Bennett et al. | Sep 2000 | A |
6138147 | Weaver et al. | Oct 2000 | A |
6148414 | Brown et al. | Nov 2000 | A |
6157051 | Allsup et al. | Dec 2000 | A |
6157955 | Narad et al. | Dec 2000 | A |
6166730 | Goode et al. | Dec 2000 | A |
6182206 | Baxter | Jan 2001 | B1 |
6192027 | El-Batal | Feb 2001 | B1 |
6208335 | Gordon et al. | Mar 2001 | B1 |
6222838 | Sparks et al. | Apr 2001 | B1 |
6233607 | Taylor et al. | May 2001 | B1 |
6240553 | Son et al. | May 2001 | B1 |
6260155 | Dellacona | Jul 2001 | B1 |
6289376 | Taylor et al. | Sep 2001 | B1 |
6314572 | LaRocca et al. | Nov 2001 | B1 |
6314573 | Gordon et al. | Nov 2001 | B1 |
6345302 | Bennett et al. | Feb 2002 | B1 |
6496692 | Shanahan | Dec 2002 | B1 |
6498897 | Nelson et al. | Dec 2002 | B1 |
6502194 | Berman et al. | Dec 2002 | B1 |
6535518 | Hu et al. | Mar 2003 | B1 |
6535557 | Saito et al. | Mar 2003 | B1 |
6594775 | Fair | Jul 2003 | B1 |
6618363 | Bahl | Sep 2003 | B1 |
6640145 | Hoffberg et al. | Oct 2003 | B2 |
6651103 | Markowitz et al. | Nov 2003 | B1 |
6687757 | Ben-Ze'ev et al. | Feb 2004 | B1 |
6732104 | Weber | May 2004 | B1 |
6732243 | Busser et al. | May 2004 | B2 |
6820144 | Smirnov et al. | Nov 2004 | B2 |
6842785 | Norcott et al. | Jan 2005 | B1 |
6876653 | Ambe et al. | Apr 2005 | B2 |
6879266 | Dye et al. | Apr 2005 | B1 |
6879598 | Zancan et al. | Apr 2005 | B2 |
6944152 | Heil | Sep 2005 | B1 |
6944585 | Pawson | Sep 2005 | B1 |
6947430 | Bilic et al. | Sep 2005 | B2 |
6956853 | Connery et al. | Oct 2005 | B1 |
6963561 | Lahat | Nov 2005 | B1 |
6971043 | McLoughlin et al. | Nov 2005 | B2 |
6981167 | Johnson et al. | Dec 2005 | B2 |
6996618 | Apostolopoulos et al. | Feb 2006 | B2 |
7035278 | Bertram et al. | Apr 2006 | B2 |
7035295 | Belleguie et al. | Apr 2006 | B2 |
7042899 | Vaida et al. | May 2006 | B1 |
7043663 | Pittelkow et al. | May 2006 | B1 |
7152027 | Andrade et al. | Dec 2006 | B2 |
7174086 | Lin et al. | Feb 2007 | B2 |
7200670 | Hearn et al. | Apr 2007 | B1 |
7228358 | McManus | Jun 2007 | B1 |
7240113 | Barry et al. | Jul 2007 | B1 |
7260576 | Pasupathy et al. | Aug 2007 | B2 |
7266609 | Bill et al. | Sep 2007 | B2 |
7359955 | Menon et al. | Apr 2008 | B2 |
7460531 | Gupta et al. | Dec 2008 | B2 |
20010004767 | Gordon et al. | Jun 2001 | A1 |
20010019336 | Gordon et al. | Sep 2001 | A1 |
20010037443 | Liu | Nov 2001 | A1 |
20020000831 | Smith | Jan 2002 | A1 |
20020007417 | Taylor et al. | Jan 2002 | A1 |
20020067745 | Coupe et al. | Jun 2002 | A1 |
20020105905 | Boyle et al. | Aug 2002 | A1 |
20020107971 | Bailey et al. | Aug 2002 | A1 |
20020107989 | Johnson et al. | Aug 2002 | A1 |
20020150123 | Ro | Oct 2002 | A1 |
20020174227 | Hartsell et al. | Nov 2002 | A1 |
20030079018 | Lolayekar et al. | Apr 2003 | A1 |
20030095783 | Binder et al. | May 2003 | A1 |
20030097481 | Richter | May 2003 | A1 |
20030108030 | Gao | Jun 2003 | A1 |
20030135577 | Weber et al. | Jul 2003 | A1 |
20030221197 | Fries et al. | Nov 2003 | A1 |
20030223735 | Boyle | Dec 2003 | A1 |
20030227943 | Hallman et al. | Dec 2003 | A1 |
20040006635 | Oesterreicher et al. | Jan 2004 | A1 |
20040006636 | Oesterreicher et al. | Jan 2004 | A1 |
20040034712 | Rajwan et al. | Feb 2004 | A1 |
20040133570 | Soltis | Jul 2004 | A1 |
20060146780 | Paves | Jul 2006 | A1 |
Number | Date | Country |
---|---|---|
1484898 | Mar 2004 | CN |
0473102 | Mar 1992 | EP |
0781002 | Oct 2003 | EP |
406227 | Sep 2000 | TW |
421972 | Feb 2001 | TW |
435028 | May 2001 | TW |
447201 | Jul 2001 | TW |
452690 | Sep 2001 | TW |
452701 | Sep 2001 | TW |
453080 | Sep 2001 | TW |
454132 | Sep 2001 | TW |
457444 | Oct 2001 | TW |
460781 | Oct 2001 | TW |
465209 | Nov 2001 | TW |
465211 | Nov 2001 | TW |
475111 | Feb 2002 | TW |
WO 0004719 | Jan 2000 | WO |
WO 0033567 | Jun 2000 | WO |
WO 0042776 | Jul 2000 | WO |
WO 0045590 | Aug 2000 | WO |
WO 0059202 | Oct 2000 | WO |
WO 0059203 | Oct 2000 | WO |
WO 0059220 | Oct 2000 | WO |
WO 0059228 | Oct 2000 | WO |
WO 0131605 | May 2001 | WO |
WO 0143434 | Jun 2001 | WO |
WO 0143438 | Jun 2001 | WO |
WO 0152537 | Jul 2001 | WO |
WO 0155860 | Aug 2001 | WO |
WO 0155877 | Aug 2001 | WO |
WO 0156290 | Aug 2001 | WO |
WO 0245308 | Jun 2002 | WO |
03089944 | Oct 2003 | WO |
WO 2003090100 | Oct 2003 | WO |
WO 2003090101 | Oct 2003 | WO |
WO 2004034707 | Apr 2004 | WO |
WO 2006012496 | Feb 2006 | WO |
Number | Date | Country | |
---|---|---|---|
20030229778 A1 | Dec 2003 | US |
Number | Date | Country | |
---|---|---|---|
60374086 | Apr 2002 | US | |
60374090 | Apr 2002 | US | |
60373991 | Apr 2002 | US | |
60374037 | Apr 2002 | US |