1. Technical Field
The present disclosure generally relates to classification of packets transmitted over a network and particularly to the determination of content to be stored in storage located along the network based on the type of data transferred in the packet. Even more particularly, the present disclosure relates to traffic analytics respective of the data and generation of a simulation respective of the analysis.
2. Description of the Related Art
Service providers worldwide are facing a mounting problem of flattening revenues coupled with increasing costs brought forth by increasing usage of bandwidth, growing churn, subscriber saturation and competitive price pressures. These mobile and broadband providers are facing challenges in delivering new revenue generating services while seeing their over the top (OTT) counterparts reaping rewards with content delivered over the networks they built and maintained over the years.
The vast majority of these OTT services are delivered over hypertext transfer protocol (HTTP), the de-facto protocol for application development and delivery. Be it video, social networking, search, or advertising, and be it over fixed line or mobile applications, these OTT services are most likely running on top of HTTP. However, this protocol is also the most processing intensive protocol for network devices. Hence, practically any increase in usage increases the strain on the service providers.
Certainly, one way to control traffic on the Internet requires various levels of understanding of the traffic that flows through the network, and that understanding is also increasing in its level of sophistication. Various systems and solutions have been offered in the related art that utilize deep-packet-inspection (DPI) to enable an ever sophisticated examination of network traffic and ability to shape the traffic on the network. This ability allows the service providers to better manage the network and its related resources and provide a higher level of quality of service (QoS) in the hopes of increasing revenues and profits.
However, the rapid increase in the delivery of heavy bandwidth consuming data, such as video, and consumption thereof, requires a new level of handling that is not available today in related art solutions. A known problem is the access of a user to a content source and subsequently the access by another user to the same content, resulting in additional load on the content provider and on the entire network. When, for example, popular video clips are accessed, there is a significant and noticeable degradation of network performance which may even lead to a network failure. Some related art solutions attempt to store all the data in caches. However, the huge amounts of data and the need to inspect each and every packet, regardless of its source, makes this a daunting and impractical task.
It would be advantageous to provide service providers with a solution that would enable them to effectively manage and control the delivery of heavy bandwidth consuming data such that the overall bandwidth requirements are loaded and better shared across the network in general, and in particular within the network of a specific service provider. It would be further advantageous to provide a system and method for performing traffic analytics that determines the effect such a solution provides or could provide.
Accordingly, one or more exemplary embodiments provide a solution that effectively manages and controls the delivery of heavy bandwidth consuming data such that the overall bandwidth requirements are loaded and better shared across the network, particularly within the network of a specific service provider. Further, techniques are provided for performing traffic analytics that determine the effect such a solution provides or could provide.
The foregoing and/or other aspects of the exemplary embodiments may be achieved by a computerized method of generating a traffic simulation respective of at least one content storing server for caching popular content. This method may include sniffing traffic between at least a first portion of a network and at least a second portion of the network, identifying, from the sniffed traffic, at least a source of content and at least a destination of the content, determining if the content is stored on a cache, simulating a notification respective of the content, and generating a simulation of traffic respective at least of: the content and the simulated notification.
The computerized method may further include determining, upon determining that the content is not stored on the cache, if the content should be acquired to be stored on the cache.
The computerized method may further include simulating, upon determining that the content should be acquired, a “store” notification respective of the content.
The computerized method may further include simulating, upon determining that the content should not be acquired, a “no deliver” notification respective of the content.
The computerized method may further include determining, upon determining that the content is stored on the cache, if the content can be delivered from the cache.
The computerized method may further include simulating, upon determining that the content can be delivered from the cache, a “potential delivery” notification respective of the content.
The computerized method may further include simulating, upon determining that the content cannot be delivered from the cache, a “no deliver” notification respective of the content.
The identifying may be performed by a deep-packet-inspection (DPI) unit.
The computerized method may further include receiving a size of the cache for storing the content, wherein the generating may further include generating the simulation of traffic respective of the cache size.
The computerized method may further include determining, upon determining that the content is not stored on the cache, if the content should be acquired to be stored on the cache.
The computerized method may further include generating, upon determining that the content should be acquired, a “store” notification respective of the content.
The computerized method may further include generating, upon determining that the content should not be acquired, a “no deliver” notification respective of the content.
The computerized method may further include determining, upon determining that the content is stored on the cache, if the content can be delivered from the cache.
The computerized method may further include generating, upon determining that the content can be delivered from the cache, a “potential delivery” notification respective of the content.
The computerized method may further include generating, upon determining that the content cannot be delivered from the cache, a “no deliver” notification respective of the content.
The foregoing and/or exemplary embodiments may be achieved by an apparatus for generating a traffic simulation respective of at least one content storing server. The apparatus may include a first network interface to sniff traffic between at least a first portion of a network and at least a second portion of the network, a storage for storing at least information respective of content received through the first network interface, a second network interface configured to communicate with the at least first portion of the network and the at least second portion of the network, a processing unit; and a memory containing instructions that, when executed by the processing unit, cause the apparatus to: sniff traffic between at least the first portion of the network and at least the second portion of the network; identify from the sniffed traffic at least a source of the content and at least a destination of the content; determine if the content is stored on a cache; simulate a notification respective of the content; and generate a simulation of traffic respective at least of: the content and the simulated notification.
The apparatus may further include a deep-packet-inspection (DPI) unit coupled to the first network interface and configured to identify at least the source of the content respective of the sniffed traffic, the DPI unit further configured to inspect one or more packets provided, through the first network interface, from the identified source of the content, each packet having at least a specific source address and a specific destination address.
The memory may further contain instructions that, when executed by the processing unit, cause the apparatus to: determine, upon determining that the content is not stored on the cache, if the content should be acquired to be stored on the cache.
The memory further contains instructions that, when executed by the processing unit, cause the apparatus to: simulate, upon determining that the content should be acquired, a “store” notification respective of the content.
The memory further contains instructions that, when executed by the processing unit, cause the apparatus to: simulate, upon determining that the content should not be acquired, a “no deliver” notification respective of the content.
The memory further contains instructions that, when executed by the processing unit, cause the apparatus to: determine, upon determining that the content is stored on the cache, if the content can be delivered from the cache.
The memory further contains instructions that, when executed by the processing unit, cause the apparatus to: simulate, upon determining that the content can be delivered from the cache, a “potential delivery” notification respective of the content.
The memory further contains instructions that, when executed by the processing unit, cause the apparatus to: simulate, upon determining that the content cannot be delivered from the cache, a “no deliver” notification respective of the content.
The foregoing and/or other aspects of the exemplary embodiments may be achieved with a non-transitory computer readable storage medium storing a program for executing a method of generating a traffic simulation respective of at least one content storing server, the content storing server operative for caching popular content, the method including sniffing traffic between at least a first portion of a network and at least a second portion of the network, identifying, from the sniffed traffic, at least a source of content and at least a destination of the content, determining if the content is stored on a cache; simulating a notification respective of the content, and generating a simulation of traffic respective at least of: the content and the simulated notification.
The foregoing and other objects, features, and advantages will become apparent and more readily appreciated from the following detailed description taken in conjunction with the accompanying drawings in which:
Below, exemplary embodiments will be described in detail with reference to accompanying drawings so as to be easily realized by a person having ordinary knowledge in the art. The exemplary embodiments may be embodied in various forms without being limited to the exemplary embodiments set forth herein. Descriptions of well-known parts are omitted for clarity, and like reference numerals refer to like elements throughout.
It is important to note that the embodiments here disclosed are only examples of the many advantageous uses of the innovative teachings herein. In general, statements made in the specification of the present application do not necessarily limit any of the various claims. Moreover, some statements may apply to some inventive features but not to others. In general, unless otherwise indicated, singular elements may be in plural and vice versa with no loss of generality.
Disclosed are systems and methods for performing a simulation of efficient usage of a bandwidth of network transferring content, such as video data, when using caching techniques. Packets from content sources are analyzed to identify content sources that provide predetermined types of data. Upon identification of such content sources, any data that arrives from such a content source is subject to a deep-packet-inspection (DPI) process to positively identify the content and determine whether or not to store it in cache storage so that when a subsequent request for the same content is received, there is no need to transmit the content from the content source, since it can simply be delivered from the system's storage. The results include at least less traffic going on the entire network, faster service, and lower operational costs. The simulation provides a prediction of potential effectiveness of the system for a given network and its respective traffic when being used.
Reference is now made to
When a UN 150 requests content from a CS 140 it is transferred, according to the exemplary embodiment, through a service delivery apparatus (SDA) 110, the function of which is described in more detail herein below. Generally, the SDA 110 may provide the requested content from its storage or, when such content, or portions thereof, are not in the SDA 110, then the request is forwarded to the appropriate CS 140 for the delivery of the content, as further described below.
The DPI unit 210 is provided with data types to look for in data packets that are transmitted through the SDA 110. Instead of inspecting each and every packet, the DPI unit 210 may inspect only one-in-a-thousand packets out of the entire traffic thereby significantly lowering the processing load. It should be understood that the method for selecting the sampled packets is typically not performed by using a simple counter to process one out of every predetermined number of packets. Instead, the source and destination addresses from each packet are fed into a hash function, and the hash function result is compared to a configurable threshold, and the result of this comparison determines if the packet is inspected or not. In addition, it should be understood that the hash function is symmetric with respect to the source and destination addresses, such that swapping the source address and the destination address does not change the hash result.
In one exemplary embodiment, the source and destination ports may also be used as part of the hash function operation. This is needed to guarantee that each flow comprising of multiple packets sent between a UN 150 and a CS 140 is either fully ignored or fully inspected.
Upon determination that a specific CS 140 provides a desired data type, the identification of that CS 140 is stored. Any future packet received from or sent to the identified CS 140 is inspected by the DPI unit 210 and if the packet contains content that may be interesting for storage, such as video content, such content is stored in the storage 220.
This kind of inspection ensures that demand for highly popular content from a popular CS 140 is likely to be quickly detected while infrequent access to a CS 140 would typically not impact the traditional operation of the system. It should be noted that identification of a CS 140 does not have to be on the first detection of data of interest and threshold levels may be used, as well as an aging mechanism so that relatively infrequently accessed CSs 140 would lose older accesses, and those older accesses would therefore not accumulate to reach a threshold value. The threshold may be static, dynamic or adaptive. Static thresholds are predetermined thresholds that remain constant. Dynamic thresholds are forcefully changed, for example, at a certain time of day, or according to information respective of the user. Adaptive thresholds are changed in response to changes in characteristics of the website engagement and may vary depending on a variety of parameters.
While DPI unit 210 operates on the packets that arrive from CSs 140, the CDU 230 operates with respect of requests for content received from the UNs 150 of the service provider network 130. Upon receipt of such a request, the DPI 210 first checks if content from the requested CS 140 actually resides in the storage 220 by first checking that the CS 140 identification is known to the SDA 110. If that is the case, then the storage 220 is checked for the possibility of delivery of the content or portions thereof. If the entire content or portions thereof are found, then these are delivered to the requesting UN 150. If the entire content is missing, or certain portions thereof are missing, then the request is forwarded to the appropriate CS 140. Storage 220 may be semiconductor media, magnetic media, or any other type of storage media appropriate for the storage of content.
Reference is now made to
In S330, it is checked if the data in the packet corresponds to the data parameters, e.g., contains video data, and if so, execution continues with S340; otherwise, execution continues with S370. In S340, the count respect of the CS 140 that is the source of the packet is updated, for example but not by way of limitation, by incrementing the value of a counter. In S350, it is checked if the count for that CS 140 has exceeded a threshold value and if so execution continues with S360; otherwise, execution continues with S370. In one exemplary embodiment, the count may also have an aging mechanism as described previously (not shown). Furthermore, different data types may have different thresholds, different count increases, and different count aging. In S360, the CS 140 is identified as a source of content eligible for storage in a storage unit, for example, storage 220. In S370, it is checked if there are more data packets to be inspected and if so execution continues with S320; otherwise execution terminates.
Reference is now made to
In S440, it is checked whether such content of interest was found and if so execution continues with S450; otherwise, execution continues with S460. In S450, the content from the received packet is stored in storage, for example, storage 220. In S460, it is checked whether more packets are received and if so execution continues with S410; otherwise, execution terminates.
Reference is now made to
In S530, it is checked whether the requested content is in storage, for example storage 220, and if so execution continues with S540; otherwise, execution continues with S560. In S540, it is checked whether the whole content is in storage and if so execution continues from S550; otherwise, execution continues with S560. The need to check storage twice (i.e., in operations S530 and S540) arises because the requested content is not the same as the whole content. In a video file, for example, the file is broken up into smaller pieces. In this exemplary embodiment, the requested content would be one of the smaller pieces, while the whole content would be the entire video file.
In S550, the content is delivered to the requesting UN 150. In S560, it is checked whether additional content requests exist and if so execution continues with S510; and if not, execution terminates.
In one exemplary embodiment, when detecting that a portion of the requested content is in the storage 220 and deliverable to the requesting UN 150, such content is delivered immediately to the UN 150 while only the missing portions of the content is requested from the CS 140. Hence a request from the CDU 230 may be for the requested content or portions thereof. It should be further understood that in an exemplary embodiment, once the DPI unit 210 determines that a CS 140 may contain content that should be stored in storage 220, the packets from such a CS 140 are consistently inspected for determination of popular content.
In S620 a content source and a content destination are identified, respective of the sniffed traffic. Further information may be identified, such as type of content, size of content, time stamps and the like. In S630 a check is performed to determine if the identified content is in a cache of an apparatus, such as SDA 110. If the content is in a cache, execution continues at S650, otherwise execution continues at S640.
In S640, a check is performed to determine if the content should be acquired. If the content should be acquired, execution continues at S645, otherwise execution continues at S660. In S645, a “store” notification is simulated, indicating that the identified content should be stored in a cache. The notification may be stored in a memory and used to generate a simulation.
In S650, a check is performed to determine if the identified content can be delivered from the cache. An example of when content might be stored in the cache but cannot be delivered would be if there are copyright issues. For instance, if the content stored in the cache is licensed for viewing only within one country, the content cannot be delivered if the UN 150 is in another country. If the content can be delivered from the cache, execution continues at S655, otherwise execution continues at S660. In S655, a “potential delivery” notification is simulated, indicating that content stored in a cache may be delivered. The notification may be stored in a memory and used to generate a simulation.
In S660, a “no deliver” notification is simulated, indicating that the identified content cannot be delivered. The notification may be stored in a memory and used to generate a simulation. In certain exemplary embodiments, a further check may be performed to determine if content which cannot be delivered should be deleted from the cache.
A check is performed in S670 to determine if the results of the simulation should be generated. For example, it may not be desirable to generate a simulation if a sufficient amount of data points have not yet been collected. If it is determined that the results of the simulation should be generated, execution continues at S680, otherwise execution continues at S610. In S680, the results of the simulation are generated, respective of at least one simulated notification of any of S645, S655 or S660, perhaps in the form of a graph as shown in
In certain embodiments, actual notifications respective of S645, S655, and S660, or combination of actual and simulated notifications may be used to generate the results of the simulation. An actual notification is generated by the SDA 110 during its course of run. A simulated notification is generated during a “what-if” scenario. For example, if a lower threshold by which acquisition decisions are made is simulated (to acquire more content), additional notifications would be created.
In an embodiment, the results of the simulation may be an illustration of the network operating with the SDA 110 described in more detail herein, and may also include a comparison to the same network operating without the SDA 110 present. The results of the simulation may be generated in real-time or at a later time using notifications stored as information in a memory. The results of the simulation may further be respective of limitations of the SDA 110, such as the physical size of the cache. In such an exemplary embodiment, the results of the simulation may be generated by simulating any cache size to determine the content which would be available for delivery, and actual notifications would correspond to the actual cache size while simulated notifications would correspond to the simulated cache size.
This allows for more efficient use of resources, since a larger (and therefore costlier) cache size would not necessarily result in more content being delivered. By simulating different cache sizes, it is possible to find an optimal cache size for the content storing server. The method shown herein may be performed by the SDA 110 itself, being appropriately configured, or otherwise, by a simulator device having access to data from the SDA 110 via its network connectivity interfaces.
The processing element 710 may be coupled to an input device 750, e.g., a mouse and/or a keyboard, and a data storage 730. Data storage 730 may be used for the purpose of holding a copy of the method executed according to the disclosed technique in
Reference is now made to
The relationship between
The principles of the disclosure are implemented as hardware, firmware, software or any combination thereof. Moreover, the software is preferably implemented as an application program tangibly embodied on a program storage unit or computer readable medium. The application program may be uploaded to, and executed by, a machine comprising any suitable architecture. Preferably, the machine is implemented on a computer platform having hardware such as one or more central processing units (“CPUs”), a memory, and input/output interfaces. The computer platform may also include an operating system and microinstruction code. The various processes and functions described herein may be either part of the microinstruction code or part of the application program, or any combination thereof, which may be executed by a CPU, whether or not such computer or processor is explicitly shown. In addition, various other peripheral units may be connected to the computer platform such as an additional data storage unit and a printing unit and/or display unit.
All examples and conditional language recited herein are intended for pedagogical purposes to aid the reader in understanding the principles of the present disclosure and the concepts contributed by the inventor to furthering the art, and are to be construed as being without limitation to such specifically recited examples and conditions. Moreover, all statements herein reciting principles, aspects, and embodiments, as well as specific examples thereof, are intended to encompass both structural and functional equivalents thereof. Additionally, it is intended that such equivalents include both currently known equivalents as well as equivalents developed in the future, i.e., any elements developed that perform the same function, regardless of structure.
This application claims priority from U.S. provisional patent application No. 61/969,418 filed Mar. 24, 2014 and is a continuation-in-part of U.S. patent application Ser. No. 13/006,785 filed Jan. 14, 2011, entitled “A System for Detection of Content Servers and Caching Popular Content Therein”, which claims priority from U.S. provisional patent application No. 61/375,836, entitled “A System for Detection of Content Servers and Caching Popular Content Therein”, filed on Aug. 22, 2010, all of the above-noted applications being assigned to a common assignee and all of the above-noted applications are hereby incorporated by reference in their entirety.
Number | Name | Date | Kind |
---|---|---|---|
5829046 | Tzelnic et al. | Oct 1998 | A |
5893140 | Vahalia et al. | Apr 1999 | A |
5933603 | Vahalia et al. | Aug 1999 | A |
5944789 | Tzelnic et al. | Aug 1999 | A |
5948062 | Tzelnic et al. | Sep 1999 | A |
6049530 | Petersen et al. | Apr 2000 | A |
6061504 | Tzelnic et al. | May 2000 | A |
6363413 | Kidder | Mar 2002 | B2 |
6473794 | Guheen | Oct 2002 | B1 |
6536037 | Guheen | Mar 2003 | B1 |
6615166 | Guheen | Sep 2003 | B1 |
6700889 | Nun | Mar 2004 | B1 |
6772193 | Igawa et al. | Aug 2004 | B1 |
6799248 | Scherr | Sep 2004 | B2 |
6823401 | Feather, Jr. | Nov 2004 | B2 |
6831893 | Ben-Nun et al. | Dec 2004 | B1 |
6873600 | Duffield | Mar 2005 | B1 |
6985956 | Luke | Jan 2006 | B2 |
6986018 | O'Rourke et al. | Jan 2006 | B2 |
7149698 | Guheen | Dec 2006 | B2 |
7281260 | Puente et al. | Oct 2007 | B2 |
7310480 | Maciocco et al. | Dec 2007 | B2 |
7349979 | Cieslak | Mar 2008 | B1 |
7436830 | Ben-Nun et al. | Oct 2008 | B2 |
7596664 | Ishikawa et al. | Sep 2009 | B2 |
7606314 | Coleman et al. | Oct 2009 | B2 |
7685254 | Pandya | Mar 2010 | B2 |
7697557 | Segel | Apr 2010 | B2 |
7719966 | Luft et al. | May 2010 | B2 |
7818402 | Zhang | Oct 2010 | B1 |
7912921 | O'Rourke et al. | Mar 2011 | B2 |
7957396 | Kohn et al. | Jun 2011 | B1 |
8009682 | Gopinath | Aug 2011 | B2 |
8065559 | Kamath et al. | Nov 2011 | B2 |
8607166 | Jalon et al. | Dec 2013 | B2 |
8621101 | Starr | Dec 2013 | B1 |
8706900 | Carver et al. | Apr 2014 | B2 |
8737407 | Shetty | May 2014 | B2 |
8937942 | Li | Jan 2015 | B1 |
20010049732 | Raciborski et al. | Dec 2001 | A1 |
20010051980 | Raciborski et al. | Dec 2001 | A1 |
20020006124 | Jimenez et al. | Jan 2002 | A1 |
20020040366 | Lahr | Apr 2002 | A1 |
20020040404 | Lahr | Apr 2002 | A1 |
20020042817 | Lahr | Apr 2002 | A1 |
20020048269 | Hong et al. | Apr 2002 | A1 |
20020062372 | Hong et al. | May 2002 | A1 |
20020136204 | Chen et al. | Sep 2002 | A1 |
20030097443 | Gillett et al. | May 2003 | A1 |
20030221127 | Risan et al. | Nov 2003 | A1 |
20040128693 | Weigand | Jul 2004 | A1 |
20040133776 | Putzolu | Jul 2004 | A1 |
20040181579 | Huck et al. | Sep 2004 | A1 |
20050015702 | Shier | Jan 2005 | A1 |
20050022237 | Nomura | Jan 2005 | A1 |
20050213514 | Su | Sep 2005 | A1 |
20050289111 | Tribble et al. | Dec 2005 | A1 |
20060129697 | Vange et al. | Jun 2006 | A1 |
20060168318 | Twiss | Jul 2006 | A1 |
20060224687 | Popkin et al. | Oct 2006 | A1 |
20060271972 | Pai et al. | Nov 2006 | A1 |
20070050686 | Keeton | Mar 2007 | A1 |
20070124781 | Casey et al. | May 2007 | A1 |
20070192474 | Decasper et al. | Aug 2007 | A1 |
20070244987 | Pedersen et al. | Oct 2007 | A1 |
20080010381 | Barraclough et al. | Jan 2008 | A1 |
20080307343 | Robert et al. | Dec 2008 | A1 |
20090083279 | Hasek | Mar 2009 | A1 |
20090119734 | Deshpande et al. | May 2009 | A1 |
20090172565 | Jackson et al. | Jul 2009 | A1 |
20090193129 | Agarwal et al. | Jul 2009 | A1 |
20090307757 | Groten | Dec 2009 | A1 |
20090313437 | Sofman et al. | Dec 2009 | A1 |
20100023726 | Aviles | Jan 2010 | A1 |
20100054257 | Dolganow et al. | Mar 2010 | A1 |
20100082774 | Pitts | Apr 2010 | A1 |
20100115072 | Payyappilly et al. | May 2010 | A1 |
20100287227 | Goel | Nov 2010 | A1 |
20110055386 | Middleton et al. | Mar 2011 | A1 |
20110078343 | Resch et al. | Mar 2011 | A1 |
20110107185 | Grube et al. | May 2011 | A1 |
20110141887 | Klein et al. | Jun 2011 | A1 |
20110153937 | Annamalaisami et al. | Jun 2011 | A1 |
20120011271 | Zhao | Jan 2012 | A1 |
20120030212 | Koopmans et al. | Feb 2012 | A1 |
20120047224 | Shemesh et al. | Feb 2012 | A1 |
20120047252 | Maor et al. | Feb 2012 | A1 |
20120057460 | Hussain et al. | Mar 2012 | A1 |
20120084464 | Cochinwala et al. | Apr 2012 | A1 |
20120117610 | Pandya | May 2012 | A1 |
20120124618 | Ruiz-Velasco et al. | May 2012 | A1 |
20120159329 | Chow et al. | Jun 2012 | A1 |
20120159558 | Whyte et al. | Jun 2012 | A1 |
20140108586 | Zhao | Apr 2014 | A1 |
20160021057 | Frost et al. | Jan 2016 | A1 |
Entry |
---|
Non-Final Office Action dated Sep. 25, 2012 for U.S. Appl. No. 13/006,785. |
Non-Final Office Action dated Jul. 26, 2012 in U.S. Appl. No. 13/006,875. |
Number | Date | Country | |
---|---|---|---|
20150193567 A1 | Jul 2015 | US |
Number | Date | Country | |
---|---|---|---|
61969418 | Mar 2014 | US | |
61375836 | Aug 2010 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 13006785 | Jan 2011 | US |
Child | 14665884 | US |