The invention relates to a method and arrangement for analyzing a network message, e.g. for the purpose of testing a network protocol.
It is time consuming to create a model of the protocol for a robustness tester. The specification of the protocol is often proprietary.
The effort required for testing a proprietary protocol may be significantly reduced by utilizing a network protocol dissector. There are dissectors available for a large number of protocols e.g. through Wireshark open-source network capture application. The dissectors require for input network capture data (e.g. data in
PCAP format), that contains the header information of all layers of the network protocol stack. In other words, in order to be usable for a widely known network protocol dissector, the network traffic must be captured at lowest protocol level (e.g. Ethernet) of the network interface.
The dissectors are able to produce user-readable data, e.g. PDML (Packet Details Markup Language) output of the PCAP. PDML data contains the content of the captured data packets with useful meta-data information about the fields of the packet. Hence the data content of the packet is easier for a human to read in the PDML format. The PDML format is highly usable format for e.g. testing purposes.
There may be also other formats beside PDML, which may be used in a similar fashion.
When reading data at the level of lowest protocol of a network interface, all data of the network interface is captured. Filters may be applied to limit the captured data, but may be difficult to define appropriate filter for a task in hand. Capturing may produce potentially a large amount of data that is irrelevant for the tested higher level protocol. Furthermore, reading data at the level of lowest protocol of a network interface of a Device Under Test (DUT) often requires administrative rights to the operating system of the device. Alternatively or additionally, some special software may need to be installed to the device to capture the network data. When testing a device, such administrative rights may not be available and/or there is no possibility to install additional special-purpose software to the device for testing purposes.
Captures can be used as samples for outgoing messages, but incoming messages at test time may pose a problem, as they are not understood by the robustness testing SW without dissecting the message first. Protocols often have sequence numbers, identifiers and checksums etc, that are needed to be correct for the message exchange of the protocol to proceed. For example, when a message is sent, the received message may contain a “session id” value that needs to be placed for the 2nd message to be sent so that the messages are correctly processed by right recipient process in correct state. In PCAP format these kind of dynamic rules are not present. They must be created by the user.
User can select the values from the received protocol content and assign them to dynamic rules and the dynamic rule output to a message that will be sent next. Rule will calculate the correct values when the capture is replayed e.g. as a test. Rule can for example copy an ID from received message and replace the old, incorrect capture sample based value in the next outgoing message. This ensures that the protocol sequence proceeds beyond the initial message exchange, improving testing coverage. To be able to assign the rules to correct message field elements, it is very beneficial to be able to show to the user good descriptive field names. This user interactive rule assigning and interesting field marking is called edit time usage. The use of PDML format simplifies significantly such edit time usage.
When executing a test case of a network protocol, it is beneficial to know, especially for performance purposes, when to stop reading data from the network. Generally, the structure of the data read from the network must be known in order to determine whether all data relevant to testing has been read from the network. If the structure of the data is unknown to the testing process that reads data from the network interface, the data read process must wait until network timeout before it can stop the read operation and proceed with the test case. The waiting of timeouts slows down the testing process significantly. This kind of optimization of processing is called execution time usage.
As mentioned earlier, a PCAP capture can be processed in a dissector to produce user understandable format that has field names in place. TShark is a well known software application that can dissect PCAP files into PDML output. One problem is caused by the TShark application dissection logic. To be able to assign field names, the application may identify the used protocol by the standard or well known port number. For example, if TCP port used in PCAP is 80, the application may assume that the protocol is HTTP. If a non standard port is used, the data may be incorrectly decoded as some other protocol or may be not decoded at all.
An object of the present invention is to provide a method and arrangement that allows utilization of readily available software tools, e.g. a network protocol dissector, for testing a network protocol. Another object of the present invention may be to enhance the performance and controllability of test execution. Yet another object of the present invention may be to reduce the user access rights requirements of executing a test.
An aspect of the invention is a method for preparing data received from data communication network, e.g. for testing purposes. The method is characterized in that it comprises steps of reading data from a communication network socket, adding at least one header data field to the read data to create a packet interpretable by a network protocol dissector, and forwarding the created data packet to the network protocol dissector for further conversion of read data to a data analysis format.
The socket may be e.g. an IP, TCP, UDP, or STCP socket. There may be other corresponding socket interfaces.
The data fields of the step of adding at least one header data field may comprise data fields needed to convert the read data into format of the protocol of a lower or the lowest layer of e.g. the OSI protocol model. One such conversion is e.g. adding Ethernet protocol, IP protocol and TCP protocol headers to dissect data received from a TCP socket. One may also need to add fake capture frames to data, which make the result look like it is captured from network interface, for the dissector to accept the data and process it as expected. An example of such capture format is libpcap.
In an embodiment, data may be transmitted in the data communication network in an encrypted form. The encrypted data may be decrypted by a lower layer protocol before it is read from the communication network socket.
In an embodiment, the added header data field may be a data field of an arbitrarily selected network protocol. The network protocol may be selected e.g. based on the performance and/or capabilities of the protocol dissector.
In an embodiment, the method may comprise the step(s) of comparing a plurality of protocols in terms of dissecting efficiency and/or capability and/or selecting, e.g. based on the comparison, a suitable protocol according to which the data field(s) are added.
The format of the data to be forwarded to the dissector may be e.g. PCAP format or its functional equivalent.
The data analysis format may be e.g. PDML format or its functional equivalent.
The method may further comprise the step of controlling the read process utilizing the data converted to data analysis format.
In an embodiment, the method may comprise the step of composing a response message to the read message utilizing the data converted to the data analysis format and at least one rule to assign a value to a data field of the response message.
Another aspect of the invention is a computer arrangement that comprises the means for executing the steps of the method disclosed herein.
Yet another aspect of the invention is a computer program product that comprises the computer executable instructions for performing the steps of the method disclosed herein.
Some embodiments of the invention are described herein, and further applications and adaptations of the invention will be apparent to those of ordinary skill in the art.
In the following, the invention is described in greater detail with reference to the accompanying drawings in which
Because the data is read from the network from a relatively high layer, e.g. from TCP or UDP socket , administrator level access to the Device Under Test (DUT), from which the data packet is read, is typically not needed. This simplifies in many occasions the administrative burden of the testing process. Testing a protocol of the higher layer of the OSI model may be performed without having extensive admin rights to the device under test. Furthermore, by making PCAP messages with fake headers from the actual received data, there is no need for low level capture software, and still the data can be made into a format that is identified by the dissector application. Low level capture SW often requires administrative rights on the PC host where they are needed. In the invention these high level access rights are not needed.
In an embodiment, the data read from the socket can be appended with the fake lower level PCAP format headers and immediately be given to e.g. TShark program for decoding. The decoding may happen while data is still being read from the socket. If the PDML-output from TShark contains the needed fields, for example a counter value that is needed in the next outgoing message, the reading and processing can be stopped mid message, read data may be emptied and processing the real message testing can be made faster. And PDML output may explicitly state when the message has been read fully, at which state the reading can be stopped and there is no need to wait for the timeout value. Timeout can be many seconds, and in test scenarios with hundreds of thousands of test cases it is not practical to wait for timeouts.
In an embodiment, the user can select the low-level protocol in use, which is mapped to standard or well known ports that are used in the fake headers. This helps dissector with the correct decoding of the received data.
Sometimes the outgoing messages may also be created by dissecting some sample traffic, either captured or for which fake headers have been created as described earlier. Such output messages have the dissected protocol structure in place, and the structure may be used to help intelligently modify the outgoing message for testing purposes. However, since dissectors do not provide rules how the field values depend from each other and from received messages, structure modifications may make the message invalid for the protocol. These rules may be provided to augment this approach. The rules may say e.g. how fields in the structure relate to each other and also how fields from incoming messages are copied or otherwise used to determine the correct field values in later outgoing messages. For example, a “session id” field from incoming message must have identical value in outgoing message “session id” field.
Also, user can mark the fields needed so that in execution time, socket reading can be further optimized by reading the message only to the point where the user marked field is received and can be utilized in next outgoing message creation.
An advantage of an embodiment of the invention is that the incoming data field of interest (for rule processing) is not always at same offset from the beginning of the message. Likewise, the value may be at different offset in the outgoing message preceded by some variable length dynamic data. There may be variable length data before the interesting data. PDML meta-data is not offset sensitive, instead it clearly labels the protocol fields based on the internal dissection information (simple model) giving a reliable place where the interesting field value begins. Thus the reliability of rule source and target data is greatly enhanced.
An embodiment of the invention provides transparency across different low-level encryption methods or unsupported transport protocols. E.g. analyzed application data may be protected by Transport Layer Security (TLS) encryption so that it is impossible for a network capture system to decrypt the data. However, with this invention the data is read after the TLS sub-system has decrypted the data.
There is no need to add any encryption for the appended fake fields and thus even the originally TLS encrypted data is in non-encrypted form and suitable for the dissection. The same applies if the data is originally received over a transport layer which would not be supported by the dissector or whose dissecting requires more processing power or memory resources than dissecting of some other protocol. As the fake headers which the dissector can dissect successfully and/or efficiently are appended, advantage of the analysis output can be taken, even when the original read data would not be dissected at all or would be dissected in a non-optimal manner.
An embodiment of the invention provides a way for more compact way of storing large amount of application data compared to saving full network captures. Full network captures make it possible to afterward dissect selected data. With this invention one can only store application data and later, when required, add the fake headers to allow dissection of data. Since storage does not need to contain the headers, the data is in more compact form.
To a person skilled in the art, the foregoing exemplary embodiments illustrate the model presented in this application whereby it is possible to design different methods, arrangements and software programs, which in obvious ways to the expert, utilize the inventive idea presented in this application.
Number | Date | Country | Kind |
---|---|---|---|
20105450 | Apr 2010 | FI | national |
This is a US national phase patent application that claims priority from PCT/FI2011/050377 filed 26 Apr. 2011, that claims priority from Finnish Patent Application No. 20105450, filed 26 Apr. 2010.
Filing Document | Filing Date | Country | Kind | 371c Date |
---|---|---|---|---|
PCT/FI2011/050377 | 4/26/2011 | WO | 00 | 10/12/2012 |