Created a tool to output a sequence diagram from a packet capture file of multiple nodes

Created a tool to output a sequence diagram from a packet capture file of multiple nodes

1. Overview

I created a tool called packetseq that outputs a packet sequence diagram in PNG format from a file that was simultaneously captured by Wireshark or tcpdump on multiple nodes on the network. For the language, I used Python and Seqdiag, a library that creates sequence diagrams.

1-1. Background of tool creation

In order to investigate the network delay of the customer environment at the company, the packets collected at each communication node were matched. Since the packet is in pcap format, I browsed it with wireshark, and at first I opened multiple Wiresharks and matched them with my eyes, but it was impossible. Wireshark has a flow diagram, a tool that displays a specific connection, but I gave up because it cannot be used to match packets collected at multiple locations. Since there is no help for it, I decided to create a packet sequence on Excel. I narrowed down the capture file to a specific port communication from the client, exported the packet, converted it to CSV format, and created a sequence diagram by hand on Excel. It was useful for the team to discuss, but it took too long and I didn't want to repeat the same thing again, so I created the tool.

Environment at that time (reference)

Http Client --- Internet --- LLB --- FW --- SLB --- Http Server

1-2. What you need to run the tool

"No.","Time","Source","Destination","Protocol","Length","Info"
 Date and time format (1973-06-14 01: 02: 03.123456)

1-3. Referenced sites

2. Flow

2-1. Environment construction

  1. Install Python 2.7
$ curl -L -O https://www.python.org/ftp/python/2.7.12/Python-2.7.12.tgz
$ tar zxvf Python-2.7.12.tgz
$ cd Python-2.7.12
$ ./configure
$ make && make altinstall
  1. Install pip
$ curl -kL https://raw.github.com/pypa/pip/master/contrib/get-pip.py | python
  1. Modified to use yum Fixed yum command interpreter to run existing Python (Version 2.6)
$ vi /usr/bin/yum
 Example) Corrected as follows
!/usr/bin/python2.6
  1. Preparation of the created tool --When using as a command
$ pip install packetseq

--When using only the program Save to the environment where packetseq.py is executed

2-2. Usage procedure (overview)

  1. Packet capture at the communication path location
  2. Export only the communication of a specific port from the capture file
  3. Save as csv file
  4. Send the csv file on the tool execution environment created this time
  5. Run the tool
  6. Enter the name of the opposite device between communication from standard input
  7. Sequence diagram created

2-3. Usage procedure (explanation)

This time I will explain how to use it based on the following environment

 Client (Windows 7) --- Proxy (Squid) --- Server (nifty.com, etc.)
  1. Packet capture at the communication path location Wireshark was used on Client (Windows 7) and tcpdump was used on Proxy (Squid). Example) Packet capture on the client
 Run wireshark and save when communication is complete
 File name is arbitrary, save as client.pcap

Example) Packet capture on a proxy server

$ tcpdump -i eth1 -w proxy.pcap
 File name is arbitrary, save as proxy.pcap
  1. Export only the communication of a specific port from the capture file Open the file you captured earlier with wireshark and narrow it down to a specific port Specifically, check the port number when sending the SYN flag of 3way handshake. Filter on wireshark
 tcp.port == port number
  1. Save as csv file Save in csv format with wireshark Example) Packet capture on the client
 File name is arbitrary, save as client.csv

Example) Packet capture on a proxy server

 File name is arbitrary, save as proxy.csv
  1. Send the csv file on the tool execution environment created this time
  2. Run the tool Since there are two collection points this time, the files captured earlier are passed as arguments in order from the client side and executed. --When installing from pip etc. and using from commands
$ packetseq client.csv proxy.csv

--When executing from saved packetseq.py

$ python packetseq.py client.csv proxy.csv 
  1. Enter the name of the opposite device between communication from standard input You will be prompted with standard input to enter the name twice, source-ip and destination-ip for each file The example below is between a client and a proxy, so I chose "Client" and "Proxy".
\########################################
file_name:client.csv
\########################################
src ip:192.168.1.3 -> src name: ???
 input src name> Client (★ standard input)
dst ip:192.168.1.62 -> dst name: ???
 input dst name> Proxy (★ standard input)

Next, input is required between proxy servers, so I chose "Proxy" and "Sever".

\########################################
file_name:proxy.csv
\########################################
src ip:192.168.1.3 -> src name: ???
 input src name> Proxy (★ standard input)
dst ip:192.168.1.62 -> dst name: ???
 input dst name> Server (★ standard input)
  1. Sequence diagram created Files out.png and out.diag are created out.png is a PNG format sequence diagram Seqdiag format file from which out.diag is the source of image generation
$ file out.*
out.diag: ASCII text, with very long lines
out.png:  PNG image data, 1856 x 31706, 8-bit/color RGBA, non-interlaced

From the left, it is a sequence diagram of "Client", "Proxy", "Web". packetseq_img.png
With this, you can see the sequence of packets at multiple locations.

3. About tools

3-1. Source code

See packetseq.py

3-2. Process flow

  1. Pass the csv file as an argument
  2. Detect between communications from the SYN flag for each file 3.2 Pass the name of the opposite node of 2 as standard input
  3. Replace the IP address with the name specified in 3.
  4. Output to Seqdiag format
  5. Pass it to Seqdiag and output the image

3-3. Optional

option Description Remarks
-h, --help Show help message -
-o, --out output file name Specify the output destination file name -
-i, --info Output type Select to output packet information on the sequence Output type is summary, info,Choose from 3 defaults
-t, --type Select png or svg as the output format -

3-4. Color specification

Css color keyword can be specified Example) Specification on the source code

tcp flag color
SYN blue
SYN ACK red
ACK green
FIN navy
FIN ACK maroon
RST purple
RST ACK fuchsia
URG olive
PSH orange
Other gray

Color settings are specified in self.color_dict of the \ __ init__ function

 #Dictionary of color information
        self.color_dict = {
            'SYN': "blue", 'SYN ACK': "red",
            'ACK': "green",
            'FIN': "navy", 'FIN ACK': "maroon",
            'RST': "purple", 'RST ACK': "fuchsia",
            'URG': "olive", 'PSH': "orange",
            'Other': "gray",
        }

3-5. Output of packet information

  1. "summary" tcp flag and send data only summary.png
  2. "info" info Full data display info.png
  3. "default" Remove sport and dport default.png

4. Challenges

5. Attention

Recommended Posts

Created a tool to output a sequence diagram from a packet capture file of multiple nodes
Write standard output to a file
How to output the output result of the Linux man command to a file
Output search results of posts to a file using Mattermost API
DataFrame of pandas From creating a DataFrame from two lists to writing a file
A python script that draws a band diagram from the VASP output file EIGENVAL
I made a tool to generate Markdown from the exported Scrapbox JSON file
From Excel file to exe and release of tool that spits out CSV
Created a tool to mechanically visualize from the repository history in order to objectively know the productivity and soundness of the development team
Created a module to monitor file and URL updates
Make a copy of a Google Drive file from Python
Python script to create a JSON file from a CSV file
Output a binary dump in binary and revert to a binary file
Procedure from environment construction to operation test of testinfra, a server environment test tool made by Python
I made a tool to automatically generate a simple ER diagram from the CREATE TABLE statement