Packet Sniffers
By Vincent JARDIN – 6WIND CTO
Packet sniffers are used to analyse packet flows for many applications (anti-virus, network debugging, accounting, etc.). If we examine packet sniffer implementations, they comprise three layers:
- Kernel layer that captures all the packets on top of Ethernet drivers (for instance with Linux see af_packet.c, DLPI on Solaris, etc.).
- Capture library that opens the userland/kernel communication API that will be used to get the copy of the packets from the kernel to userland (for instance with Linux, you can open an AF_PACKET socket). The most widely-known library which works on any OS is libpcap. It abstracts packet capture from the underlying kernel implementation.
- Packet dissector which parses the content and either displays the payload in a human readable manner or stores the packets; the most widely-known packet dissectors are tcpdump and wireshark (ex ethereal).
Based on this architecture, you can capture packets which are forwarded by the kernel itself, but you cannot capture packets which are forwarded by a system made of 10Gbps ports running a dedicated fast data plane outside the OS.
Requirements for Packet Sniffers in Fast Data Plane
A packet processing solution based around a Fast Path must include the capability for packet captures. In order to maximize software reuse and minimize development time, the capture library (libpcap) and the packet dissectors (tcpdump, wireshark) must be re-used as-is without any modification of their source code. It means that the Fast Path has to provide a tcpdump-like API that can be used for flow inspection.
Some vendors sell dedicated tapping boards based on FPGAs or network processors, but these boards are very expensive to integrate and customize. So, we will skip this option and assume that the packet sniffers have to be part of the Fast Path.
The requirements for the Fast Path tcpdump-like flow inspection modules are:
- Reuse Linux tcpdump, wireshark and libpcap code
- Divert all packets from a specific network interface:
tcpdump -ni eth0 - Divert only a subset of flows: for example capture SMTP traffic using
tcpdump -n 'dst port 25' - Divert only a specific set of traffic that is needed for network accounting:
tcpdump -n 'dst port 25 and tcp[1] & 2 != 0' - Minimize the impact on the performance of non-tapped traffic
BPF programs
In order to avoid overloading userland, tcpdump and libpcap push some filter rules into the kernel. For instance:
# tcpdump –n ‘dst port 25’
leads to the following filters:
# tcpdump -d -n 'dst port 25'
(000) ldh [12]
(001) jeq #0x86dd jt 2 jf 8
(002) ldb [20]
(003) jeq #0x84 jt 6 jf 4
(004) jeq #0x6 jt 6 jf 5
(005) jeq #0x11 jt 6 jf 19
(006) ldh [56]
(007) jeq #0x19 jt 18 jf 19
(008) jeq #0x800 jt 9 jf 19
(009) ldb [23]
(010) jeq #0x84 jt 13 jf 11
(011) jeq #0x6 jt 13 jf 12
(012) jeq #0x11 jt 13 jf 19
(013) ldh [20]
(014) jset #0x1fff jt 19 jf 15
(015) ldxb 4*([14]&0xf)
(016) ldh [x + 16]
(017) jeq #0x19 jt 18 jf 19
(018) ret #96
(019) ret #0
This assembly language is parsed for every packet passing into the kernel so tcpdump will only get SMTP traffic. It is named a BPF (Berkley Packet Filter, http://en.wikipedia.org/wiki/Berkeley_Packet_Filter) program.
If you are interested in tcpdump filter syntax, you should start with: http://www.safepatrolsolutions.com/papers/Tcpdump.pdf
Implementation of BPF programs in the Fast Path
Let’s take this example and analyse how it can be handled by the Fast Path:
* Inbound SYN Filter. This filter monitors any inbound SYN connections to ports that you are not expecting traffic on:
# tcpdump "tcp and (tcp[13] & 0x02 != 0) and
(tcp[13] & 0x10 = 0)and (not dst port 53) and
(not dst port 80)
and (not dst port 25) and (not dst port 21)"
then, add –d argument, you can see how powerful the BPF instructions can be into the dataplane for a complex rule which is described into this PDF:
# tcpdump -d "tcp and (tcp[13] & 0x02 != 0) and
(tcp[13] & 0x10 = 0)and (not dst port 53) and
(not dst port 80)
and (not dst port 25) and (not dst port 21)"
(000) ldh [12]
(001) jeq #0x86dd jt 17 jf 2
(002) jeq #0x800 jt 3 jf 17
(003) ldb [23]
(004) jeq #0x6 jt 5 jf 17
(005) ldh [20]
(006) jset #0x1fff jt 17 jf 7
(007) ldxb 4*([14]&0xf)
(008) ldb [x + 27]
(009) jset #0x2 jt 10 jf 17
(010) jset #0x10 jt 17 jf 11
(011) ldh [x + 16]
(012) jeq #0x35 jt 17 jf 13
(013) jeq #0x50 jt 17 jf 14
(014) jeq #0x19 jt 17 jf 15
(015) jeq #0x15 jt 17 jf 16
(016) ret #96
(017) ret #0
The Fast Path must implement the same BPF program in order to divert only SMTP traffic. It means that the Fast Path is responsible for two things in flow inspection:
- Parse packets according to BPF programs coming from the Control Plane
- Divert packets which match the BPF programs
High speed packet capture or high speed sniffer tools may look easy to implement, but this is not as easy as copying packets. Maintaining backward compatibility and supporting BPF program are requirements for the design of the Fast Path.
As a next step, once you get your BPF-ready Fast Path, you can reuse pcap applications from this long list: http://www.stearns.org/doc/pcap-apps.html
More information about 6WINDGate architecture can be found here.
You can download more detailed documents here.
You can check 6WINDGate FAQ here.
Recent Comments