author: Nathan Acks
date: 2022-08-11

I’m attending DEF CON for the first time this year. Not much is happening on Thursday (except for LineCon), but I am attending the “TCP Deep Dive with Wireshark” workshop in an attempt to improve my packet-analysis chops (so far I’ve only done a single “room” about packet analysis and Wireshark on TryHackMe).

TCP Deep Dive with Wireshark


What is TCP?

TCP = Transmission Control Protocol

It operates at OSI layer 4 and is an IEEE standard.

RFC793: “[A] [h]ighly reliable host-to-host protocol. [It] [p]rovides connection, reliability, and flow control.

TCP/IP was formalized in September 1981.

Demo: The TCP Handshake

Wireshark supports “profiles” (accessible in the lower right in the status bar) that are just collections of settings, filters, layouts, etc.

Wireshark allows filters to be saved for quick access; use the “+” button to the right of the filter bar. Drop-down menus can be created by separating menu components with // (for example, Menu//Submenu//Filter Name).

By default, the “Time” column is in seconds and starts at zero (0) for the first packet captured. PCAP files actually include the full packet timestamp, so the format of the “Time” column can be changed using “View > Time Display Format”.

If you click on a field for a packet, the filter name is displayed in the lower left. (For example, TCP SYN packets are tcp.flags.syn.) You can automatically apply a filter (or “prepare” a filter, which lets you see the filter before it’s applied) by right-clicking on it and choosing “Apply as Filter” (or “Prepare as Filter”).

Color rules are just filters; they’re matched in order (from top down), with the first matched coloring rule applying. Note that Wireshark is not really consistent about applying new coloring rules - it’s best to reload the file after creating/modifying rules.

TCP “conversations” (related packets) can be automatically filtered by right-clicking on a packet and selecting “Conversation Filter > TCP” (this matches all packets with the same source/destination IP/port). This really just automatically applies a filter.

TCP client source ports are random client ports chosen from the pool of available (high) ports. Destination ports are generally reserved.

Wireshark adds some additional packet information, designated as [Wireshark Data]. Some examples: “Stream index” is an index of the streams/conversations displayed in the current filter, starting from 0. “TCP Segment Len” is the packet data length (this is useful for figuring out weird things, like SYN packets with data). “Conversation completeness” tries to measure how much of the current conversation/stream is present in the packet capture (for TCP conversations, Wireshark expects a handshake, some data, and then a FIN or RST).

The reason the first packet of the three-way handshake is a SYN because it’s synchronizing sequence numbers. Note that each host in a TCP connection uses its own series of sequence numbers.

The first sequence number is random, and is called the ISN (“initial sequence number”).

The “acknowledgement number” is set by the receiving host, and is equal to the current sequence number for the requesting system + the number of bytes of data returned (or 1 if the segment length is zero - this is called a “ghost byte”).

The vast majority of pure SYN packets use 0 for their acknowledgement number - using non-zero acknowledgement numbers in the initial SYN packets and observing system behavior is one way that nmap fingerprints operating systems.

When you select an ACK packet in Wireshark, a checkmark will be displayed next to the packet that is being acknowledged.

The “initial round trip time” (IRTT) is the time taken for the initial SYN packet in the TCP handshake to the final ACK packet in the initial three-way handshake. Most TCP implementations will initially wait for up to 0.5 seconds until retransmitting a packet, but will dynamically adjust this to 3x - 4x the IRTT after the initial handshake. Wireshark will report this in the final ACK packet of the three-way handshake.

The TCP “Window” is the maximum number of bits that the sending system expects to receive from a request (it represents the current buffer size for that connection on that system). This was originally a 2-byte value, though this has since been extended using the “Window scale” option (which sets a multiplier for the “Window” size as as a power of two - such that a “Window scale” of 7 is a multiplier of 2⁷ = 128). Window scales can be up to 14, which allows (once multiplied with the maximum window size) up to 1 GB of data to be transmitted before an ACK on modern systems. Differences in how the window and window scale numbers changes in a conversation, and what they are set to initially, can be used to fingerprint TCP stacks and OSes.

It’s common to see a window scale of 2 for web servers (clients tend use larger scales).

The “Maximum segment size” (MSS) TCP option represents the maximum data segment size that a system can receive. This is different than the window size, which is the amount of data that a system expects before it gets an ACK (it’s basically the buffer for this connection).

Note that nmap SYN scans have a larger MSS than window size (TCP scans, as they use the operating system to construct packets, have a value for MSS that works in a saner fashion). This is one way that nmap scans can be recognized in packet scans.

The SACK (“selective acknowledgement”) TCP option allows for receiving systems to acknowledge packets as they’re received, rather than only at the end of a transmission; without SACK, only packets up to the first dropped packet are acknowledged, which can require more data retransmission if only a few packets are dropped. SACK is generally not supported by low-power IoT systems, as it requires more system resources to track every successfully received packet.


Recommended resource: “TCP/IP Illustrated, 2nd Edition”.

Lab 1: The TCP Handshake

Use “Statistics > Conversations” to get a quick overview of the conversations in a packet capture.

Use “Statistics > Capture File Properties” to see information about capture files (you can also include/save comments here!).

TCP “options” are set in the initial handshake (the initiating host will propose in the SYN packet, and the receiving host will reply with what it supports in the SYN/ACK packet). Note that each system sets its own window scale and MSS values, however.

The “no op” (NOP, 01) option is just a blank option value used to pad things out, since header size must be a power of two bytes but (1) options need to fall on a byte boundary and (2) we may not have enough options to fill out this space. How NOPs are used is highly implementation-dependent. NOPs can also be used by middle-boxes (firewalls, routers, etc.) to strip options.

The window scale and MSS options, despite being system-specific, can only be used if set on both ends of the conversation!

TCP allows for header sizes up to 60 bytes, which caps the number of TCP options.

One motivation for QUIC is to work around the lack of header space. It’s thus better to think of QUIC as a replacement for TCP, rather than just a replacement for HTTP.

The amount of delay between the SYN and SYN/ACK packet will be very low if capturing server-side.

To see (suspected) retransmissions in Wireshark, filter for the tcp.analysis.retransmission flag. Some retransmissions may be “fast”, in that the packet is transmitted before the TCP timer would have expired. This happens when SACK is used, typically after a packet isn’t acknowledged in after three consecutive SACK blocks.

SACK numbers live in the TCP options block.

Clients typically initiate handshakes. What is the client IP?

What is the network roundtrip time between these two endpoints?

What port is the client trying to connect to on the server?

What TCP Options are being used?

What is the MSS size of the client? The server?

How many TCP retransmissions are in this pcap?

Lab 2: NMAP Port Scan

You can filter on a negation using !. (For example, !arp filters out ARP packets.)

To see SYN/ACK packets, filter with tcp.flags.syn == 1 && tcp.flags.ack == 1.

You can limit conversation statistics to the current filter view using the “Limit to display filter” checkbox.

Protocol names can be resolved in the Wireshark preferences by checking the “Name Resolution > Resolve transport names” option.

Wireshark can resolve IP addresses using “Name Resolution > Resolve network (IP) addresses”. It can pull from either DNS querier in the packet capture itself, or from the resolver itself. Using an external resolver can get pretty noisy though.

Filter for the very first TCP SYN in the pcap. Is there anything in this header that looks strange? Maybe not usual for a regular TCP SYN?

What ports is the attacker trying to scan?

What ports are open on the server?

Look at the first TCP SYN in the pcap. Look at the options and TCP window size. Does this look normal to you? Why or why not?

Nmap typically uses a window size of 1024.

Lab 3: OS Fingerprinting

A host can set a window scale of 0 (which multiplies the window by 1) to allow the other half of the TCP conversation to use a window scale without using one itself.

The order of the TCP options is also useful for OS fingerprinting.

Wireshark will also helpfully note some unusual TCP packet behaviors (such as the lack of an MSS option).