Breaking down HTTP response at Packet Level [Wireshark Tutorial]

We will cover packet level inspection protocol by protocol in this article. To capture all network activity, Wireshark must be started to listen our network interface during computer booting process and continue to capture packets until we open our web browser and create a http request. As soon as we get a http response back from the server, we will stop the capturing.

Packet capturing methods

There are multiple ways to capture network activity during computer booting process. We can name some of them.

1. Installing a virtual machine and capturing packets from your main computer ( using Vmware Workstation or Virtual Box is one of the most common method. It is up to you what to choose.)
2. Mirroring your network with help of your switch, in other saying configuring a SPAN port.
3. Using a TAP device.

We will go with the first option. Linux distros are less noisy compared to Windows in terms of creating network activity during booting process, so I will pick Kali distro. I assume that the virtual machine installation is complete. We will take it from there. The steps are below.

1) Open Wireshark on your main computer and start to capture packets. If you do not want to see packets belong to the other communications (some broadcasts or multicasts) , you can use a display filter to select the packets you are interested in.

2) Power on your virtual machine like below.

3) Open your web browser in the virtual machine and visit any web site you want. I will visit "http://www.example.com". Once the page loads, we can stop packet capturing.

3) I captured 20 packets after applying a display filter based on my MAC and IP address. See figure below for what I captured.

4) We can categorize the packets into 5 types: dhcp, arp, dns, tcp and http packets.

Obtaining IP from dhcp server

1) When the virtual machine boots up, it needs an IP address for network communication and broadcasts a dhcp discover packet with destination IP and MAC of 255.255.255.255. Information carried in a dhcp discover message includes client’s MAC address, hostname, parameter request list which is filled by dhcp server in an offer packet with information like gateway (Router) IP address, dns server IP and subnet mask. As seen in the figure below, since my virtual machine wants to retain its old IP, it also adds "Requested IP Option (192.168.1.16)" into the packet. you can see more details below.

2) When the dhcp server receives the discover packet, it processes the request and responses back to the virtual machine with the offer packet, in short saying "Hey client, do you want this IP address with these options". See the details below for my capture.

3) The virtual machine broadcasts a dhcp request packet, saying "Hey server, I accept what you have sent me".

4) The dhcp server answers with a dhcp ack packet which lets the virtual machine know that the IP address is registered for it during period of leasing time. The whole process is summarized below.

Resolving DNS server MAC address using ARP (Address Resolution Protocol)

When I type www.example.com in the address bar and press enter button, my browser notices that it needs to convert www.example.com into IP address so it will create a dns query packet, asking the sever to translate the domain name into IP address, but the client (virtual machine) does not know dns server MAC address (it learns IP address of dns server from dhcp server during obtaining an IP address). The client broadcasts an ARP request and asks “Who has this IP address, please send me your MAC address”. The dns server receives the ARP packet and prepares a response packet that contains its own MAC address. See the details below.

The summarized process is below.

Now, the virtual machine has dns server MAC and IP and can create a dns query to ask the server to translate the domain name into an IP address.

Resolving domain name into IP

DNS (Domain Name System) service is used to translate a domain name into an IP address. Our web browser creates two dns queries for both ipv4 and ipv6. When a client asks for an IPv4 address resolution, it sets query type as “A”, whereas type is in form of “AAAA” when the client wants an ipv6 address resolution. The figure below shows the query packets and their responses from our capture.

Dns flows are below.

TCP (Transmission Control Protocol) 3 way handshaking

Once our web browser resolves the domain name into the IP address with help of the dns server, a Tcp connection is started. TCP is a connection oriented protocol and uses 3 way handshaking to create a reliable connection before both part exchange data. First 3 packets are called 3 way handshaking during which client and server synchronize and negotiate some connection parameters.

1) As seen below, we send a syn packet that includes starting sequence number, syn flag, window size of 64240 and some options like maximum segment size.

2) www.example.com server sends back a response packet with syn/ack flags set.

3) We process the packet comes from the server and response back with a packet in which ack flag is set, which means we are ready to start exchanging the data. It is summarized below.

Creating http (hyper text transfer protocol) packets

Once reliable tcp 3 way handshake finishes, our browser creates a http request as below.

In the http request packet, we can see it uses http version 1.1 which is a text based protocol. If we wanted to create this request manually, it would have been like below.

GET / HTTP/1.1
Host: www.example.com
User-Agent: curl/7.74.0
Accept: */*

It simply says “I am using http 1.1 version and my request method is GET”, specifying destination host as www.example.com and client application as curl. As soon as the server receives the request, it prepares a response packet with response code of 200 OK which indicates that the request was successfully received, understood, and accepted. See figure below for our capture.

We can see the response packet not only contains html source code but also some headers which indicate if client should cache the content, until when cached content should be kept valid and so on. So far the browser created a http request and got a http response, now it is time for the browser to close the connection. Since the transport protocol is tcp, there is a special procedure to close the connection. First it acknowledges the server that it got the http response then it sends a packet with fin and ack flag set which tells the server “I finish the connection”. The server responses back with fin and ack flag set, saying “ we can finish”. The browser sends back a response packet that indicates it has got the server packet with fin flag set. The figure below shows closing procedure for the tcp connection.

Conclusion

When I started my career in networking, it was not easy to understand OSI (Open System Interconnect) model due to everything on the paper was in theory until I discovered Wireshark which let me dive into packets and protocols world. Since I was interacting with something concrete, it gave me a boost in learning not only networking but also troubleshooting as well as network forensic. With understanding how common protocols work, I do not worry about which vendor’s switch, router or firewall is used in the infrastructure. As long as you know the protocols, the rest is just a bunch of commands to configure the devices.

He is proficient in System Administration, Python, Computer Network, Network Engineering, PHP, Web Testing, Penetration Testing, Wireshark, RADIUS, Cisco Router, TCP/IP, Kali Linux, OSPF, NPS, and Multiprotocol BGP. You can connect with him on his LinkedIn Profile.

Can't find what you're searching for? Let us assist you.

Enter your query below, and we'll provide instant results tailored to your needs.

If my articles on GoLinuxCloud has helped you, kindly consider buying me a coffee as a token of appreciation.

For any other feedbacks or questions you can send mail to admin@golinuxcloud.com

Thank You for your support!!

1 thought on “Breaking down HTTP response at Packet Level [Wireshark Tutorial]”

I was hoping to learn from this article how each section corresponds to a display filter in Wireshark. This entire tutorial series is impressive. Thank you. Reply

Leave a Comment Cancel reply

Wireshark Tutorial

• Getting Started with Wireshark
  • How to Use Wireshark
  • Wireshark IP Filters [Cheat Sheet]
  • Learn TCP Seq and Ack Numbers
  • Create Wireshark Configuration Profiles
  • Break down HTTP response at Packet Level
  • Analyze LDAP traffic
  • Analyse TCP Receive Window
  • Analyse TCP Zero Window
  • TCP Retransmission Analysis
  • Analyze IPsec Traffic with Wireshark
  • Analyze TLS and mTLS Authentication
  • Analyze TLS Failures using Wireshark
  • Log4j2 Vulnerability Analysis
  • Kerberos Authentication Packet Analysis
  • Malicious Resource Detection
  • Detect Rogue DHCP Server
  • Find Duplicate IP Address in Network
  • Troubleshoot Packet Fragmentation Issues
  • Troubleshoot with TTL
  • Troubleshoot Common TFTP Errors
  • Check if ISP is blocking website
  • FreeRADIUS: LDAP Authentication and Authorization
  • FreeRADIUS: PAP & CHAP Authentication
  • FreeRADIUS: Integrate with Active Directory