HTTP/2 with NGINX & Wireshark in Ubuntu (Part 3)

Part 3: Wireshark install and HTTP/2 traffic capture

This article continues the HTTP/2 server push series by describing how to capture and decrypt HTTP/2 traffic with Wireshark. Since HTTP/2 requires TLS (HTTPS) we need to be able to decrypt any captured traffic before it can be inspected. This article describes how to do this with Firefox Developer and Wireshark. I hope you enjoy it.

Previously…

Check out these previous articles which describe how to configure NGINX web server for HTTP/2 server push in a docker container.

Set an environment variable

First, set a new environment variable SSLKEYLOGFILE with a path to where the SSL logs will be stored.

  1. Open a terminal and enter the below command. I’ve used $HOME to reference my home directory (/home/codeooze/). Ensure to use a valid path for your Ubuntu instance.

    export SSLKEYLOGFILE=$HOME/sslmaster.txt
    
  2. You can check the variable is set correctly by using the export command and checking the value of SSLKEYLOGFILE is set as expected.

  3. Note that this variable will only exist in your current session. So if you log out or restart Ubuntu, you’ll need to set it again.

Install Firefox Developer

Firefox Developer can be used to capture the SSL key used for encryption between the client (browser) and the web server.

  1. In Ubuntu, open Firefox and navigate to https://www.mozilla.org/en-US/firefox/developer/ and download Firefox Developer.

    The version used in this article is 66.0b6 (64-bit) which is the latest at the time of writing.

  2. Extract the archive file.

  3. You can launch Firefox Developer from a terminal by entering the below command. Ensure the path matches the location where you extracted the archive file.

    Downloads/firefox-66.0b6/firefox/firefox &
    
  4. Firefox should automatically create the sslmaster.txt when launched. Open a terminal and change into the location where the sslmaster.txt file is expected (e.g. /home/codeooze/) and verify it has been created. If not, try closing Firefox Developer and re-opening it.

Install and configure Wireshark

Wireshark is a popular network traffic analyzer that we’ll use to capture and inspect HTTP/2 traffic.

  1. Open a new terminal window and enter the following command:

    sudo apt-get install wireshark
    
  2. As part of the install process you will be prompted to decide if non-super users should be allowed to capture packets.

    If you are security conscious you should probably select No as recommended by the prompt.

    If, however, you are performing these steps in a virtual machine created specifically for this exercise, you can probably select Yes and move on.

    I have selected No.

    wireshark install prompt

  3. The Wireshark installation will proceed. When finished, launch Wireshark by entering:

    sudo wireshark &
    
  4. Now you need to point Wireshark to the sslmaster.txt file so it can use the captured SSL key to decrypt traffic between the browser and NGINX web server.

    In Wireshark, from the top menu select Edit > Preferences

  5. In the left pane expand Protocols, then scroll down and select SSL.

    Note: There are a lot of protocols listed, but thankfully they are in alphabetic order.

    wireshark ssl preferences

  6. Beside the (Pre)-Master-Secret log filename field click Browse and select the sslmaster.txt file

    For example, /home/codeooze/sslmaster.txt

    Click OK to save the changes.

    wireshark ssl log file

Capture HTTP/2 traffic

Now we’re ready to capture some HTTP/2 traffic.

  1. If the NGINX docker container is not already running, start it now

    sudo docker run -dit --name nginx-http -p 80:80 -p 443:443 nginx-http2
    
  2. In Wireshark, from the top menu select Capture > Start (or alternatively hit Ctrl + E) to start the traffic capture.

    Note: If you have multiple network interfaces you may need to ensure the correct one is selected for the capture. I’ve used the docker0 interface which is the network interface for the docker container running NGINX, however it will also work on the loopback interface.

    wireshark start capture

  3. In Firefox Developer navigate to https://http2.localhost/home.html

  4. In Wireshark, stop the capture using Capture > Stop (or alternatively hit Ctrl + E again).

    You should see some captured HTTP/2 packets. If not, try clearing the history and cache in Firefox, start the capture again, then refresh the website.

    Note: You can filter the captured packets to show HTTP2 packets by typing http2 in the filter field and hitting Enter.

    wireshark http2 filter

  5. The traffic will already be decrypted. To see what the encrypted traffic looks like, select Edit > Preferences > Protocols > SSL and clear (Pre)-Master-Secret log filename. Also clear http2 in the filter.

    The encrypted data will show the protocol as TLSv1.2 and the Info column will just show Application Data.

    You can confirm these are HTTP/2 packets by comparing the packet numbers. For example, in the above decrypted packet screenshot we have packet numbers 10, 12, 13, 14 and 15 showing as HTTP/2 when decrypted. These same packet numbers are highlighted in the encrypted capture below and show as Application Data:

    wireshark http2 encrypted traffic

  6. Set (Pre)-Master-Secret log filename again so the traffic is decrypted and filter for http2 packets.

  7. Examine the packet with the GET /home.html request (packet 14 in the screenshots). This is the browser’s GET request for home.html.

    Note: your packet numbers will likely be different, so use the screenshots to help guide you.

    In packet 15 the server has replied with the home.html resource, along with a PUSH_PROMISE for each of /bootstrap.css, /bootstrap.js, /img1.jpg, and /img2.jpg.

    Following are several DATA packets which represent the pushed files. These packets demonstrate HTTP/2 server push and multiplexing.

    wireshark http2 packets

HTTP/2 packet analysis

In HTTP/2, each request and response is assigned a stream id. A single packet can then be split into multiple frames, allowing data from more than one resource to be sent in a single packet. This also allows a single resource to be spread across multiple packets. On the client side, all the frames with the same stream id are collected together to re-form the resource (e.g. /bootstrap.css).

Examine the packet with PUSH_PROMISE[x] (packet 15 in the screenshots) by selecting it in the upper pane. (Note: your packet numbers and stream IDs will likely be different, so use the screenshots to help guide you.) In the middle pane, expand HyperText Transfer Protocol 2 and locate this line:

Stream: PUSH_PROMISE, Stream ID: 15, Length 114, GET /bootstrap.css

wireshark http2 promises

This is the PUSH_PROMISE for /bootstrap.css. It means the web server is telling the client it will push this resource without the client needing to request it separately. Stream ID: 15 is the current Stream ID of this frame containing the promises. You can also see the stream ID value in the packet list, e.g PUSH_PROMISE[15] has stream ID 15 (it also happens to be packet 15).

Look down a little further and you will see Promised-Stream-ID: 2 (highlighted in the above screenshot). This is the future Stream ID that will be used when the server eventually sends frames containing the /bootstrap.css resource. This tells the client where to look for /bootstrap.css in future packets. You can easily identify the later packets using Stream ID 2 in the packet list, for example DATA[2]:

stream id in packet list

Keep scrolling through this packet and you will see this repeated for each of the resources the web server has promised to push to the client (/bootstrap.js, /img1.jpg, and /img2.jpg). They will all have a unique Promised-Stream-ID value.

Scroll down even further and you will find this packet also contains the actual /home.html resource requested by the client.

wireshark http2 encrypted traffic

Finally, select one of the DATA packets below the PUSH_PROMISE[x] packet (packet 15 in the screenshots). I’ve selected packet 20 which is DATA[2]. In the middle pane, expand HyperText Transfer Protocol 2 and examine it. This packet contains a frame from stream id 2 which we know is /bootstrap.css. This is corroborated by the line Header: context-type: text/css. This file is spread across many frames in many packets, all with stream id 2.

Wireshark also tells us that the fully assembled /bootstrap.css resource can be found in packet 73 later in the capture.

wireshark http2 encrypted traffic

Conclusion

I have provided a very brief analysis of the captured HTTP/2 traffic. I encourage you to explore your packet capture further to help understand HTTP/2. There are many excellent resources detailing how the protocol works, such as the Google Developers Web Fundamentals article Introduction to HTTP/2. Wireshark is a very powerful tool and I encourage you to learn more about it as well.

I hope you have enjoyed this series of articles and have learned something new along the way.