Hey,

some days ago HTTP2 server push has been added to Nginx (at least the open source version).

That’s great news since this was one of the most interesting features from HTTP2 that Nginx was lacking.

Given that I didn’t use server push myself so far, I decided to learn a bit more about it and give a try to Nginx' implementation.

Example of an architecture using NGINX HTTP2 Server Push acting as a reverse proxy

A minimal HTTP2 Server push example in Go

To establish a base around knowing whether the HTTP2 push functionality is indeed working or not, we can implement quick example in Go.

Following the example of Yoshiki, the server serves two endpoints:

/               -->     index.html   ( <img src="/image.svg" />
/image.svg      -->     image.svg itself

That way we can see whether Chrome would retrieve the image pushed by the / handler when we request /.

I started tailoring main.go:

// handleImage is the handler for serving `/image.svg`.
//
// It does nothing more than taking the byte array that
// defines our SVG image and sending it downstream.
func handleImage(w http.ResponseWriter, r *http.Request) {
	var err error

	w.Header().Set("Content-Type", "image/svg+xml")
	_, err = w.Write(assets.Image)
	must(err)
}

// main provides the main execution of our server.
//
// It makes sure that we're providing the required
// flags: cert and key.
//
// These two flags are extremely important because
// browsers will only communicate via HTTP2 if we
// serve the content via HTTPS, meaning that we must
// be able to properly terminate TLS connections, thus,
// need a private key and a certificate.
func main() {
	flag.Parse()

	if *key == "" {
		fmt.Println("flag: key must be specified")
		os.Exit(1)
	}

	if *cert == "" {
		fmt.Println("flag: cert must be specified")
		os.Exit(1)
	}

	http.HandleFunc("/", handleIndex)
	http.HandleFunc("/image.svg", handleImage)

	must(http.ListenAndServeTLS(":"+strconv.Itoa(*port), *cert, *key, nil))
}

Nothing fancy there, the big deal comes next: the handler for index.html.

This handler is the one that knows about what are the assets the index.html have (in our example, image.svg) such that it can tell the browser to start fetching them ahead of time.

With Go1.8+ we do that obtaining an http.Pusher by casting the writer supplied to our handler and then using the Push method:

// handleIndex is the handler for serving `/`.
//
// It first checks if it's possible to push contents via
// the connection. If so, then it pushes `/image.svg` such
// that at the same moment  that the browser is fetching
// `index.html` it can also start retrieving `image.svg`
// (even before it knows about the existence in the html).
func handleIndex(w http.ResponseWriter, r *http.Request) {
	var err error

	pusher, ok := w.(http.Pusher)
	if ok {
		must(pusher.Push("/image.svg", nil))
	}

	w.Header().Add("Content-Type", "text/html")
	_, err = w.Write(assets.Index)
	must(err)
}

Compile it and let it run.

Verifying HTTP2 Server Push with Google Chrome

Running the application on port 443 (requires some privileges), head to Chrome at https://localhost, open the Networks tab in the developer tools and see the push indication:

Chrome indicating that our image has been retrieved via HTTP2 Server Push

ps.: the Not Secure indication is not a big deal - the browser doesn’t trust the certificate we provided. If you wan’t to see it green you can either get a real certificate (LetsEncrypt is very handy for this if you have a domain and a webserver on such domain that you can use to retrieve a cert from them) or, in Mac, use Keychain to trust your certificate.

To get the details around how that push got initiated, we can use the chrome://net-internals page to debug the HTTP2 connection that we got established:

Chrome network internals showing the http2 server push promise receive event

Note that we’re not pushing contents downstream right from the index.html handler, but instead in the push functionality we just signalize that the browser should perform another request.

ps.: From the Go’s side we can’t consume those push frames yet: see https://github.com/golang/go/issues/18594 and PR https://go-review.googlesource.com/#/c/net/+/85577/.

Inspecting the HTTP2 streams using Wireshark

Aside from using chrome://net-internals, we can use Wireshark.

Chrome, Firefox and even Go are able to produce NSS-formatted key log files that gives the necessary secrets to Wireshark so it can decrypt the streams and interpret them.

Using either Chrome of Firefox you can get the file written to our filesystem by initializing them with the environment variable ``SSLKEYLOGFILE` that indicates where this file should live.

For instance, using MacOS:

SSLKEYLOGFILE=~/tlskey.log \
        /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome &

Once the browser is running, open Wireshark, go to preferences, then protocols and then finally reach the SSL configurations. There you set the (Pre)-Master-Secret log filename to the name you put on SSLKEYLOGFILE environment variable:

Wireshark SSL configuration screen

Now head to https://localhost and see the HTTP2-filtered packets flowing on the loopback interface:

Wireshark showing HTTP2 Server Push event captured

Done!

ps.: make sure you’re running the application on port 443. For some reason http2 filter won’t work on a different port. I guess that’s because the http2 filter is probably hardcoded to 443, but who knows?

Installing NGINX from source

I’m not very sure if there’s already a released version of NGINX with the latest commit from some days ago, so I decided to go with a build right from source.

Here I’m using Ubuntu Artful (17.10) on VirtualBox.

For that I set up a quick Vagrantfile:

Vagrant.configure(2) do |config|
  config.vm.hostname = "artful"
  config.vm.box = "ubuntu/artful64"
  config.vm.box_check_update = false
  config.vm.network "forwarded_port", guest: 443, host: 443

  config.vm.provider "virtualbox" do |v|
    v.memory = 2048
    v.cpus = 3
  end

  config.vm.synced_folder ".", "/vagrant", disabled: true
end

With the virtual machine up, I get there and start the actual thing process.

# Clone the git mirror (the official version control
# system used by nginx is mercurial)
git clone https://github.com/nginx/nginx


# Get into the cloned repository
cd ./nginx


# Run auto configuration to check for dependencies and
# prepare installation files.
#
# Make sure you run this command from the `nginx` directory
# instead of `./auto`. 
./auto/configure


# Here it should probably tell you that you that it 
# didn't find some dependencies (like `pcre` and others).
#
# Let's install them
sudo apt install -y \
        libpcre3 libpcre3-dev \
        zlib1g-dev \
        libssl-dev


# Configure our build with some modules.
# - http_v2_module      provides support for HTTP/2
# - http_ssl_module     provides the necessary support
#                       for dealing with TLS
./auto/configure \
        --with-http_v2_module \
        --with-http_ssl_module


# After running the command we should have a Makefile
# in the `nginx` directory (current working directory)
ls | grep Makefile
Makefile

Now that we have the Makefile ready, it’s a matter of triggering the build:

# Run the build process with a concurrency of 4
make -j4


# Check that `nginx` has been produced
./objs/nginx -v
nginx version: nginx/1.13.9

All set! Time to explore this new NGINX version.

Configuring NGINX with HTTP2

To get started with the most minimal configuration possible, I set up a static website configuration with HTTP2 support with server push configured for our root path:

# Do not daemonize - this makes it easier to test
# new configurations as any stupid error would be
# more easily caught when developing.
daemon off;

events {
        worker_connections              1024;
}

http {

        # Explicitly telling what mime types we're
        # supporting just for the sake of explicitiness
        types {
                image/svg+xml           svg svgz;
                text/html               html;
        }

        server {
                # Listen on port 8443 with http2 support on.
                listen                  8443 http2;


                # Enable TLS such that we can have proper HTTP2
                # support using browsers
                ssl on;
                ssl_certificate         certs/cert_example.com.pem;
                ssl_certificate_key     certs/key_example.com.pem;


                # For the root location (`index.html`) we perform
                # a server push of `/image.svg` when serving the
                # content to the end user.
                location / {
                        root            www;
                        http2_push      "/image.svg";
                }


                # When pushing the asset (`image.svg`) there's no need
                # to push additional resurces.
                location /image.svg {
                        root            www;
                }
        }
}

That works exactly as we wanted: if you head over / (that serves index.html) you get index.html and also the PUSH_PROMISE to retrieve image.svg:

Chrome network tab showing successful NGINX HTTP2 server push

Unsurprisingly, this is almost identical to the Go example we set up.

HTTP2 Server push using Nginx as a Proxy

As it’s very common to use NGINX as a reverse proxy, this new feature also supports such mode.

Although NGINX does not support HTTP2 backends, it can act as an HTTP2 frontend that translates requests back to origin servers as HTTP1.1, letting TLS termination and HTTP2 capabilities to itself.

The tricky part of server push is knowing what to push - something that a proxy doesn’t know.

To get over that, NGINX adhered to the spec of using the Link header as means of informing it what to push.

Let’s go back to Go to implement and HTTP1.1 server that informs NGINX when to push then.


// In the case of HTTP1.1 we make use of the `Link` header
// to indicate that the client (in our case, NGINX) should
// retrieve a certain URL.
//
// See more at https://www.w3.org/TR/preload/#server-push-http-2.
func handleIndex(w http.ResponseWriter, r *http.Request) {
	var err error

	if *http2 {
		pusher, ok := w.(http.Pusher)
		if ok {
			must(pusher.Push("/image.svg", nil))
		}
	} else {
                // This ends up taking the effect of a server push
                // when interacting directly with NGINX.
                //
                // Note that I'm returning `/proxy/image.svg` instead
                // of `/image.svg`. 
                //
                // This has the effect of NGINX taking this as a normal 
                // standard request and processing it through its location
                // pipeline (which ends up in the image Go handler we defined
                // here).
		w.Header().Add("Link", 
			"</proxy/image.svg>; rel=preload; as=image")
	}

	w.Header().Add("Content-Type", "text/html")
	_, err = w.Write(assets.Index)
	must(err)
}

In NGINX, not muched changed as well:

http {
 
         # Explicitly telling what mime types we're
@@ -16,6 +18,11 @@ http {
                 text/html               html;
         }
 
+        # Add an upstream server to proxy requests to
+        upstream sample-http1 {
+                server localhost:8080;
+        }
+
         server {
                 # Listen on port 8443 with http2 support on.
                 listen                  8443 http2;
@@ -27,6 +34,10 @@ http {
                 ssl_certificate         certs/cert_example.com.pem;
                 ssl_certificate_key     certs/key_example.com.pem;
 
+                # Enable support for using `Link` headers to indicate
+                # origin server push
+                http2_push_preload on;
+
 
                 # For the root location (`index.html`) we perform
                 # a server push of `/image.svg` when serving the
@@ -42,6 +53,15 @@ http {
                 location /image.svg {
                         root            www;
                 }
+
+                # Act as a reverse proxy for requests going to /proxy/*.
+                # Because we don't want to rewrite our endpoints in the
+                # Go app, rewrite the path such that `/proxy/lol` ends up
+                # as `/lol`.
+                location /proxy/ {
+                        rewrite         /proxy/(.*) /$1 break;
+                        proxy_pass      http://sample-http1;
+                }
         }
 }

I enabled http2_push_preload for the whole server and then made all request to /proxy/ be proxied to our upstream sample-http1 server.

The result? The following:

Chrome showing both HTTP2 server push from NGINX and the normal request for the sample image

At first that didn’t make much sense to me: two image.svg? What?

However, looking at the paths, it was clear: there was a request being made by the browser (/image.svg) and another coming from the server push (/proxy/image.svg). There was that extra one because I didn’t change the HTML, thus, Chrome was requesting it (as it should!).

If you’re curious about what goes on at the server host, this is a tcpdump of what goes on:

# Collect some packets within the host.
# -i: interface to snoop (loopback)
# -A: present body in ASCII format
# pos' args: only packets destined to 8080
sudo tcpdump \
        -i lo \
        -A \
        dst port 8080


# When requesting `/proxy/` nginx picks our
# proxy location and then recreates the request
# as an HTTP1.0 request to `sample-http1` upstream.
GET / HTTP/1.0
Host: sample-http1
Connection: close


# In consequence of the server-push the browser 
# performs the request (reusing the HTTP2 conenction)
# which ends up rewritten by NGINX as an HTTP1.0 request
# to our origin.
GET /image.svg HTTP/1.0
Host: sample-http1
Connection: close

Closing thoughts

It’s pretty cool to see that NGINX is integrating server push. While it doesn’t support HTTP2 backends, having server push at the proxy level is a very neat way of letting HTTP1 clients take advantage of this functionality.

I hope HAProxy comes next with an implementation that also makes use of the Link header (as that seems to be the standard).

If you have any questions or notice a mistake, please let me know!

All of the code cited here is aggregated in this repository: cirocosta/sample-nginx-http2. Feel free to open issues / PR if you find something to be improved.

I’m cirowrc on Twitter, by the way.

Have a good one!

finis