PUBLIC OBJECT

The Last HttpURLConnection

Published on

An awkward API

OkHttp 1.0 started out as an optimized implementation of HttpURLConnection. This old API is awkward to implement because there is an implicit state machine that corresponds to the underlying network I/O.

GET / HTTP/1.1
Host: publicobject.com
Accept: text/html

HTTP/1.1 200 OK
Content-Length: 300

<html>
<head><title>Public Object</title></head>
...

For example, calling getResponseCode() forbids future calls to setRequestProperty() because request headers can’t be edited after they’ve been transmitted.

HttpURLConnection connection = urlFactory.open(url);
int responseCode = connection.getResponseCode();

// This fails with an exception at runtime!
connection.setRequestProperty("Accept", "text/html");

What’s safe to call is especially awkward because the state machine itself is dynamic: setFixedLengthStreamingMode() and setChunkedStreamingMode() cause request body writes to lock in request headers.

An uncomfortable implementation

With OkHttp 2.0 we introduced a new API to complement HttpURLConnection. Instead of an implicit state machine the new API uses types to make an explicit model: Request has the inputs, Response has the outputs, and Call does the action.

Request request = new Request.Builder()
    .url(url)
    .header("Accept", "text/html")
    .build();

Call call = client.newCall(request);

Response response = call.execute();
int responseCode = response.code();

With this API it’s easier to do the right thing.

But throughout OkHttp 2.x and current releases of OkHttp 3.x, HttpURLConnection’s state machine has permeated the codebase. This is because we’ve maintained the caller’s ability to manipulate an in-flight connection.

HttpsURLConnection connection = urlFactory.open(url);
connection.setDoOutput(true);

// With this line, OkHttp creates a live socket to the server but
// does not transmit any data to it.
connection.connect();

// The socket can be interrogated. But this could be wrong if
// the if the forthcoming response is a redirect.
Principal peerPrincipal = connection.getPeerPrincipal();

// This write will be buffered. If we were in a streaming mode
// this would also cause OkHttp to write the request headers.
connection.getOutputStream().write('a');

// This terminates the request body and consume the response
// headers. The response body is ready to be streamed.
int responseCode = connection.getResponseCode();

In order to handle any API call at any point of the HTTP lifecycle, OkHttp’s internals were ugly. There were several fields with similar-sounding names to track where we were and what should happen next. Combine this with failure recovery and retries and the whole codebase was difficult to understand.

A satisfying solution

With OkHttp 3.4.0-RC1, we’ve fought our way out of this mess. Our solution may seem obvious in hindsight but it wasn’t always!

We deleted most of our old HttpURLConnection code and reimplemented it on top of our Call API. For almost all of OkHttp’s users this is a simple indirection and everything just works.

But what about the awkward case above where the application can connect without sending data? For this case we cheat wildly: we do the initial connection setup on an asynchronous background thread. An interceptor holds that thread until the application asks it to proceed:

@Override public Response intercept(Chain chain) {
  synchronized (lock) {
    connectPending = false;
    proxy = chain.connection().route().proxy();
    handshake = chain.connection().handshake();
    lock.notifyAll();

    while (!proceed) {
      lock.wait(); // Wait until proceed() is called.
    }
  }

  return chain.proceed(chain.request());
}

The background thread makes it possible for two different threads to be active simultaneously:

  • The application thread can step through the implicit state machine by making method calls on HttpURLConnection.
  • OkHttp’s background thread can do HTTP without external interference.

Since making this change we’ve been able to simplify OkHttp’s internals substantially. The code is faster and easier to understand: the whole thing is just a stack of built-in interceptors.

HttpURLConnection’s boring future

With this change our ongoing cost of maintaining the HttpURLConnection API is nearly zero. There’s a fancy interceptor to do the connection handoff and some adapter boilerplate. I plan to extract okhttp-urlconnection into its own standalone project where it can quietly languish and be forgotten.

If you’re using OkHttp’s HttpURLConnection API, please try out OkHttp 3.4.0-RC1 and confirm that everything still works.

And if you’re already using the Call API you’ll like this release. It’s simpler, faster, and more streamlined than ever before.