Atom Feed SITE FEED   ADD TO GOOGLE READER

Speed up REST with HTTP Pipelining

Lots of apps download data over HTTP. It's what we use for the Issues Browser demo, and our up-and-coming Amazon Browser demo. In the simple web service paradigm REST, HTTP is the transport layer. HTTP is not just for web browsers!

If you're writing an app that uses HTTP, you may have some code where you make multiple requests to the same server. Some examples:
  • grabbing 10 images from Flickr's web service
  • posting 2 events to Google Calendar
  • updating a dozen business objects in your Amazon S3 shared storage

    Multiple requests can be made in multiple ways:
  • One connection for each request, maing a new connection when the previous connection completes
  • One connection for each request, making all connections in parallel
  • One connection for all requests, sending a new request when the previous response completes
  • One connection for all requests, sending all requests upfront

    Option 4 is called "pipelining" and is quite possibly the fastest way to do batch HTTP requests. Unfortunately, support for HTTP pipelining is desperately lacking among Java HTTP client libraries! According to the Oakland Software comparison:
  • J2SE 1.6 No.
  • Jakarta Commons Nope
  • Oakland Software Nopers!
  • innovation.ch Yes!

    Although the innovation.ch HTTPClient is a bit stale (no update since 2001), it implements HTTP pipelining properly. It's also LGPL, so if you find yourself wanting to freshen things up a bit, you can. Here's the basics to pipeline three files:
      HTTPConnection httpConnection = new HTTPConnection("swank.ca");
    HTTPResponse indexResponse = httpConnection.Get("/index.html");
    HTTPResponse lensesResponse = httpConnection.Get("/slacker/notopaque/lenses.png");
    HTTPResponse nintendodsResponse = httpConnection.Get("/slacker/notopaque/nintendods.png");

    InputStream indexStream = indexResponse.getInputStream();
    // code that reads indexStream ...
    InputStream lensesStream = lensesResponse.getInputStream();
    // code that reads lensesStream ...
    InputStream nintendoDsStream = nintendoDsResponse.getInputStream();
    // code that reads nintendoDsStream ...


    I strongly recommend you try adding HTTP pipelining to your app - you just might get the speed boost you're looking for!
  • i do not think it's entirely true, you can enable pipelining with HTTPClient too:
    http://www.javaworld.com/javaworld/jw-03-2008/jw-03-asynchhttp.html
    please ignore my previous statement, it's in alpha only.