All right – now, we’re finally ready to get to the meat of what JSR 315 brings to the table – asynchronous servlets.
Prior to JSR 315, the world was a very simple and pleasant place. It was the world of the synchronous web. The server maintained a server socket that waited at the gates for weary incoming requests, and as each one arrived, assigned a worker to help it accomplish its needs. In this model, there was a finite set of workers (or server threads), and once all available workers were busy, new incoming requests had to wait at the gate until a worker completed processing its current request.
The process of setting up and tearing down socket connections is an expensive operation, and the process of TCP bandwidth throttling means that the earlier transmissions over a new connection are slower than later ones. As a result, HTTP 1.1 introduced its keep-alive connection model, which allowed a client to hold the connection open (i.e. to have a worker thread assigned) until all resources on a given page were retrieved.
This model works very well, and is both very performant and scalable, since requests come in at staggered intervals, allowing a small thread pool to successfully handle a large volume of client requests.
However, all was not well in the good old days.
First, was the issue of thread starvation. Each thread consumes server side resources – both memory and CPU resources. As a result, the number of threads within the pool must be constrained to some finite number. Furthermore, if the processing that the thread must perform for a client requires some blocking operation (say waiting on some slow external resource such as a database or web service), that thread is effectively unavailable to the server. In other words, it is possible that all threads in the pool are soon blocked and incoming requests can no longer be effectively handled. This is not just a theoretical problem. With the prevalence of fine grained Ajax-requests, and the adoption of SOA, the load on web servers is on an upward trend.
Second was the issue of staleness. At any given point, a client gets a view of the web application’s state at the instant that the request was procesed. Any changes to state between two consecutive requests from a client, are effectively invisible to that client. In other words, each client only gets a static snapshot view into what is effectively a dynamic system. And the currency of that view decays the longer the time period between requests. Ajax’s fine grained interactions, do mitigate this problem, but it does not completely solve the currency issue.
Third was the issue of non standard implementations. The developers of every servlet container decided to solve this problem of the synchronous web by implementing their own custom solutions. Tomcat’s Comet, WebLogic’s FutureResponseServlet, and WebSphere’s Asynchronous Request Dispatcher are examples of developers not wanting to wait for the standards body to catch up.
A key piece of JSR 315 therefore was to solve these issues with the synchronous web, by introducing us to the asynchronous web.
The problem of thread starvation is solved by having the main request processing worker thread delegate processing to an asynchronous thread, and return to the thread pool. In other words, if processing will require blocking operations, the worker can put the request processing context on ice, and hand it off to an asynchronous thread for processing. It can then return to the thread pool and wait for the next incoming request.
The problem of staleness is solved by allowing the client to make preemptive requests. I.e., just as soon as the client receives a snapshot, it can turn around and immediately make a preemptive request for the next snapshot. This request can be handed off to an asynchronous processor on the server, which will be notified when a state change occurs, and can immediately return a response to the waiting client. As far as the client is concerned, this looks just like any other long running request. Of course, with the advantage that there’s hardly a lag between a server side state change, and a client notification.
Finally, this solves the standardization issue by codifying asynchronous behavior into the servlet specification. This is now the one approach to rule them all!
So what do we need to implement asynchronous servlets?
- Thread Pool and Queue
If you haven’t read my previous blog post on thread pools in Java, this is the time to go do so. An Executor and ExecutorService provide us with a mechanism to implement our pool of asynchronous workers. - AsyncContext
The context of the request must be stored for later use by the asynchronous worker thread. This is achieved using an AsyncContext instance, which holds the request and response that was provided by the container to the original servlet. - Runnable instance
This instance encapsulates the actual processing of the blocking operation on the asynchronous thread. - Servlet/Filter chain
All components that will participate in an asynchronous request must be marked as supporting asynchronous operation.
Let’s walk through the recipe of creating an asynchronous servlet:
- At context startup, a context listener is used to set up an Executor.
- Servlets that might involve a blocking operation are identified and marked as asyncSupported (annotation or web.xml declaration). Any servlets/filters in the processing chain for this servlet are similarly marked.
- Within its service() method, the asynchronous servlet then:
- Invokes startAsync() on the request
- Wraps the returned AsyncContext in a Runnable
- Hands the Runnable off to an Executor for asynchronous processing
- Returns from the service() method. For an asynchronous servlet, returning from this method does not automatically commit the response. As a result, the client has no idea that it is being given the run around – it patiently waits for the response.
- The main thread returns to the server’s pool, ready to handle the next incoming request.
- The Executor assigns an async thread to process the Runnable.
- The Runnable:
- Uses the request and response stored in its AsyncContext
- Accesses any request attributes/parameters that are stored in its request to perform its processing
- Stores the final result of the processing as a request attribute
- “Completes” the response – either by generating the response by itself, or by dispatching to a server side resource.
- The client now displays the resource.
To the client, this interaction is no different from a slow server taking an inordinate amount of time to respond.
To the server, the server worker thread spent only a fraction of the time it would have normally taken to handle such a request, helping its throughput significantly.
The actual processing was handled by a thread in a separate pool – without impacting the main server pool.
The servlet context listener does nothing more than set up a ThreadPoolExecutor with 10 threads, and a request queue with a limit of 100 pending requests.
package com.swengsol.listeners;
/**
* User: Damodar Chetty
* Date: Jul 23, 2010
* Time: 10:48:14 PM
* An introduction to Servlet 3.0 and Java 6 (TC JUG)
* (c) Software Engineering Solutions, Inc.
*/
import javax.servlet.*;
import javax.servlet.annotation.WebListener;
import java.util.EnumSet;
import java.util.concurrent.Executor;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.ThreadPoolExecutor;
import java.util.concurrent.TimeUnit;
import java.util.logging.Logger;
@WebListener
public class MyServletContextListener implements ServletContextListener {
private static final Logger logger = Logger.getLogger("com.swengsol");
// Public constructor is required by servlet spec
public MyServletContextListener() {
}
// -------------------------------------------------------
// ServletContextListener implementation
// -------------------------------------------------------
public void contextInitialized(ServletContextEvent sce) {
logger.info("Context Listener > Initialized");
//2. Creation of a global async Executor
Executor executor =
new ThreadPoolExecutor(10, 10, 50000L,
TimeUnit.MILLISECONDS,
new LinkedBlockingQueue<Runnable>(100));
sce.getServletContext().setAttribute("executor", executor);
}
public void contextDestroyed(ServletContextEvent sce) {
}
}
The servlet is marked with an annotation to indicate that it supports async processing. Some key elements to note are:
1) The presence of the dispatch parameter on the query string will allow us to control how the async request completes (whether it is handled completely by the asynchronous processor or whether it is forwarded on to another server side resource).
2) The timeout parameter allows us to set a timeout for an asynchronous request. The request is aborted if it does not complete within the specified timeout.
3) The call to startAsync() builds up the AsyncContext, which represents the context of the current request.
4) An AsyncListener can be registered to be notified of events within the asynchronous process– such as timeouts.
5) The thread pool with our async worker threads is retrieved from the servlet context, and is given an instance of our Runnable to process – an AsyncRequestProcessor.
6) The main worker thread now returns without committing the response.
package com.swengsol.servlets;
import com.swengsol.AsyncRequestProcessor;
import com.swengsol.listeners.MyAsyncListener;
import javax.servlet.AsyncContext;
import javax.servlet.ServletException;
import javax.servlet.annotation.WebServlet;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.util.Date;
import java.util.concurrent.*;
import java.util.logging.Logger;
/**
* User: Monsty
* Date: Jul 23, 2010
* Time: 9:43:25 PM
* An introduction to Servlet 3.0 and Java 6 (TC JUG)
* (c) Software Engineering Solutions, Inc.
*/
@WebServlet(urlPatterns = "/AsyncServlet", asyncSupported=true)
public class AsyncServlet extends HttpServlet {
private static final Logger logger = Logger.getLogger("com.swengsol");
protected void doPost(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
doGet(request, response);
}
protected void doGet(HttpServletRequest request,
HttpServletResponse response)
throws ServletException, IOException {
logger.info("AsyncServlet: Processing request: " +
request.getParameter("id") + ". on thread: " +
Thread.currentThread().getId() + ":" +
Thread.currentThread().getName() + "[" +
new Date() + "]");
request.setAttribute("receivedAt", new Date());
boolean dispatch = false;
String dispatchParam = request.getParameter("dispatch");
if ("true".equalsIgnoreCase(dispatchParam)) dispatch = true;
boolean timeout = false;
String timeoutParam = request.getParameter("timeout");
if ("true".equalsIgnoreCase(timeoutParam)) timeout = true;
AsyncContext asyncCtx = request.startAsync();
asyncCtx.addListener(new MyAsyncListener());
if (timeout) {
asyncCtx.setTimeout(1000);
}
Executor executor =
(Executor)request.getServletContext().getAttribute("executor");
//delegate long running process to an "async" thread
executor.execute(new AsyncRequestProcessor(asyncCtx, dispatch));
logger.info("AsyncServlet: Returning after request: " +
request.getParameter("id") + ". on thread: " +
Thread.currentThread().getId() + ":" +
Thread.currentThread().getName()+ "[" +
new Date() + "]");
//Watch out for concurrency issues at this point
// between the "main" thread and the "async" thread
// main thread returns now -
// - returning from service() does not commit the response
// - so client still waits patiently
}
}
The nice thing about this is that the servlet itself is pretty formulaic – and shouldn’t change much from one async servlet to the next. But all the complexity of processing has to go somewhere – and that somewhere is the AsyncRequestProcessor.
Also note the comment at the bottom of the file that indicates that there is now a short window of time here when the request and response objects are no longer thread safe. Ideally, you shouldn’t be mucking around with the request and response objects once you’ve handed them off to the async processor.
The asynchronous request processor is a Runnable instance that needs access to the AsyncContext that we previously retrieved from the request. In our example, note that the long running blocking operation is encapsulated within the longRunningProcess() method.
package com.swengsol;
import javax.servlet.AsyncContext;
import javax.servlet.ServletRequest;
import java.io.PrintWriter;
import java.util.Date;
import java.util.Random;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* User: Damodar Chetty
* Date: Jul 23, 2010
* Time: 9:45:22 PM
* An introduction to Servlet 3.0 and Java 6 (TC JUG)
* (c) Software Engineering Solutions, Inc.
*/
public class AsyncRequestProcessor implements Runnable {
private static final Logger logger = Logger.getLogger("com.swengsol");
private AsyncContext asyncContext;
private boolean dispatch;
public AsyncRequestProcessor(AsyncContext asyncContext) {
this.asyncContext = asyncContext;
dispatch = true;
}
public AsyncRequestProcessor(AsyncContext asyncContext,
boolean dispatch) {
this(asyncContext);
this.dispatch = dispatch;
}
public void run() {
//do something asynchronously –
// wait for a resource to become available
// or for a long running computation to finish
String reqId = asyncContext.getRequest().getParameter("id");
if (null == reqId || reqId.length() == 0) reqId = "<unk>";
long id = Thread.currentThread().getId();
String threadName = Thread.currentThread().getName();
if (null == threadName || threadName.length() == 0)
threadName = "";
String threadId = id + ":" + threadName;
String result = longRunningProcess(reqId, threadId);
//Store the result of the computation in a "view helper"
ServletRequest req = asyncContext.getRequest();
if (null != req && req.isAsyncStarted())
req.setAttribute("result", result);
else
return;
//once done, dispatch back to the final destination
if (dispatch) {
asyncContext.dispatch("/results.jsp");
} else {
//alternatively handle it ourselves
String html = "Result of the process for request id: " +
reqId + "<br/> Started at:" +
asyncContext.getRequest().getAttribute("receivedAt") +
".<br/> Completed at:" + result +
"<br/> Called complete.";
Squawker squawker = new Squawker(html);
asyncContext.getResponse().setContentType("text/html");
try {
PrintWriter out = asyncContext.getResponse().getWriter();
out.println(squawker.getHTML());
} catch(Exception e) {
logger.log(Level.WARNING, e.getMessage(), e);
}
asyncContext.complete();
}
}
public String longRunningProcess(String reqId, String threadId) {
Random randomGenerator = new Random();
int randomInt = 5000 + randomGenerator.nextInt(10000);
logger.info("AsyncRequestProcessor: Begin long run process (" +
randomInt + " ms) for request: " +
reqId + ". on thread: " + threadId +
"[" + new Date() + "]");
try {
Thread.sleep(randomInt);
} catch (InterruptedException ie) {
logger.log(Level.WARNING, ie.getMessage(), ie);
}
logger.info("AsyncRequestProcessor: Done processing request: " +
reqId + ". on thread: " +
threadId + "[" + new Date() + "]");
return new Date().toString();
}
}
In this example, the long running process simply sleeps for a random amount of time to simulate the randomness of accessing a remote resource. It takes anywhere from 5 to 15 seconds to complete its work – an eternity on the web. You can also imagine report generation processes that might take much longer.
Once the long running process has returned, we either dispatch the request to be handled by a server side resource (results.jsp), or we handle it ourselves by handcrafting the HTML to be returned. In the latter case, we have to explicitly call complete() on the AsyncContext instance. In the former, the container does so implicitly.
The server side resource that we forward our asynchronous request to is results.jsp. This file demonstrates how request attributes added by the async processor can be retrieved and used.
<%--
* User: Damodar Chetty
* Date: Jul 23, 2010
* Time: 9:45:22 PM
* An introduction to Servlet 3.0 and Java 6 (TC JUG)
* (c) Software Engineering Solutions, Inc.
--%>
<%@ page contentType="text/html;charset=UTF-8" language="java" %>
<html>
<head>
<title>Introducing Servlet 3.0</title>
<link rel=stylesheet type="text/css" href="styles.css">
</head>
<body>
<p class="mainpara">
results.jsp:
completed processing of long running activity with id:
<span style="color:#ff3300">
<%=request.getParameter("id")%>
</span>
<br/>
received at: <%=request.getAttribute("receivedAt")%>: <br/>
completed at: <%=request.getAttribute("result")%>
</p>
<p class="copyright">Damodar Chetty,
Software Engineering Solutions, Inc.</p>
</body>
</html>
My sample code running on GlassFish v3 did not run until I added an XML declaration for my async servlet. To me this looks like a bug in the RI.
web.xml
<?xml version="1.0" encoding="UTF-8"?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://java.sun.com/xml/ns/javaee
http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
version="3.0">
<servlet>
<servlet-name>AsyncServlet</servlet-name>
<servlet-class>com.swengsol.servlets.AsyncServlet</servlet-class>
<async-supported>true</async-supported>
</servlet>
</web-app>
The final class we will visit is our AsyncListener which is rather trivial in my sample implementation. If you check your GlassFish logs, you’ll notice entries for the async listener which indicate when the async processing was completed.
package com.swengsol.listeners;
/**
* User: Damodar Chetty
* Date: Jul 24, 2010
* Time: 2:36:08 PM
* An introduction to Servlet 3.0 and Java 6 (TC JUG)
* (c) Software Engineering Solutions, Inc.
*/
import javax.servlet.AsyncEvent;
import javax.servlet.AsyncListener;
import java.util.logging.Logger;
public class MyAsyncListener implements AsyncListener {
private static final Logger logger =
Logger.getLogger("com.swengsol");
// Public constructor is required by servlet spec
public MyAsyncListener() { }
public void onComplete(AsyncEvent ae) {
logger.info("AsyncListener: onComplete for request: " +
ae.getAsyncContext().getRequest().getParameter("id"));
}
public void onTimeout(AsyncEvent ae) {
logger.info("AsyncListener: onTimeout for request: " +
ae.getAsyncContext().getRequest().getParameter("id"));
}
public void onError(AsyncEvent ae) {
logger.info("AsyncListener: onError for request: " +
ae.getAsyncContext().getRequest().getParameter("id"));
}
public void onStartAsync(AsyncEvent ae) {
logger.info("AsyncListener: onStartAsync");
}
}
This servlet can be invoked using the URL:
http://localhost:8008/jsr315/AsyncServlet?id=1&dispatch=true&timeout=false
where, jsr315 is my context path, /AsyncServlet is the URL pattern for my servlet, and the dispatch and timeout query string parameters can be used to test out alternative behaviors.
That’s all folks!
