Request Pipeline

When a request reaches Lift there are a number of points where you can jump in an control what Lift does, or send back a different kind of response, or control access. This chapter looks at the pipeline through examples of different kinds of LiftResponse and configurations.

You can get a great overview of the pipeline, including diagrams, from the Lift pipeline Wiki page at http://www.assembla.com/spaces/liftweb/wiki/HTTP_Pipeline.

See https://github.com/LiftCookbook/cookbook_pipeline for the source code that accompanies this chapter.

Debugging a Request

Problem

You want to debug a request and see what’s arriving to your Lift application.

Solution

Add an onBeginServicing function in Boot.scala to log the request. For example:

LiftRules.onBeginServicing.append {
  case r => println("Received: "+r)
}

Discussion

The onBeginServicing call is called quite early in the Lift pipeline, before S is set up, and before Lift has the chance to 404 your request. The function signature it expects is Req ⇒ Unit. We’re just logging, but the functions could be used for other purposes.

If you want to select only certain paths, you can. For example, to track all requests starting /paypal:

LiftRules.onBeginServicing.append {
  case r @ Req("paypal" :: _), _, _) => println(r)
}

This pattern will match any request starting /paypal, and we’re ignoring the suffix on the request if any, and the type of request (e.g., GET, POST or so on).

There’s also LiftRules.early which is called before onBeginServicing. It expects a HTTPRequest ⇒ Unit function, so is a little lower-level than the Req used in onBeginServicing. However, it will be called by all requests that pass through Lift. For example, you could mark a request as something that the container should handle by itself:

LiftRules.liftRequest.append {
  case Req("robots" :: _, _, _) => false
}

Which this in place a request for robots.txt will be logged by LiftRules.early but won’t make it to any of the other methods described in this recipe.

If you need access to state (e.g., S), use earlyInStateful, which is based on a Box[Req] not a Req:

LiftRules.earlyInStateful.append {
  case Full(r) => // access S here
  case _ =>
}

It’s possible for your earlyInStateful function to be called twice. This will happen when a new session is being set up. You can prevent this by only matching on requests in a running Lift session:

LiftRules.earlyInStateful.append {
  case Full(r) if LiftRules.getLiftSession(r).running_? => // access S here
  case _ =>
}

Finally, there’s also earlyInStateless which like earlyInStateful works on a Box[Req] but in other respects is the same as onBeginServicing. It is triggered after early and before earlyInStateful.

As a summary, the functions described in this recipe are called in this order:

LiftRules.early
LiftRules.onBeginServicing
LiftRules.earlyInStateless
LiftRules.earlyInStateful

Running Code when Sessions are Created (or Destroyed)

Problem

You want to carry out actions when a session is created or destroyed.

Solution

Make use of the hooks in LiftSession. For example, in Boot.scala:

LiftSession.afterSessionCreate ::=
 ( (s:LiftSession, r:Req) => println("Session created") )

LiftSession.onBeginServicing ::=
 ( (s:LiftSession, r:Req) => println("Processing request") )

LiftSession.onShutdownSession ::=
 ( (s:LiftSession) => println("Session going away") )

If the request path has been marked as being stateless via LiftRules.statelessReqTest, the above example would only execute the onBeginServicing functions.

Discussion

The hooks in LiftSession allow you to insert code at various points in the session lifecycle: when the session is created, at the start of servicing the request, after servicing, when the session is about to shutdown, at shutdown… the pipeline diagrams mentioned at the start of this chapter are a useful guide to these stages.

Note that the Lift session is not the same as the HTTP Session. Lift bridges from the HTTP session to it’s own session management. This is described in some detail in Exploring Lift (see See Also).

The full list of session hooks is:

onSetupSession — this will be the first hook called when a session is created.
afterSessionCreate — called after all onSetupSession functions have been called.
onBeginServicing — at the start of the request processing.
onEndServicing — and the end of request processing.
onAboutToShutdownSession — called just before a session is shutdown, for example when a session expires or the Lift application is being shutdown.
onShutdownSession — called after all onAboutToShutdownSession functions have been run.

If you are testing testing these hooks, you might want to make session expire faster than the 30 minutes of inactivity used by default in Lift. To do this, supply a millisecond value to LiftRules.sessionInactivityTimeout:

// 30 second inactivity timeout
LiftRules.sessionInactivityTimeout.default.set(Full(1000L * 30))

There are two other hooks in LiftSession: onSessionActivate and onSessionPassivate. These may be of use if you are working with a servlet container in distributed mode, and want to be notified when the servlet HTTP session is about to be serialized (passivated) and de-serialized (activated) between container instances. These hooks are rarely used.

Run Code when Lift Shuts Down

Problem

You want to have some code executed when your Lift application is shutting down.

Solution

Append to LiftRules.unloadHooks.

LiftRules.unloadHooks.append( () => println("Shutting down") )

Discussion

You append functions of type () ⇒ Unit to unloadHooks, and these functions are run right at the end of the Lift handler, after sessions have been destroyed, Lift actors have been shutdown, and requests have finished being handled.

This is triggered, in the words of the Java servlet specification, "by the web container to indicate to a filter that it is being taken out of service".

Running Stateless

Problem

You want to force your application to be stateless at the HTTP level.

Solution

In Boot.scala:

LiftRules.enableContainerSessions = false
LiftRules.statelessReqTest.append { case _ => true }

All requests will now be treated as stateless. Any attempt to use state, such as via SessionVar for example, will trigger a warning in developer mode: "Access to Lift’s statefull features from Stateless mode. The operation on state will not complete."

Discussion

HTTP session creation is controlled via enableContainerSessions, and applies for all requests. Leaving this value at the default (true) allows more fine-grained control over which requests are stateless.

Using statelessReqTest allows you to decide, based on the StatelessReqTest case class, if a request should be stateless (true) or not (false). For example:

def asset(file: String) =
  List(".js", ".gif", ".css").exists(file.endsWith)

LiftRules.statelessReqTest.append {
  case StatelessReqTest("index" :: Nil, httpReq) => true
  case StatelessReqTest(List(_, file),  _) if asset(file) => true
}

This example would only make the index page and any GIFs, JavaScript and CSS files stateless. The httpReq part is a HTTPRequest instance, allowing you to base the decision on the content of the request (cookies, user agent, etc).

Another option is LiftRules.statelessDispatch which allows you to register a function which returns a LiftResponse. This will be executed without a session, and convenient for REST-based services.

If you just need to mark an entry in Sitemap as being stateless, you can:

Menu.i("Stateless Page") / "demo" >> Stateless

A request for /demo would be processed without state.

Catch Any Exception

Problem

You want a wrapper around all requests to catch exceptions and display something to the user.

Solution

Declare an exception handler in Boot.scala:

LiftRules.exceptionHandler.prepend {
  case (runMode, request, exception) =>
    logger.error("Failed at: "+request.uri)
    InternalServerErrorResponse()
}

In the above example, all exceptions for all requests at all run modes are being matched, causing an error to be logged and a 500 (internal server error) to be returned to the browser.

Discussion

The partial function you add to exceptionHandler needs to return a LiftResponse (i.e., something to send to the browser). The default behaviour is to return an XhtmlResponse, which in Props.RunModes.Development gives details of the exception, and in all other run modes simply says: "Something unexpected happened".

You can return any kind of LiftResponse, including RedirectResponse, JsonResponse, XmlResponse, JavaScriptResponse and so on.

The example above just sends a standard 500 error. That won’t be very helpful to your users. An alternative is to render a custom message, but retain the 500 status code which will be useful for external site monitoring services if you use them:

LiftRules.exceptionHandler.prepend {
  case (runMode, req, exception) =>
    logger.error("Failed at: "+req.uri)
    val content = S.render(<lift:embed what="500" />, req.request)
    XmlResponse(content.head, 500, "text/html", req.cookies)
}

Here we are sending back a response with a 500 status code, but the content is the Node that results from running src/main/webapp/template-hidden/500.html. Create that file with the message you want to show to users:

<html>
<head>
  <title>500</title>
</head>
<body data-lift-content-id="main">
<div id="main" data-lift="surround?with=default;at=content">
  <h1>Something is wrong!</h1>
  <p>It's our fault - sorry</p>
</div>
</body>
</html>

You can also control what to send to clients when processing Ajax requests. In the following example, we’re matching just on Ajax POST requests, and returning custom JavaScript to the browser:

import net.liftweb.http.js.JsCmds._

val ajax = LiftRules.ajaxPath

LiftRules.exceptionHandler.prepend {
  case (mode, Req(ajax :: _, _, PostRequest), ex) =>
    logger.error("Error handing ajax")
    JavaScriptResponse(Alert("Boom!"))
}

You could test out this handling code by creating an Ajax button that always produces an exception:

package code.snippet

import net.liftweb.util.Helpers._
import net.liftweb.http.SHtml

class ThrowsException {
  private def fail = throw new Error("not implemented")

  def render = "*" #> SHtml.ajaxButton("Press Me", () => fail)
}

This Ajax example will jump in before Lift’s default behaviour for Ajax errors. The default is to retry the Ajax command three times (LiftRules.ajaxRetryCount), and then execute LiftRules.ajaxDefaultFailure, which will pop up a dialog saying: "The server cannot be contacted at this time"

Streaming Content

Problem

You want to stream content back to the web client.

Solution

Use OutputStreamResponse, passing it a function that will write to the OutputStream that Lift supplies.

In this example we’ll stream all the integers from one, via a REST service:

package code.rest

import net.liftweb.http.{Req,OutputStreamResponse}
import net.liftweb.http.rest._

object Numbers extends RestHelper {

  // Convert a number to a String, and then to UTF-8 bytes
  // to send down the output stream.
  def num2bytes(x: Int) = (x + "\n") getBytes("utf-8")

  // Generate numbers using a Scala stream:
  def infinite = Stream.from(1).map(num2bytes)

  serve {
    case Req("numbers" :: Nil, _, _) =>
      OutputStreamResponse( out => infinite.foreach(out.write) )
  }
}

Scala’s Stream class is a way to generate a sequence with lazy evaluation. The values being produced by infinite are used as example data to stream back to the client.

Wire this into Lift in Boot.scala:

LiftRules.dispatch.append(Numbers)

Visiting http://127.0.0.1:8080/numbers will generate a 200 status code and start producing the integers from 1. The numbers are produced quite quickly, so you probably don’t want to try that in your web browser, but instead from something that is easier to stop, such as cURL.

Discussion

OutputStreamResponse expects a function of type OutputStream ⇒ Unit. The OutputStream argument is the output stream to the client. This means the bytes we write to the stream are written to the client. In the above example…

OutputStreamResponse(out => infinite.foreach(out.write))

…we are making use of the write(byte[]) method on Java’s OutputStream (out), and sending it the Array[Byte] being generated from our infinite stream.

For more control over status codes, headers and cookies, there are a variety of signatures for the OutputStreamResponse object. For the most control, create an instance of the OutputStreamResponse class:

case class OutputStreamResponse(
  out: (OutputStream) => Unit,
  size: Long,
  headers: List[(String, String)],
  cookies: List[HTTPCookie],
  code: Int)

Any headers you set (such as Content-type), or status code, may already have been set by the time your output function is called. Note that setting size to -1 causes the Content-length header to be skipped.

There are two related types of response: InMemoryResponse and StreamingResponse.

InMemoryResponse

InMemoryResponse is useful if you have already assembled the full content to send to the client. The signature is straightforward:

case class InMemoryResponse(
  data: Array[Byte],
  headers: List[(String, String)],
  cookies: List[HTTPCookie],
  code: Int)

As an example, we can modify the recipe and force our infinite sequence of numbers to produce the first few numbers as a Array[Byte] in memory:

import net.liftweb.util.Helpers._

serve {
  case Req(AsInt(n) :: Nil, _, _) =>
    InMemoryResponse(infinite.take(n).toArray.flatten, Nil, Nil, 200)
}

The AsInt helper in Lift matches on an integer, meaning that a request starting with a number matches and we’ll return that many numbers from the infinite sequence. We’re not setting headers or cookies, and this request produces what you’d expect:

$ curl http://127.0.0.1:8080/3
1
2
3

StreamingResponse

StreamingResponse pulls bytes into the output stream. This contrasts with OutputStreamResponse, where you are pushing data to the client.

Construct this type of response by providing a class with a read method that can be read from:

case class StreamingResponse(
  data: {def read(buf: Array[Byte]): Int},
  onEnd: () => Unit,
  size: Long,
  headers: List[(String, String)],
  cookies: List[HTTPCookie],
  code: Int)

Notice the use of a structural type for the data parameter. Anything with a matching read method can be given here, including java.io.InputStream-like objects, meaning StreamingResponse can act as a pipe from input to output. Lift pulls 8k chunks from your StreamingResponse to send to the client.

Your data read function should follow the semantics of Java IO and return "the total number of bytes read into the buffer, or -1 is there is no more data because the end of the stream has been reached".

Serving a File with Access Control

Problem

You have a file on disk, you want to allow a user to download it, but only if they are allowed to. If they are not allowed to, you want to explain why.

Solution

Use RestHelper to serve the file or an explanation page.

For example, suppose we have the file /tmp/important and we only want selected requests to download that file from the /download/important URL. The structure for that would be:

package code.rest

import net.liftweb.util.Helpers._
import net.liftweb.http.rest.RestHelper
import net.liftweb.http.{StreamingResponse, LiftResponse, RedirectResponse}
import net.liftweb.common.{Box, Full}
import java.io.{FileInputStream, File}

object DownloadService extends RestHelper {

  // (code explained below to go here)

  serve {
    case "download" :: Known(fileId) :: Nil Get req =>
      if (permitted) fileResponse(fileId)
      else Full(RedirectResponse("/sorry"))
  }
}

We are allowing users to download "known" files. That is, files which we approve of for access. We do this because opening up the file system to any unfiltered end-user input pretty much means your server will be compromised.

For our example, Known is checking a static list of names:

val knownFiles = List("important")

object Known {
 def unapply(fileId: String): Option[String] = knownFiles.find(_ == fileId)
}

For requests to these known resources, we convert the REST request into a Box[LiftResponse]. For permitted access we serve up the file:

private def permitted = scala.math.random < 0.5d

private def fileResponse(fileId: String): Box[LiftResponse] = for {
    file <- Box !! new File("/tmp/"+fileId)
    input <- tryo(new FileInputStream(file))
 } yield StreamingResponse(input,
    () => input.close,
    file.length,
    headers=Nil,
    cookies=Nil,
    200)

If no permission is given, the user is redirected to /sorry.html.

All of this is wired into Lift in Boot.scala with:

LiftRules.dispatch.append(DownloadService)

Discussion

By turning the request into a Box[LiftResponse] we are able to serve up the file, send the user to a different page, and also allow Lift to handle the 404 (Empty) cases.

If we added a test to see if the file existed on disk in fileResponse that would cause the method to evaluate to Empty for missing files, which triggers a 404. As the code stands, if the file does not exist, the tryo would give us a Failure which would turn into a 404 error with a body of "/tmp/important (No such file or directory)".

Because we are testing for known resources via the Known extractor as part of the pattern for /download/, unknown resources will not be passed through to our File access code. Again, Lift will return a 404 for these.

Guard expressions can also be useful for these kinds of situations:

serve {
  case "download" :: Known(id) :: Nil Get _ if permitted => fileResponse(id)
  case "download" :: _ Get req => RedirectResponse("/sorry")
}

You can mix and match extractors, guards and conditions in your response to best fit the way you want the code to look and work.

Access Restriction by HTTP Header

Problem

You need to control access to a page based on the value of a HTTP header.

Solution

Use a custom If in SiteMap:

val HeaderRequired = If(
  () => S.request.map(_.header("ALLOWED") == Full("YES")) openOr false,
  "Access not allowed"
)

// Build SiteMap
val entries = List(
  Menu.i("Header Required") / "header-required" >> HeaderRequired
)

In this example header-required.html can only be viewed if the request includes a HTTP header called ALLOWED with a value of YES. Any other request for the page will be redirected with a Lift error notice of "Access not allowed".

This can be tested from the command line using a tool like cURL:

$ curl http://127.0.0.1:8080/header-required.html -H "ALLOWED:YES"

Discussion

The If test ensures the () ⇒ Boolean function you supply as a first argument returns true before the page it applies to is shown. In this example we’ll get true if the request contains a header called "ALLOWED", and the optional value of that header is Full("YES"). This is a LocParam (location parameter) which modifies the site map item. It can be appended to any menu items you want using the >> method.

Note that without the header, the test will be false. This will mean with link to the page will not appear the menu generated by Menu.builder.

The second argument to the If() is what Lift does if the test isn’t true when the user tries to access the page. It’s a () ⇒ LiftResponse function. This means return whatever you like, including redirects to other pages. In the example we are making use of a convenient implicit conversation from a String ("Access not allowed") to a redirection that will take the user to the home page.

If you visit the page without a header, you’ll see a notice saying "Access not allowed". This will be the home page of the site, but that’s just the default.

You can request that Lift show a different page by setting LiftRules.siteMapFailRedirectLocation in Boot.scala:

LiftRules.siteMapFailRedirectLocation = "static" :: "permission" :: Nil

If you then try to access header-required.html without the header set, you’ll be redirected to /static/permission and shown the content of whatever you put in that page.

Accessing `HttpServletRequest`

Problem

To satisfy some API you need access to the HttpServletRequest.

Solution

Cast S.request:

import net.liftweb.http.S
import net.liftweb.http.provider.servlet.HTTPRequestServlet
import javax.servlet.http.HttpServletRequest

def servletRequest: Box[HttpServletRequest] = for {
  req <- S.request
  inner <- Box.asA[HTTPRequestServlet](req.request)
} yield inner.req

You can then make your API call:

servletRequest.foreach { r => yourApiCall(r) }

Discussion

Lift abstracts away from the low-level HTTP request, and from the details of the servlet container your application is running in. However, it’s reassuring to know, if you absolutely need it, there is a way to get back down to the low-level.

Note that the results of servletRequest is a Box because there might not be a request when you evaluate servletRequest — or you might one day port to a different deployment environment and not be running on a standard Java servlet container.

As your code will have a direct dependency on the Java Servlet API, you’ll need to include this dependency in your SBT build:

"javax.servlet" % "servlet-api" % "2.5" % "provided->default"

Force HTTPS Requests

Problem

You want to ensure clients are using HTTPs.

Solution

Add an earlyResponse function in Boot.scala redirecting http requests to https equivalents. For example:

LiftRules.earlyResponse.append { (req: Req) =>
  if (req.request.scheme != "https") {
    val uriAndQuery = req.uri + (req.request.queryString.map(s => "?"+s) openOr "")
    val uri = "https://%s%s".format(req.request.serverName, uriAndQuery)
    Full(PermRedirectResponse(uri, req, req.cookies: _*))
  }
  else Empty
}

Discussion

The earlyResponse call is called early in the Lift pipeline. It is used to execute code before a request is handled and, if required, exit the pipeline and return a response. The function signature expected is Req ⇒ Box[LiftResponse].

In this example we are testing for a request that is not "https", and then formulating a new URL that starts "https" and appends to it the rest of the original URL and any query parameters. With this created, we return a redirections to the new URL, along with any cookies that were set.

By evaluating to Empty for other requests (i.e., https requets), Lift will continue passing the request through the pipeline as usual.

The ideal method to ensure requests are served using the correct scheme would be via web server configuration, such as Apache or Nginx. This isn’t possible in some cases, such as when your application is deployed to a PaaS such as CloudBees.

Amazon Load Balancer

For Amazon Elastic Load Balancer note that you need to use X-Forwarded-Proto header to detect HTTPS. As mentioned in their Overview of Elastic Load Balancing document, "Your server access logs contain only the protocol used between the server and the load balancer; they contain no information about the protocol used between the client and the load balancer."

In this situation modify the above test from req.request.scheme != "https" to:

req.header("X-Forwarded-Proto") != Full("https")

Files

06-Pipline.asciidoc

Latest commit

History

06-Pipline.asciidoc

File metadata and controls

Request Pipeline

Debugging a Request

Problem

Solution

Discussion

See Also

Running Code when Sessions are Created (or Destroyed)

Problem

Solution

Discussion

See Also

Run Code when Lift Shuts Down

Problem

Solution

Discussion

See Also

Running Stateless

Problem

Solution

Discussion

See Also

Catch Any Exception

Problem

Solution

Discussion

See Also

Streaming Content

Problem

Solution

Discussion

InMemoryResponse

StreamingResponse

See Also

Serving a File with Access Control

Problem

Solution

Discussion

See Also

Access Restriction by HTTP Header

Problem

Solution

Discussion

See Also

Accessing HttpServletRequest

Problem

Solution

Discussion

Force HTTPS Requests

Problem

Solution

Discussion

Amazon Load Balancer

See Also

Accessing `HttpServletRequest`