Run Drupal with Varnish 3.x on Ubuntu

On this tutorial I will show you how to configure Varnish as a proxy server so you can re route you requests to any Apache or Nginx Server. 
Configuring Varnish on Ubuntu on distributed Servers is not a complex task, we just need to download some packages and then we will have to copy/paste the configurations below in order to start. Feel free to modify the provided configurations according to your needs. This post assumes there is another server Apache/Nginx listening on port 80 and running a Drupal site. 
Ok, Let's begin.

Install Varnish 3.0.5 from Ubuntu 14.04 repositories:

$sudo apt-get install varnish

Configure varnish deamon:

$sudo nano /etc/default/varnish

DAEMON_OPTS="-a :80,:443 \
-T 192.168.10.15:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,512M"

Here we are telling the deamon to listen on ports 80 and 443, we are being explicit by setting the IP address of the Varnish server, we also tell the deamon the location of the security secret key and how much memory the deamon should use

I look over several default configurations suggested for Varnish that are floating on the web and I recommend using the one pointed by Lullabot, I just made some tweaks to it so it fits this tutorial. I'm also including a quick fix for the Lullabot's file since it is missing a Session Cookie Rule when requests come from SSL, click here for further explanation.

$sudo nano /etc/varnish/default.vcl

# This is a basic VCL configuration file for varnish.  See the vcl(7)
# man page for details on VCL syntax and semantics.
#
# Default backend definition.  Set this to point to your content
# server.
#

## Webserver default - APACHE or NGINX
backend default {
  .host = "192.168.100.50";
  .port = "80";
  .max_connections = 500;
  .connect_timeout = 300s;
  .first_byte_timeout = 300s;
  .between_bytes_timeout = 300s;
}
##
sub vcl_recv {

  ## Default http
  set req.backend = default;

  # Modify HTTP X-Forwarded-For header.
  # This will replace Varnish's IP with actual client's.
  if (req.restarts == 0) {
    if (req.http.x-forwarded-for) {
      set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip;
    }
    else {
      set req.http.X-Forwarded-For = client.ip;
    }
  }

  # A great functionality of Varnish is to check
  # your web server's health and serve stale pages
  # if necessary.
  # In case of web server lag, let's return the
  # request with stale content.
  if (req.backend.healthy) {
    set req.grace = 60s;
  } else {
    set req.grace = 30m;
    # Use anonymous, cached pages if all backends are down.
    unset req.http.Cookie;
  }

  # Verify HTTP request methods.
  if (req.request != "GET" &&
      req.request != "HEAD" &&
      req.request != "PUT" &&
      req.request != "POST" &&
      req.request != "TRACE" &&
      req.request != "OPTIONS" &&
      req.request != "DELETE") {
    /* Non-RFC2616 or CONNECT which is weird. */
    return (pipe);
  }
  if (req.request != "GET" && req.request != "HEAD") {
    /* We only deal with GET and HEAD by default */
    return (pass);
  }

  # Handling of different encoding types.
  if (req.http.Accept-Encoding) {
    if (req.url ~ "\.(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {
      remove req.http.Accept-Encoding;
    } elseif (req.http.Accept-Encoding ~ "gzip") {
      set req.http.Accept-Encoding = "gzip";
    } elseif (req.http.Accept-Encoding ~ "deflate") {
      set req.http.Accept-Encoding = "deflate";
    } else {
      remove req.http.Accept-Encoding;
    }
  }

  # Do not cache these URL paths.
  if (req.url ~ "^/status\.php$" ||
    req.url ~ "^/update\.php$" ||
    req.url ~ "^/ooyala/ping$" ||
    req.url ~ "^/admin"        ||
    req.url ~ "^/admin/build/features" ||
    req.url ~ "^/admin/.*$"    ||
    req.url ~ "^/user"         ||
    req.url ~ "^/user/.*$"     ||
    req.url ~ "^/users/.*$"    ||
    req.url ~ "^/info/.*$"     ||
    req.url ~ "^/flag/.*$"     ||
    req.url ~ "^.*/ajax/.*$"   ||
    req.url ~ "^.*/ahah/.*$"   ||
    req.url ~ "^.*/media/.*$")
  {
      return (pass);
  }

  # Pipe these paths directly to Apache for streaming.
  if (req.url ~ "^/admin/content/backup_migrate/export") {
    return (pipe);
  }

  # Always cache the following file types for all users.
  if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm|pdf|doc)(\?[a-z0-9]+)?$"){
    unset req.http.Cookie;
  }

  # Remove all cookies that Drupal doesn't need to know about. ANY remaining
  # cookie will cause the request to pass-through to Apache. For the most part
  # we always set the NO_CACHE cookie after any POST request, disabling the
  # Varnish cache temporarily. The session cookie allows all authenticated users
  # to pass through as long as they're logged in.

  if (req.http.Cookie) {
    set req.http.Cookie = ";" + req.http.Cookie;
    set req.http.Cookie = regsuball(req.http.Cookie, "; +", ";");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(SESS[a-z0-9]+|NO_CACHE)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";(SSESS[a-z0-9]+|NO_CACHE)=", "; \1=");
    set req.http.Cookie = regsuball(req.http.Cookie, ";[^ ][^;]*", "");
    set req.http.Cookie = regsuball(req.http.Cookie, "^[; ]+|[; ]+$", "");

    if (req.http.Cookie == "") {
      # If there are no remaining cookies, remove the cookie header. If there
      # aren't any cookie headers, Varnish's default behaviour will be to cache
      # the page.
      unset req.http.Cookie;
    } else {
      # If there is any cookies left (a session or NO_CACHE cookie), do not
      # cache the page. Pass it on to Apache directly.
      return (pass);
    }

  }

  /* Not cacheable by default */
  if (req.http.Authorization || req.http.Cookie) {
    return (pass);
  }

  //END
  return (lookup);
}
##
sub vcl_hash {

  # Use special internal SSL hash for https content
  # X-Forwarded-Proto is set to https by Pound
  if (req.http.X-Forwarded-Proto ~ "https") {
    hash_data(req.http.X-Forwarded-Proto);
  }
}
# Set a header to track a cache HIT/MISS.
sub vcl_deliver {

  if (obj.hits > 0) {
    set resp.http.X-Varnish-Cache = "HIT";
  } else {
    set resp.http.X-Varnish-Cache = "MISS";
  }

  if (!(req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js)(\?[a-z0-9]+)?$")) {
    set resp.http.Cache-Control = "no-cache, must-revalidate, post-check=0, pre-check=0";
  }

}
##
# Code determining what to do when serving items from the Apache servers.
# beresp == Back-end response from the web server.
sub vcl_fetch {

  # Don't allow static files to set cookies.
  if (req.url ~ "(?i)\.(png|gif|jpeg|jpg|ico|swf|css|js|html|htm)(\?[a-z0-9]+)?$") {
    unset beresp.http.set-cookie;
  }
  # Allow items to be stale if needed.
  set beresp.grace = 6h;
}
## Error
sub vcl_error {
  set obj.http.Content-Type = "text/html; charset=utf-8";
  set obj.http.Retry-After = "5";
  synthetic {"
    <?xml version="1.0" encoding="utf-8"?>
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
    "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html>
    <head>
    <title>"} + obj.status + " " + obj.response + {"</title>
    </head>
    <body>
      <h1>Error "} + obj.status + " " + obj.response + {"</h1>
      <p>"} + obj.response + {"</p>
      <h3>Guru Meditation:</h3>
      <p>XID: "} + req.xid + {"</p>
      <hr>
      <p>Varnish cache server</p>
     </body>
     </html>
"};

 return(deliver);
}

Restart Varnish:

$sudo service varnishd restart

Get Varnish Control Secret key:

$sudo cat /etc/varnish/secret

Now let's move to our Drupal site, download and install the Varnish Drupal Module. Then visit:  /admin/config/development/varnish Varnish config page and set values as follows:

• Varnish Control Terminal, set the value of your varnish server ip i.e. 192.168.10.15:6082
• Set Varnish Version to 3.x
• by the time of this post this bug is present on the Varnish Drupal module 7.x-beta-3, but there is a patch that fixes the issue here: https://www.drupal.org/node/2371907
• Get the Secret Key value from the command above.

add Varnish Cache to your settings.php by adding the following:

// Add Varnish as the page cache handler.
// Tell Drupal it's behind a proxy.
$conf['reverse_proxy'] = TRUE;
//Varnish Server IP; this can also be configured through the module's UI
$conf['reverse_proxy_addresses'] = array('127.0.0.1');
// Bypass Drupal bootstrap for anonymous users so that Drupal sets max-age < 0.
$conf['page_cache_invoke_hooks'] = FALSE;
// Make sure that page cache is enabled.
$conf['cache'] = 1;
$conf['cache_lifetime'] = 0;
$conf['page_cache_maximum_age'] = 21600;
$conf['cache_backends'][] = 'sites/all/modules/varnish/varnish.cache.inc';
$conf['cache_class_cache_page'] = 'VarnishCache';
$conf['reverse_proxy_header'] = 'HTTP_X_FORWARDED_FOR';
$conf['omit_vary_cookie'] = TRUE;

Alright, now you have a powerful Proxy Server set up using Varnish.