Reverse Proxy Services with Varnish (Part Two)

reverse proxy

Worker thread settings and VCL settings in Varnish under pfSense 2.1.3.

In the last article, we introduced Varnish, a powerful reverse proxy server, and covered installation and basic configuration. In this article, we will continue to look at some of the configuration options.

We will begin by looking at the settings for the second tab, “Settings“. We already covered “Daemon options” in the previous article. Next is “Storage type“. Here, you can select which storage type Varnish uses. By default, the cache is stored in memory (which makes it faster), but you can switch to disk storage instead. You can also specify the cache size in megabytes here (obviously, you do not want to exceed your RAM/storage space).

Next is “Worker thread configuration“. Here you can set the minimum number of Varnish worker threads, the maximum number of Varnish worker threads, and the worker thread timeout. The next subheading is “General VCL Settings“. The first item is the “Client identity method“. This setting is relevant if you have load balancing configured and you want to load balance based on something the client provides: an IP address, a requested URL, or a user agent. The “Don’t cache posts” check box causes Varnish to stop caching POST requests. The “streaming support” check box enables streaming support. “Session Cache” defines Varnish’s behavior with respect to cookies. By default, Varnish does not cache content with cookies in it. This is the recommended behavior, but should you want to cache even when there are cookies involved (and changing the web application is not possible), you can set up a session cache on a per-user basis here (which makes for a fairly low cache hit ratio). It requires your system to not change the cookie on each page hit. “Cache static content” specifies when Varnish should cache static content. If set to “When possible“, Varnish will cache only static content and per user objects when the session cache is set to it. “Always” will cause Varnish to cache all static content; when cookies are present, Varnish will unset it from the object before caching. “Never” will cause Varnish to never cache static content.

The “Fix gzip compression” check box causes Varnish to ignore compression for image files and unknown compression algorithms if checked. The “Be rFC2616 compliant” check box will cause Varnish to ignore requests that do not comply with RFC 2616. “Forward client IP” lets you select how to log the client’s IP address: “set X-Forwarded For“, “append X-Forwarded For“, “set X-Forwarded-Varnish“, and “unset” to leave it unlogged. “Fetch Grace” allows you to set how long Varnish will keep cached objects.

The last section on this page is “Error Settings“. “Retries” determines how many times Varnish will try before it sends an error message. “Saintmode” determines how long Varnish will send cached objects from a backend which has gone offline. Finally, “Custom HTML error message” lets you paste a custom html error page code.

That covers the settings in the Varnish “Settings” tab. In the next article, we will continue our look at Varnish settings.

External Links:

Caching, even when cookies are present at

Reverse Proxy Services with Varnish (Part One)

reverse proxy

Backend server settings page for Varnish under pfSense 2.1.3.

I recently ran a series of articles on Squid, a proxy server which began life as a client-side cache and can be used under pfSense as such. In contrast, Varnish is a reverse proxy HTTP accelerator designed for content-heavy dynamic web sites. It is focused exclusively on HTTP, unlike other reverse proxy servers that support other network protocols.

The Varnish Reverse Proxy: Architecture and Performance

Varnish works by storing data in virtual memory and leaving the task of deciding what is stored in memory and what gets paged out to disk to the operating system. This helps avoid the situation where the operating system starts caching data while it is moved to disk by the application. In addition, Varnish is heavily threaded, with each client connection being handled by a separate worker thread, and an overflow queue to handle incoming connections once the configured limit on the number of active worker threads is reached. When the overflow queue limit is reached, incoming connections will be rejected.

Varnish uses the Varnish Configuration Language (VCL), which is used to write hooks that are called at critical points in the handling of each request. When a VCL script is loaded, it is translated to C, compiled into a shared object by the system compiler, and loaded directly into the accelerator.

The creators of Varnish claim that it speeds up delivery by a factor of 300-1000 times, depending on your architecture. When the Varnish cache is enabled, performance is bound by the speed of the network, thus turning web server performance into a non-issue. It is free software licensed under a two-clause BSD license, also known as the FreeBSD license.

The Varnish Reverse Proxy: Installation and Configuration

reverse proxy

The Settings tab in Varnish.

Installation of Varnish under pfSense is similar to installation of other packages. From the pfSense web GUI, navigate to System -> Packages and scroll down. You will see both “Varnish” and “Varnish 3” on the packages list. Click on the “plus” sign to the right of the listing for Varnish 3 to install the Varnish reverse proxy. On the next screen, press the “Confirm” button to confirm that you want the package installed. It should take about 3-4 minutes for installation to complete.

Once Varnish is installed, you will have a new item on the “Services” menu called “Varnish“. If you navigate there, you will see seven tabs covering different settings for the Varnish reverse proxy server: “Backends“, “Settings“, “Custom VCL“, “LB Directors“, ‘XMLRPC Sync“, “View Configuration“, and “VarnishSTAT“. Before you can enable Varnish, you need to configure at least one backend server, so click on the “Backends” tab.

Varnish has a concept of “backend” or “origin” servers. A backend server is the server providing the content Varnish will accelerate. To add a backend server, click on the “plus” button on the “Backends” tab. There are four sections on this page. “Backend settings” covers the most basic settings. “Backend name” is the name of the backend web server, and “IP address” is the IP address (on your local network) of the backend web server. At “Port“, you enter the port of the web server (usually 80), and for “Description“, you can enter a description.

The next section is “Performance metrics“. “First byte timeout” represents the time to wait for the first byte for the backend and the time to wait between each received byte. “Connect timeout” is simply the time to wait for a backend connection.

Next is “Probe settings“. If this is configured properly, the Varnish reverse proxy will check the health of each backend with a probe. “Probe URL” is the URL that Varnish will use to ensure that the backend is healthy. “Probe interval” specifies how often we should poll. “Timeout” simply specifies the timeout of the probe; this is the amount of time (in seconds) that Varnish will wait for a backend probe. “Probe Window” specifies how many probes Varnish will retain when considering backend health. “Threshold” specifies how many of the probes specified by “Window” must have succeeded for us to consider the backend healthy. Finally, checking the “Disable Probe” check box disables probing for this backend. In the last section, “Backend Mappings“, you can map either a hostname or a URL to this server. It can either match a string or a regular expression.

The second tab is the “Settings” tab. Under “Daemon options” you can enable Varnish by checking the “Enable Varnish” check box. At “Listening port” you can set the port Varnish will listen on. The “Management interface” specifies the IP address and port for the management interface, and “Advanced startup options” specifies any startup options to include in the rc.d configuration file.

If you specify settings for these options and check the “Enable Varnish” check box, you should have Varnish up and running and working with any web servers you specified as a backend. There are several other settings that you might find useful, and we will cover those in the next article.

External Links:

The official Varnish web site

Varnish at Wikipedia

© 2013 David Zientara. All rights reserved. Privacy Policy