{"id":2530,"date":"2017-02-13T23:36:02","date_gmt":"2017-02-13T22:36:02","guid":{"rendered":"http:\/\/www.nivas.hr\/blog\/?p=2530"},"modified":"2017-02-14T00:15:28","modified_gmt":"2017-02-13T23:15:28","slug":"apache-sending-vary-host-making-things-uncacheable-varnish","status":"publish","type":"post","link":"https:\/\/www.nivas.hr\/blog\/2017\/02\/13\/apache-sending-vary-host-making-things-uncacheable-varnish\/","title":{"rendered":"Apache sending “Vary: Host” making things uncacheable for Varnish"},"content":{"rendered":"

TLDR;
\nUsing %{HTTP_HOST}<\/em> in .htaccess<\/em>, will cause Apache to included a \u201cVary: Host\u201d<\/em> field in response.
\nSubsequently “Vary: Host”<\/em> header from Apache will force Varnish not to cache otherwise cacheable content.<\/strong><\/p>\n

HTTP Vary is not a trivial concept. It is by far the most misunderstood HTTP header. (Varnish Docs)<\/p><\/blockquote>\n

On a project I’ve been working, I could not make Varnish hard cache the site no matter what I did.
\nIt was mostly read-only site and I wanted to achieve that in case of backend failures – site would still run from the cache. To avoid surprises, I was using my own configuration template which was working exactly as I wanted it on different project. <\/p>\n

But, no matter what I did – Varnish was not caching everything. Instead I got ‘per-user<\/em>‘ cache: if user visited eg, homepage, in case of backend failure – user could reload homepage and Varnish would serve homepage from stale cache to the user. Visiting any other page user did not visit before backend failure (and therefore not in his cache), would result in Varnish freaking that backend is down.<\/p>\n

What I noticed was strangely large Vary-Header in Varnish response<\/p>\n

\r\nVary:Host,Accept-Encoding,User-Agent\r\n<\/pre>\n
After spending hours of tunneling, debugging Varnish configuration, analysing header responses, comparing server configurations, hunting for cookies that could have been somehow magically slip through… last place I looked was my main .htaccess. I was using mod_deflate there, but as it checked out everything was fine.<\/p>\n
On a side note, I have been experimenting with per host configuration in .htaccess, so Basic Authentication would not kick in dev enviroment. It was working great up until now, and it goes something like this:<\/p>\n
\r\n<if "%{HTTP_HOST} == 'dev.nivas.hr'">\r\n\r\nRequire valid-user\r\n...\r\nAllow from facebook.com\r\n\r\n<\/if>\r\n<\/pre>\n
What I had to find out the hard way, is that if special Apache enviroment variable %{HTTP_HOST}<\/strong> was used in a eg. .htaccess, Apache would change response header. Server was returning a header which included a \u201cVary: Host<\/strong>\u201d field, which means that the server didn\u2019t serve a regular static page, but its reply depends on the \u201cHost\u201d field in the HTTP request. Browsers interpret this as \u201cthe content returned is dynamic, don\u2019t cache it (source<\/a>).<\/p>\n
\r\ncurl -I http:\/\/localhost\/\r\nHTTP\/1.1 200 OK\r\nDate: Mon, 13 Feb 2017 14:48:14 GMT\r\nServer: Apache\/2.4.6 (CentOS)\r\nVary: Host,User-Agent\r\nContent-Type: text\/html; charset=UTF-8\r\n<\/pre>\n
It is really strange I did not hit this before because RewriteCond can add “Host” to the Vary-Header as well. eg. a RewriteCond that evaluates %{HTTP_HOST} automatically adds “Host” to the Vary-Header. This is unnecessary and not permitted according to https:\/\/tools.ietf.org\/html\/rfc7231#section-7.1.4<\/a>. The issue was reported and has been sitting in Apache bugtraq for a while<\/a> without clear resolution.<\/p>\n
I can understand why Varnish is not caching, but cannot understand Apache logic. I do have VirtualHost defined, so therefore my request do vary on Host. There is no need in forcing this out in response.  <\/p>\n
After removing %{HTTP_HOST} from .htaccess, site was cacheable as we wanted.<\/p>\n
\r\nHTTP\/1.1 200 OK\r\nDate: Mon, 13 Feb 2017 22:18:33 GMT\r\nContent-Type: text\/html; charset=UTF-8\r\nVary: Accept-Encoding\r\nAge: 3707\r\nX-Nivas-Crew: loves you :) http:\/\/www.nivas.hr\r\nX-Backend: backend_app1\r\nX-Cache: HIT\r\nX-Cache-Hits: 786332\r\nX-Vudu-Url-Cache: hit\r\n<\/pre>\n
Don’t forget to normalize your Vary in Varnish, chance are without normalization it will never see a cache hit.<\/p>\n
Happy caching!<\/p>\n
\nHave a cool Varnish project you need help on? Contact us<\/a>.
\n<\/h2>\n","protected":false},"excerpt":{"rendered":"
TLDR; Using %{HTTP_HOST} in .htaccess, will cause Apache to included a \u201cVary: Host\u201d field in response. Subsequently “Vary: Host” header from Apache will force Varnish not to cache otherwise cacheable content. HTTP Vary is not a trivial concept. It is by far the most misunderstood HTTP header. (Varnish Docs) On a project I’ve been working,…<\/p>\n","protected":false},"author":3,"featured_media":2544,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1,38],"tags":[50,49],"_links":{"self":[{"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/posts\/2530"}],"collection":[{"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/comments?post=2530"}],"version-history":[{"count":30,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/posts\/2530\/revisions"}],"predecessor-version":[{"id":2561,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/posts\/2530\/revisions\/2561"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/media\/2544"}],"wp:attachment":[{"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/media?parent=2530"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/categories?post=2530"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.nivas.hr\/blog\/wp-json\/wp\/v2\/tags?post=2530"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}