{"id":22129,"date":"2025-12-11T17:07:19","date_gmt":"2025-12-11T16:07:19","guid":{"rendered":"https:\/\/www.transparentedge.eu\/en\/blog\/"},"modified":"2025-12-12T11:46:41","modified_gmt":"2025-12-12T10:46:41","slug":"the-vary-header-leveraging-content-negotiation","status":"publish","type":"post","link":"https:\/\/www.transparentedge.eu\/en\/blog\/the-vary-header-leveraging-content-negotiation\/","title":{"rendered":"The Vary header: leveraging content negotiation to do (a little) of everything"},"content":{"rendered":"\n

If you’ve been working with CDNs or caching systems like Nginx or Varnish for a while, or if you’re an expert in the HTTP protocol and web performance, you’ve probably come across the Vary<\/code> header more than once. It’s one of those largely misunderstood (due to its complexity) elements of the protocol, and it’s a double-edged sword; it can be your best ally for delivering the right content or your worst enemy, destroying your hit-ratio.<\/p>\n\n\n\n

Recently, one of the creators of Django<\/a> was talking about our competition in the world of caching and web performance, highlighting a limitation (which, of course, we don’t have B-) ). Beyond that, it made me think about the lack of awareness that exists, even among prestigious professionals, regarding the use of this header.<\/p>\n\n\n\n

Today we’re going to break down exactly what Vary<\/code> does, why it’s crucial for content negotiation, and how to tame the beast so your cachet soars in a relatively painless way.
<\/p>\n\n\n\n

What exactly is Vary<\/code>?<\/strong><\/h2>\n\n\n\n

The basic rule of thumb that everyone uses is: \u201cOne URL = One cached object.\u201d<\/p>\n\n\n\n

If a user requests \/home in a GET <\/code>request, the cache saves the response and serves it to the next request, as long as the response headers allow it.<\/p>\n\n\n\n

But the modern web is complex. Sometimes, the same URL needs to return different content depending on who requests it or how. Enter content negotiation.<\/em><\/p>\n\n\n\n

The Vary<\/code> response header tells the cache (and the browser):<\/p>\n\n\n\n

“Hey, to decide whether to serve this saved copy, don’t just look at the URL. Also look at these headers in the user’s request.”<\/em><\/p>\n\n\n\n

And what the hell is this going to do me any good, you might ask? Let’s get to the classic problem:<\/p>\n\n\n\n

The \u201cjack, queen and king\u201d: the compression of responses<\/strong><\/h2>\n\n\n\n

The most common case is Vary: Accept-Encoding<\/code>.<\/p>\n\n\n\n

    \n
  1. Browser A (supports gzip) requests \/styles.css. The server responds with the compressed file.<\/li>\n\n\n\n
  2. Browser B (Curl, for example) asks for \/styles.css:<\/li>\n<\/ol>\n\n\n\n

    If the cache ignored Vary<\/code>, it could deliver to browser B the compressed (gzip) version it saved from browser A. Result: the user sees garbage on the screen.<\/p>\n\n\n\n

    With Vary: Accept-Encoding<\/code>, the cache stores different variations of the same URL based on the client’s compression capabilities.<\/p>\n\n\n\n

    This mechanism is so common that most cache servers or CDNs use it by default. Furthermore, since there are few different “values” for Accept-Encoding<\/code>, the impact on the hit ratio is minimal.<\/p>\n\n\n\n

    The impact on the CHR: the attack of the clones<\/strong><\/h2>\n\n\n\n

    This is where we need to put on our systems architect hats. Every variation you add reduces the probability of a cache hit.<\/p>\n\n\n\n

    Let’s say that your backend application returns:<\/p>\n\n\n\n

    Vary: User-Agent<\/code><\/p>\n\n\n\n

    This is one of the most dangerous configurations. There are thousands of different User-Agent strings (Chrome version X, Y, Z; Safari on iOS 14, 15, etc.). If you use Vary: User-Agent<\/code> without any control, Varnish will save an identical copy of your page for every single version of every browser. And that’s not even considering Vary: cookie<\/code> or Vary: Accept-Language<\/code>. For those, you can just clear the cache and save yourself a step in your web suicide.<\/p>\n\n\n\n

    Consequences of this type of error:<\/p>\n\n\n\n

      \n
    1. Memory fill-up: If you’re using a shared cache, you’ll be evicting important content to fill the cache with duplicates. Your caching system had better be able to handle the increased memory usage, or you’ll fill it up very quickly.<\/li>\n\n\n\n
    2. Low Hit-Ratio: Most users will receive a MISS because their User-Agent varies slightly from the previous one.<\/li>\n\n\n\n
    3. Backend stress: your origin server will work much harder.<\/li>\n<\/ol>\n\n\n\n

      Common use cases (and how to handle them)<\/strong><\/h2>\n\n\n\n

      1. Accept-Encoding (Compression)<\/h3>\n\n\n\n

      It’s required if you serve dynamically compressed content. The good news is that most caching servers or CDNs do it by default.<\/p>\n\n\n\n