Caching, ETag and 304: fast and always fresh

Content APIs usually make you choose: fast or fresh. We would rather not. Responses are cached on our side yet never go stale there, every response carries an ETag so unchanged content costs you almost nothing, and a CRM edit reaches the API the moment the editor hits “Save”. This page explains how that works and how to get the most out of it on your side.

#From a CRM edit to your page

Here is the whole journey of a change, from the editor's keyboard to your visitor's screen:

1. The editor saves

   Someone fixes a typo in the CRM or right on the live site. Any edit counts — a text, a picture, a page setting, a blog post.

2. The site's content version goes up

   Every edit bumps the content version of the site. Our server-side cache (Redis) is tied to that version, so all the old cached answers become irrelevant at once. Nobody has to “clear the cache”.

3. The next API request gets fresh data

   The very next request to the API builds its response from the new content. On our side there is no delay at all: no TTL to wait out, no purge queue.

4. Caches between us and the visitor catch up

   What a visitor actually sees can lag a little: the browser or a CDN may keep the previous response for up to 60 seconds, and may show it once more while quietly fetching the new one in the background (that is stale-while-revalidate). Your own server cache, if you have one, adds its own lifetime on top.

#Cache headers explained

A typical successful response comes with these headers:

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: public, max-age=60, stale-while-revalidate=600
ETag: W/"a41f9c0e7b2d58f3"
Vary: x-crm-key, Accept-Encoding
Access-Control-Allow-Origin: *

CORS is open to any origin, the API explicitly allows the If-None-Match request header and exposes ETag to JavaScript — so everything on this page works from the browser too.

#ETag and 304 Not Modified

The ETag is the cheapest way to ask “has anything changed?”. Remember the ETag from the last response, send it in If-None-Match next time, and if the content is the same you get 304 Not Modified with an empty body. Your code keeps using the copy it already has.

GET /content/v1/pages/home?lang=en HTTP/1.1
Host: back.sitecog.com
x-crm-key: pk_3f9c2a7e1b4d4c0e8a6f5b2d9c1e7a40

HTTP/1.1 200 OK
Cache-Control: public, max-age=60, stale-while-revalidate=600
ETag: W/"a41f9c0e7b2d58f3"
Vary: x-crm-key, Accept-Encoding

{ "id": 26, "marker": "home", "name": "Home", "content": { … } }

GET /content/v1/pages/home?lang=en HTTP/1.1
Host: back.sitecog.com
x-crm-key: pk_3f9c2a7e1b4d4c0e8a6f5b2d9c1e7a40
If-None-Match: W/"a41f9c0e7b2d58f3"

HTTP/1.1 304 Not Modified
Cache-Control: public, max-age=60, stale-while-revalidate=600
ETag: W/"a41f9c0e7b2d58f3"

As soon as an editor changes something on that page, the body changes, the hash changes, and the same request returns a fresh 200 with a new ETag.

#Caching recipes

#In the browser: already done

If you fetch content right in the browser, you do not need a single line of caching code. A plain fetch uses the browser's HTTP cache: for 60 seconds it reuses the response, after that it revalidates with If-None-Match all by itself.

const res = await fetch('https://back.sitecog.com/content/v1/pages/home?lang=en', {
  headers: { 'x-crm-key': 'pk_3f9c2a7e1b4d4c0e8a6f5b2d9c1e7a40' },
  // cache: 'default' is the default — the browser handles max-age and ETag for you
});
const page = await res.json();

Do not be surprised that your code never sees a 304: the browser swaps it for the cached 200 before it reaches you. Open the Network tab in DevTools to see what really happened — that is where you will find the 304 or “disk cache”.

#On a Node server: a tiny ETag cache

Server-side fetch in Node has no HTTP cache of its own, so every call goes all the way to the API. A small Map fixes that: reuse the copy for 60 seconds (just like max-age), then ask with If-None-Match and download the body only if it changed.

const API = 'https://back.sitecog.com/content';
const TTL = 60_000; // same as max-age=60

type Entry = { etag: string | null; data: unknown; at: number };
// One process, one site key. Using several keys? Put the key into the cache key too.
const cache = new Map<string, Entry>();

export async function getContent<T>(path: string): Promise<T> {
  const url = API + path;
  const cached = cache.get(url);

  // 1. Fresh enough — do not even call the API
  if (cached && Date.now() - cached.at < TTL) return cached.data as T;

  // 2. Ask "has it changed?" with the ETag we already have
  const headers: Record<string, string> = { 'x-crm-key': process.env.CRM_KEY! };
  if (cached?.etag) headers['if-none-match'] = cached.etag;

  const res = await fetch(url, { headers });

  if (res.status === 304 && cached) {
    cached.at = Date.now(); // same content, another 60 seconds of peace
    return cached.data as T;
  }
  if (!res.ok) throw new Error(`Content API ${res.status} for ${path}`);

  const data = (await res.json()) as T;
  cache.set(url, { etag: res.headers.get('etag'), data, at: Date.now() });
  return data;
}

// usage
const home = await getContent('/v1/pages/home?lang=en');

#Next.js: revalidate and on-demand refresh

In the App Router the fetch cache does the work for you. revalidate: 60 matches our max-age: Next.js keeps the response for a minute, then refreshes it in the background.

export default async function Home() {
  const res = await fetch('https://back.sitecog.com/content/v1/pages/home?lang=en', {
    headers: { 'x-crm-key': process.env.CRM_KEY! },
    next: { revalidate: 60, tags: ['crm-content'] },
  });
  const page = await res.json();

  return <h1>{page.content.hero.content.hero_title.content.en}</h1>;
}

Want a shorter lag? Lower revalidate — but keep an eye on the limits. Want a “publish now” button for a big launch? Tag your fetches and expose a small route that drops the tag:

import { revalidateTag } from 'next/cache';

export async function POST(req: Request) {
  if (req.headers.get('x-revalidate-secret') !== process.env.REVALIDATE_SECRET) {
    return new Response('Nope', { status: 401 });
  }
  // Next.js 16 takes a second argument; in Next.js 15 it is just revalidateTag('crm-content')
  revalidateTag('crm-content', { expire: 0 });
  return Response.json({ revalidated: true });
}

Call it from wherever suits your team: a deploy script, a bookmarklet, a chat bot, a button in your own admin panel. The secret is yours alone — it has nothing to do with the site key.

#Live mode: skip every cache

When an editor opens your site in the CRM Live mode, they want to see their change right after saving — not a minute later. Inside the CRM frame the URL gets ?crm_live=1. Good practice (and exactly what our reference client does): when the page is inside an iframe or has crm_live, fetch with cache: 'no-store'. Our side is already fresh, so that is all it takes.

const isLive =
  window.self !== window.top ||
  new URLSearchParams(location.search).has('crm_live');

const res = await fetch('https://back.sitecog.com/content/v1/pages/home?lang=en', {
  headers: { 'x-crm-key': 'pk_3f9c2a7e1b4d4c0e8a6f5b2d9c1e7a40' },
  cache: isLive ? 'no-store' : 'default',
});

type Props = { searchParams: Promise<{ crm_live?: string }> };

export default async function Home({ searchParams }: Props) {
  const live = (await searchParams).crm_live === '1';

  const res = await fetch('https://back.sitecog.com/content/v1/pages/home?lang=en', {
    headers: { 'x-crm-key': process.env.CRM_KEY! },
    // editors get fresh data on every request, visitors get the cached copy
    ...(live ? { cache: 'no-store' as const } : { next: { revalidate: 60 } }),
  });
  const page = await res.json();
  // …
}

More about Live mode and the markup behind it is on the Widget & markup page.

#“Why don't I see my change?”

The most popular question about any cache, ever. Go through this list top to bottom — it runs from “ten seconds to check” to “make yourself a cup of tea”.

1. Ask the API directly. A plain curl has no cache, and our side is always fresh. If curl shows the new text, the API is fine and the old copy lives in a cache somewhere on the way. If curl shows the old text, keep going down the list.
2. Was it actually saved? Check in the CRM. For blog posts: drafts are never returned, and a post scheduled for the future stays hidden until its date (and may appear a few minutes late).
3. Right site? A key belongs to exactly one site. Staging and production with different keys read different content.
4. Right language? There is no server-side fallback. If the editor changed the German text and you render English, nothing visible happens. An empty translation comes back as "", and your fallback code may quietly show another language instead. See Languages & fallbacks.
5. Right block? Markers are case-sensitive, and block markers are unique only within a section — /v1/blocks/title without ?section= returns the oldest block with that marker, which may not be the one that was edited.
6. Hidden section? Sections with show: false are still returned. If the editor hid a section and it is still on the site, your template is not checking show.
7. Your own cache. revalidate, ISR, an in-memory map, a CDN in front of your site, a page built once at build time. This is the usual suspect.
8. The browser. Up to 60 seconds plus one background refresh. A hard reload (Ctrl+Shift+R or Cmd+Shift+R) settles it.
9. Editing in Live mode? Make sure that inside the CRM frame you fetch with cache: 'no-store' (see above).

curl -i "https://back.sitecog.com/content/v1/pages/home?lang=en" \
  -H "x-crm-key: pk_3f9c2a7e1b4d4c0e8a6f5b2d9c1e7a40"