Rate limits and how not to hit them

The Content API is generous, but it is not a bottomless pit. A few simple limits keep it fast for everyone — and a site that caches sensibly will never meet them. Think of this page as the speed signs on a road you will almost always drive well below the limit.

#The limits

#Fixed one-minute windows

Counters work in fixed one-minute windows, not a sliding average. Requests fill the counter, and when the minute is over it starts again from zero. Two practical consequences:

• A burst at the very start of a minute is fine as long as the total for that minute stays under the limit.
• Once you get a 429, nothing you do in the same minute will help. Waiting until the next one will.

#Which limit you will meet first

It depends on where your requests come from:

• Server-side rendering on one machine. Every request leaves from your server's IP, so the 300 per IP is the ceiling you would reach first. Several servers each have their own 300, but together they still share the key's 600.
• Fetching in the browser. Every visitor has their own IP, so the per-IP limit almost never matters. But all visitors share one site key, and the key gets 600 per minute for the whole site. The browser cache helps each visitor individually, not the crowd.

#What a 429 looks like

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Cache-Control: no-store

{"message":"rate_limit_exceeded"}

That is the whole thing. There is no Retry-After header and no RateLimit-* headers either — so don't look for them. Because the windows are fixed minutes, the rule is simple: wait about a minute (with a little randomness, see below) and try again.

#Retry with backoff and jitter

A small wrapper around fetch that does the right thing for each kind of failure:

• 429 — sit out the window: about 60 seconds plus a few random seconds.
• 5xx and network errors — exponential backoff with jitter: roughly 0.5–1 s, then 1–2 s, then 2–4 s.
• Any other 4xx — no retries at all. Waiting will not fix a typo in a marker.

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

type RetryOptions = {
  retries?: number;   // how many extra attempts
  maxWaitMs?: number; // never sleep longer than this; return the error instead
};

export async function fetchWithRetry(
  url: string,
  init: RequestInit = {},
  { retries = 3, maxWaitMs = 70_000 }: RetryOptions = {},
): Promise<Response> {
  for (let attempt = 0; ; attempt++) {
    let res: Response | undefined;
    let error: unknown;

    try {
      res = await fetch(url, init);
      // 2xx, 304, or a 4xx that waiting won't fix (400, 401, 404…): hand it back as is
      if (res.status !== 429 && res.status < 500) return res;
    } catch (err) {
      error = err; // network trouble: DNS, timeout, connection reset
    }

    const wait =
      res?.status === 429
        ? 60_000 + Math.random() * 5_000             // the window is a minute: wait it out, plus jitter
        : 2 ** attempt * 500 * (1 + Math.random());  // 5xx / network: 0.5–1 s, 1–2 s, 2–4 s…

    if (attempt >= retries || wait > maxWaitMs) {
      if (res) return res; // let the caller see the 429 / 5xx
      throw error;
    }
    await sleep(wait);
  }
}

Use it patiently where nobody is waiting, and impatiently where a person is:

const headers = { 'x-crm-key': process.env.CRM_KEY! };
const url = 'https://back.sitecog.com/content/v1/pages/home?lang=en';

// A build script or a background job: happy to wait out a 429
const res = await fetchWithRetry(url, { headers });

// Rendering a page for a visitor: nobody waits a minute for a page.
// A 429 comes straight back, and you serve the copy you cached earlier.
const quick = await fetchWithRetry(url, { headers }, { retries: 1, maxWaitMs: 2_000 });

#The SSR maths: how many requests does your server make?

Let's count. Say a typical page render fetches two things: the page itself (/v1/pages/about) and the shared header and footer (/v1/pages/common). That is two requests per page view. The site has 30 pages in 2 languages.

The last row is the whole point: with a server-side cache, each unique URL is fetched at most once per minute, so your API usage is capped by the size of your site, not by its popularity. Going viral stops being an infrastructure problem.

#Watch your builds

Static generation is a burst by nature: a build that renders 400 pages in 2 languages in parallel can fire 800 requests in a few seconds from one machine — well over 300 per minute. Limit concurrency to a handful of pages at a time, and use the patient fetchWithRetry above so that a 429 makes the build pause for a minute instead of failing.

#How to stay far away from the limits

1. Cache on your side. revalidate in Next.js, ISR, or a small in-memory cache on any Node server. Recipes are on the Caching page.
2. One page request instead of many blocks. /v1/pages/:marker returns every section and block in one go. Ordering a pizza one slice at a time is fun only for the courier.
3. Fetch shared content once. Header and footer blocks on a common page are the same for every route — let one cached request serve them all.
4. Use ETags. They do not reduce the number of requests, but they make each repeat request nearly free in bandwidth. See ETag and 304.
5. Don't poll. Asking the API every second whether something changed is exactly the traffic limits exist for. A 60-second cache gives editors a fast enough feedback loop, and Live mode gives them an instant one.
6. Keep keys tidy. An old preview deployment or a forgotten cron job with a revoked key, running on the same server as production, can burn the 20 bad-key requests in a minute — and then production on that IP gets 429 too. One bad apple really does spoil the bunch.

curl -s -o /dev/null -w "%{http_code}\n" \
  "https://back.sitecog.com/content/v1/langs" -H "x-crm-key: pk_3f9c2a7e1b4d4c0e8a6f5b2d9c1e7a40"
# 200 — all good, 401 — fix the key before anything starts retrying it

Got a 429 anyway and want to know what else can go wrong? Every status is on the Errors page.