OpenRouter Free-Tier 429s: Automatic Fallback and Retry-After Done Right

If you build on OpenRouter's :free models, you will eventually see 429 Rate limit exceeded. Retrying in a loop doesn't fix it, and neither does adding more OpenRouter keys. This guide covers what the limits are (from OpenRouter's own docs), which retries help and which make things worse, and how to fail over to a different provider without hammering one that has asked you to wait. There's a do-it-yourself version and a gateway version, and both have working code.

As of 2026-10-20. Limits change, so check the linked OpenRouter pages before you rely on the numbers.

What the free-tier limits are

OpenRouter's Limits page lists the caps for model IDs ending in :free:

Credits purchased (all time)	Requests per minute	Requests per day
Less than 10	20	50
At least 10	20	1000

Three details from the same page matter more than the table:

1. More keys or accounts do not help. OpenRouter: "Making additional accounts or API keys will not affect your rate limits, as we govern capacity globally." Rotating keys on one account doesn't get around a free-tier cap.
2. A 429 can come from two places. It can be OpenRouter's own platform limit (free-model requests per minute or per day, or DDoS protection), or the upstream provider serving that model being at capacity. OpenRouter already retries other providers for the same model before the error reaches you.
3. Mid-stream limits don't come back as HTTP 429. If the limit hits after streaming has started, the error arrives as an SSE event with finish_reason: "error", because the 200 status has already been sent. Your stream parser has to check for it.

You can check your daily free-model counter with GET https://openrouter.ai/api/v1/key (the free_model_daily_requests field). The per-minute limit isn't reported there.

Retries that help, and retries that hurt

A 429 tells you to wait. How long to wait, and whether to wait at all, depends on which limit you hit:

Situation	What to do
Per-minute limit (20 rpm)	Wait. Honor Retry-After when it's present, otherwise back off exponentially with jitter. Retrying is fine.
Daily free-model cap reached	Don't retry the same free model today. Every :free model on that account shares the cap, and OpenRouter counts it per UTC day. Switch models or providers.
Upstream provider at capacity	OpenRouter's model fallbacks (models: [...]) can move you to another model.
Every option is cooling down	Return a fast, honest error with a Retry-After to your caller instead of holding the request open.

Two traps:

• OpenRouter's own models fallback stays inside OpenRouter. It's useful against one model's providers being down. But if you hit the account-wide free cap, falling back to another :free model hits the same cap. OpenRouter also notes that if a :free model is followed by a paid model in models, you're billed for the paid one.
• Hot-loop retries make it worse. If a client retries a 429 immediately, the request it just failed gets tried again, and that uses up the per-minute window for nothing.

So the fix that actually helps is a cross-provider fallback (for example Groq or Gemini on their own keys), plus memory: once a model returns 429, stop sending it traffic until its cooldown is over.

Option A: do it yourself (OpenAI SDK, no gateway)

This works against any OpenAI-compatible endpoint. It keeps a cooldown per model, honors Retry-After, and falls back across providers in your preferred order.

# pip install openai
import os, time
from openai import OpenAI, RateLimitError, APIStatusError

PROVIDERS = [  # (name, client, model) in preference order
    ("openrouter", OpenAI(base_url="https://openrouter.ai/api/v1", api_key=os.environ["OPENROUTER_API_KEY"], max_retries=0),
     "meta-llama/llama-3.3-70b-instruct:free"),
    ("groq", OpenAI(base_url="https://api.groq.com/openai/v1", api_key=os.environ["GROQ_API_KEY"], max_retries=0),
     "llama-3.3-70b-versatile"),
]
cooldown_until: dict[str, float] = {}  # "provider/model" -> unix time

def retry_after_seconds(err: APIStatusError, default: float = 60.0) -> float:
    value = err.response.headers.get("retry-after") if err.response is not None else None
    try:
        return max(1.0, float(value))
    except (TypeError, ValueError):
        return default  # no (or an HTTP-date) Retry-After: assume a minute

def chat(messages):
    last_error = None
    for name, client, model in PROVIDERS:
        key = f"{name}/{model}"
        if time.time() < cooldown_until.get(key, 0):
            continue  # this model asked us to wait; don't spend a request proving it again
        try:
            return client.chat.completions.create(model=model, messages=messages)
        except RateLimitError as e:
            cooldown_until[key] = time.time() + retry_after_seconds(e)
            last_error = e
        except APIStatusError as e:
            if e.status_code < 500:
                raise  # 400/401/403: our request is wrong; another provider won't fix it
            last_error = e
    raise RuntimeError("every provider is cooling down; retry later") from last_error

print(chat([{"role": "user", "content": "Reply with exactly: pong"}]).choices[0].message.content)

max_retries=0 matters: the OpenAI SDK retries 429s on its own by default (twice, waiting out Retry-After), so without it each "fallback" first sits through the SDK's retries against the provider that's limiting you. That's about 40 lines. In production you'll also want the cooldowns shared across processes (Redis, a database), a separate longer cooldown for the daily cap, multiple keys per provider for paid tiers, mid-stream error handling, and a sensible order for the fallback list. That's where a gateway earns its keep.

Option B: let a gateway do it (SentioGate)

SentioGate is an OpenAI-compatible, bring-your-own-key (BYOK) router. You keep your OpenRouter, Groq, Gemini, Mistral, DeepSeek or xAI keys in its encrypted Vault (or pass them per request), send model: "auto", and it handles the failover. Here's what it does on a 429, as implemented today:

1. Remembers the cooldown. A 429 puts that model on that key into a cooldown that's shared across requests. It honors the upstream Retry-After, capped at 2 minutes, and defaults to 1 minute when there's no Retry-After.
2. Recognizes the daily free-model cap. When OpenRouter's error says the daily free-model limit was hit, all :free models on that key cool down until 00:00 UTC (or the Retry-After, if one is sent), instead of being tried one by one.
3. Tries your next key for the same model, then (with auto) the next model. Same model on your next key first (useful for paid keys and other providers; OpenRouter itself says extra keys don't raise its free limits). With model: "auto" it then moves to the next-ranked model in your enabled pool, which can be on another provider, skipping models that are cooling down. A pinned provider/model only gets key failover; it never silently switches models. On the hosted service an auto request makes at most 2 model attempts, inside a 30-second budget.
4. Fails fast and honestly. If nothing is available, you get 429 with code: "upstream_rate_limited" and a Retry-After header. The request isn't left hanging.

auto only routes to models you've enabled in the Vault's model list, so enable the :free variants you want there, next to your Groq or Gemini models.

Python (SDK 1.2.0)

# pip install ai-auto-router-client==1.2.0
import os
from ai_auto_router_client import AutoRouterClient, RouterClientError

router = AutoRouterClient(
    api_key=os.environ["SENTIOGATE_API_KEY"],  # sg_live_...
    max_retries=2,       # off by default; retries only transient 429s, only after Retry-After (>= 1 s)
    max_retry_wait=30,   # never sleeps longer than this; a longer Retry-After raises instead
)
try:
    resp = router.chat(messages=[{"role": "user", "content": "Reply with exactly: pong"}])  # model="auto"
    print(resp["model"], resp["choices"][0]["message"]["content"])
    print(resp.get("router", {}).get("attempts"))  # BYOK: which provider/model attempts were made
except RouterClientError as e:
    print(e.status_code, e.code, e.retry_after)  # e.g. 429 upstream_rate_limited 60.0

JavaScript / TypeScript (SDK 1.2.0, streaming)

// npm i @hushamdev/ai-auto-router-sdk@1.2.0
import { AutoRouter, RouterClientError } from '@hushamdev/ai-auto-router-sdk';

const router = new AutoRouter({ apiKey: process.env.SENTIOGATE_API_KEY, maxRetries: 2, maxRetryWaitMs: 30000 });
try {
  for await (const chunk of router.streamChat({ messages: [{ role: 'user', content: 'Count from 1 to 5.' }] })) {
    process.stdout.write(chunk.choices?.[0]?.delta?.content || '');
  }
} catch (e) {
  // 429s and 403s arrive before the first chunk, so you can still retry or fall back cleanly.
  if (e instanceof RouterClientError) console.error(e.statusCode, e.code, e.retryAfter);
  else throw e;
}

Any OpenAI client

import os
from openai import OpenAI
client = OpenAI(base_url="https://api.sentiogate.dev/v1", api_key=os.environ["SENTIOGATE_API_KEY"])
r = client.chat.completions.create(model="auto", messages=[{"role": "user", "content": "hi"}])

The OpenAI SDK has its own retry logic for 429s. If you also set SentioGate SDK retries, pick one layer to do the retrying, not both.

No keys yet? The community pool, honestly

With a SentioGate API key and no provider keys, requests go to a free community pool. During the beta it is one fixed model (reported as sentiogate-community), shared by everyone, with limits of 150 requests per 5 hours per user and 1,500 per 5 hours across all users. When its upstream rate-limits, every caller gets a fast 429 upstream_rate_limited with Retry-After set to the time left on the shared cooldown (60 s to 15 min). There's no failover on the pool, because there's nothing to fail over to. It's good for trying the API, not for production traffic. Add your own keys for that.

Checklist

• Parse Retry-After; never retry a 429 sooner than it says.
• Treat the daily free cap as "stop using free models on this account until reset", not "retry later".
• Remember cooldowns across requests, not just inside one retry loop.
• Fall back across providers. OpenRouter-only fallbacks share the same free cap.
• Handle mid-stream errors (finish_reason: "error") in your SSE parser.
• Have exactly one layer doing retries.

Sources

• OpenRouter, API Credit & Rate Limits (accessed 2026-10-20)
• OpenRouter, Model Fallbacks (accessed 2026-10-20)
• SentioGate SDKs: ai-auto-router-client on PyPI, @hushamdev/ai-auto-router-sdk on npm
• SentioGate API reference