Skip to content

Rate limits

Current behavior

pokkit does not enforce its own rate limits. All rate limiting is handled by the upstream provider for the model you requested.

If you hit a rate limit, you will receive a 429 Too Many Requests response passed through directly from the upstream:

Provider Rate limit docs
Anthropic docs.anthropic.com/en/api/rate-limits
Together AI docs.together.ai/reference/rate-limits
OpenRouter openrouter.ai/docs/limits

When you receive a 429, retry with exponential backoff. A safe default:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import time
import random

def call_with_backoff(client, **kwargs, max_retries=5):
    for attempt in range(max_retries):
        try:
            return client.chat.completions.create(**kwargs)
        except Exception as e:
            if "429" not in str(e) or attempt == max_retries - 1:
                raise
            wait = (2 ** attempt) + random.uniform(0, 1)
            time.sleep(wait)

Planned: per-client rate limits

A future release will add configurable per-client rate limits enforced at the pokkit layer, before the request reaches the upstream provider. This will allow you to:

  • Set a requests-per-minute cap per API token
  • Get consistent 429 responses regardless of which upstream was targeted
  • Avoid surprise upstream billing from runaway clients

Subscribe to the changelog for updates.