Rate Limits

SocialCrawl keeps throttling simple. Volume is gated by credits, with two hard request controls on every authenticated /v1 key: a requests-per-minute window and a concurrency cap. Both 429s are unbilled.

The limits

1. Request rate: 600 / minute per key

Every authenticated request counts toward a sliding 1-minute window of 600 requests per API key, including cache hits (they are free of credit cost but still load Redis/Postgres). The default is intentionally generous so legitimate traffic never notices it; it exists to stop runaway clients, not to shape normal product usage.

{
  "success": false,
  "error": {
    "type": "RATE_LIMITED",
    "message": "Request rate limit exceeded. Limit: 600 requests per minute. Honor the Retry-After header, then back off with jitter (see /docs/rate-limits).",
    "status": 429,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#rate-limited"
  },
  "credits_remaining": null,
  "request_id": "req-abc123"
}

A 429 RATE_LIMITED never costs credits - the limiter runs after auth and before credit deduction.

2. Concurrency: 50 in-flight requests per key

Separately, a key may have at most 50 requests in flight at once. This is a concurrency limit, not a time window: a slot frees the instant one of your in-flight requests returns. Send the 51st request while 50 are still open and it is rejected:

{
  "success": false,
  "error": {
    "type": "CONCURRENCY_LIMIT",
    "message": "Too many concurrent requests. Limit: 50. Honor the Retry-After header, then back off exponentially with jitter (see /docs/rate-limits).",
    "status": 429,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#concurrency-limit"
  },
  "credits_remaining": null,
  "request_id": "req-abc123"
}

A concurrency 429 never costs you credits - the limiter rejects the request before any billing step runs, so nothing is deducted in the first place. (credits_remaining is null on this response because the balance is never read.)

3. Credits: the real volume limit

There is no daily or monthly request cap beyond the 600/min window. Your credit balance is the volume limit for sustained work. Flat endpoint costs run from 0 to 50 credits per call, and a set of composite endpoints is metered — they bill a per-request figure inside a published range that scales with how far the call fans out. Price every endpoint your job touches from Endpoint Pricing before you size the job; see Credits for how charging and refunds work. Size a batch job by credits, not by an artificial request quota: 20,000 credits is 20,000 standard calls, run them as fast as the 50-concurrency cap and 600/min window allow.

4. Per-key credit limits: an optional cap you set

The balance above is shared by every key on the account, so by default any one key can spend all of it. You can cap an individual key with a credit limit, after which that key returns 402 KEY_BUDGET_EXCEEDED and stops spending — while your other keys carry on untouched.

This is the control to reach for when a test, CI, or staging key shares an account with production traffic. It is off by default; a key with no limit behaves exactly as it always has. Set one in Dashboard → API Keys, and see Authentication for how the counter and resets work.

5. Anonymous Explorer gate: 5 / day / IP

The unauthenticated Explorer embed on the marketing site is capped at 5 calls per UTC day per IP. This gate exists only on the anonymous proxy - it does not apply to authenticated API keys. Sign up for a free key to remove it; you get a credit balance and only the per-key rate and concurrency limits apply.

Rate-limit headers

Every response from the authenticated API carries live headroom headers so you (or an agent) can self-throttle before hitting the wall:

These two families are not aliases of each other. X-RateLimit-* is the requests-per-window limiter; X-Concurrency-* is the in-flight cap. Generic clients that only understand X-RateLimit-* now read the real rate limit.

On RATE_LIMITED, Retry-After is the seconds until the window resets. On CONCURRENCY_LIMIT, Retry-After is a short static hint (1 second) because a slot can free the moment any in-flight request returns.

If the limiter's backing store is briefly unavailable we fail open: nothing is throttled, and X-RateLimit-Remaining reports the nominal cap rather than a measured figure. A missing X-RateLimit-Reset is the signal to distrust the other two on that response. The headroom headers are also absent on 401 responses, which are rejected before the limiter runs.

Retry guidance

When you get a 429:

1. Read error.type. RATE_LIMITED means slow down your request rate; CONCURRENCY_LIMIT means narrow your worker pool.
2. Honor Retry-After. Wait at least that many seconds before the first retry.
3. Then back off exponentially with jitter. If a retry also 429s, double the wait and add random jitter so a burst of your own workers does not resynchronize and stampede the cap together.
4. Cap the retries. Give up after a handful of attempts and surface the error.

A drop-in TypeScript helper:

async function callWithBackoff(
  url: string,
  init: RequestInit,
  maxRetries = 5,
): Promise<Response> {
  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, init);

    // Only 429s are retryable here. 402 (out of credits) and other
    // 4xx client errors are not - retrying them just wastes time.
    if (res.status !== 429 || attempt >= maxRetries) return res;

    const retryAfter = Number(res.headers.get("Retry-After")) || 1;
    // Exponential backoff with full jitter, seeded by Retry-After.
    const backoff = retryAfter * 2 ** attempt;
    const jittered = backoff * (0.5 + Math.random() * 0.5);
    await new Promise((r) => setTimeout(r, jittered * 1000));
  }
}

Sizing a batch job with confidence

To drain a large account safely: keep your worker pool at or under 50 concurrent requests, stay under 600 requests per minute, watch X-Concurrency-Remaining and X-RateLimit-Remaining to stay off the wall, budget credits per page, and loop your paginated endpoints until has_more is false. See the Recipes for a full "drain an account" loop.

Which errors are safe to retry - and which are not - is covered on the Error Handling page, which also ships copy-paste backoff-with-jitter loops in cURL, Python, and Node that honor Retry-After.