Rate Limits

SocialCrawl deliberately keeps throttling simple. There is no requests-per-second or requests-per-minute window on the authenticated API. Instead there are two limits, and the credit system is the real ceiling on volume.

The three limits

1. Concurrency: 50 in-flight requests per key

The only throttle on request rate is a cap of 50 requests in flight at once, per API key. This is a concurrency limit, not a time window: a slot frees the instant one of your in-flight requests returns. Send the 51st request while 50 are still open and it is rejected:

{
  "success": false,
  "error": {
    "type": "CONCURRENCY_LIMIT",
    "message": "Too many concurrent requests. Limit: 50. Honor the Retry-After header, then back off exponentially with jitter (see /docs/rate-limits).",
    "status": 429,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#concurrency-limit"
  },
  "credits_remaining": 1234,
  "request_id": "req_abc123"
}

Because credits are refunded on upstream failure, a 429 here never costs you credits — the request never reached upstream.

2. Credits: the real volume limit

There is no daily or monthly request cap. Your credit balance is the volume limit. Each endpoint costs 1, 5, 10, or 20 credits (see Endpoint Pricing and Credits). Size a batch job by credits, not by an artificial request quota: 20,000 credits is 20,000 standard calls, run them as fast as the 50-concurrency cap allows.

3. Anonymous Explorer gate: 5 / day / IP

The unauthenticated Explorer embed on the marketing site is capped at 5 calls per UTC day per IP. This gate exists only on the anonymous proxy — it does not apply to authenticated API keys. Sign up for a free key to remove it; you get a credit balance and only the 50-concurrency cap applies.

Rate-limit headers

Every response from the authenticated API carries live headroom headers so you (or an agent) can self-throttle before hitting the wall — not by discovering the cap the hard way:

Retry-After is a short static hint (1 second) because this is a concurrency cap, not a fixed window: a slot can free the moment any in-flight request returns.

Retry guidance

When you get a 429:

1. Honor Retry-After. Wait at least that many seconds before the first retry.
2. Then back off exponentially with jitter. If a retry also 429s, double the wait and add random jitter so a burst of your own workers does not resynchronize and stampede the cap together.
3. Cap the retries. Give up after a handful of attempts and surface the error; a sustained 429 means your worker pool is wider than 50 — narrow it instead.

A drop-in TypeScript helper:

async function callWithBackoff(
  url: string,
  init: RequestInit,
  maxRetries = 5,
): Promise<Response> {
  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, init);

    // Only concurrency 429s are retryable here. 402 (out of credits) and
    // 4xx client errors are not — retrying them just wastes time.
    if (res.status !== 429 || attempt >= maxRetries) return res;

    const retryAfter = Number(res.headers.get("Retry-After")) || 1;
    // Exponential backoff with full jitter, seeded by Retry-After.
    const backoff = retryAfter * 2 ** attempt;
    const jittered = backoff * (0.5 + Math.random() * 0.5);
    await new Promise((r) => setTimeout(r, jittered * 1000));
  }
}

Sizing a batch job with confidence

To drain a large account safely: keep your worker pool at or under 50 concurrent requests, watch X-Concurrency-Remaining to stay off the wall, budget credits per page, and loop your paginated endpoints until has_more is false. See the Recipes for a full "drain an account" loop.

Which errors are safe to retry — and which are not — is covered on the Error Handling page.