SocialCrawl

Rate Limits

600 requests per minute and 50 concurrent requests per API key, the headroom headers, and how to size a batch job

Every authenticated API key gets 600 requests per minute and 50 simultaneous requests. Both ceilings are per API key, both reject with a 429, and neither rejection costs credits. Volume beyond that is gated by your credit balance, not by a request quota.

The limits are the same on every plan. We do not sell higher rate limits, so there is no upgrade path to ask about: credits are the commercial lever.

The limits at a glance

LimitApplies toValueOn breach
Request rateAuthenticated API, per API key600 requests / minute429 RATE_LIMITED, with Retry-After and X-RateLimit-* headers
ConcurrencyAuthenticated API, per API key50 simultaneous requests429 CONCURRENCY_LIMIT, with a short Retry-After header
CreditsAuthenticated API, per accountYour remaining balance402 INSUFFICIENT_CREDITS. This is the real volume limit
Per-key credit limitAuthenticated API, per API keyOptional, you set it402 KEY_BUDGET_EXCEEDED. Only that key stops; the account is unaffected
JSON request bodyAuthenticated API, per request1 MB413 PAYLOAD_TOO_LARGE, rejected before parsing
Anonymous Explorer gateUnauthenticated /explore embed5 calls / UTC day / IP429, daily limit reached
API keysPer account5 active keysKey creation fails; revoke one first. See Authentication

Request rate: 600 per minute per key

Every authenticated request counts toward a sliding 1-minute window of 600 requests per API key. Cache hits count too: they cost no credits but still touch our infrastructure. The ceiling is set to stop runaway clients rather than to shape normal usage.

{
  "success": false,
  "error": {
    "type": "RATE_LIMITED",
    "message": "Request rate limit exceeded. Limit: 600 requests per minute. Honor the Retry-After header, then back off with jitter (see /docs/rate-limits).",
    "status": 429,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#rate-limited"
  },
  "credits_remaining": null,
  "request_id": "req-abc123"
}

The limiter runs after authentication and before credit deduction, so a RATE_LIMITED rejection never costs credits.

Concurrency: 50 in flight per key

Separately, one key may have at most 50 requests in flight at once. This is not a time window: a slot frees the instant one of your open requests returns. Send the 51st request while 50 are still open and it is rejected.

{
  "success": false,
  "error": {
    "type": "CONCURRENCY_LIMIT",
    "message": "Too many concurrent requests. Limit: 50. Honor the Retry-After header, then back off exponentially with jitter (see /docs/rate-limits).",
    "status": 429,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#concurrency-limit"
  },
  "credits_remaining": null,
  "request_id": "req-abc123"
}

This limiter rejects before any billing step runs, so nothing is deducted. credits_remaining is null on this response because the balance is never read.

Credits are the volume limit

There is no daily or monthly request cap beyond the 600 per minute window. Your credit balance is what bounds sustained work.

Flat endpoint costs run from 0 to 50 credits per call, and a set of composite endpoints is metered: they bill a per-request figure inside a published range that scales with how far the call fans out. Price every endpoint your job touches on Endpoint pricing before you size the job, and see Credits for how charging and refunds work.

Size a batch job by credits, not by an artificial request quota. 20,000 credits is 20,000 standard calls, run as fast as the 50-concurrency cap and the 600 per minute window allow.

Per-key credit limits

The account balance is shared by every key, so by default any one key can spend all of it. You can cap an individual key with a credit limit. Past that point the key returns 402 KEY_BUDGET_EXCEEDED and stops spending, while your other keys carry on untouched.

Reach for this when a test, CI, or staging key shares an account with production traffic. It is off by default, and a key with no limit behaves exactly as before. Set one in Dashboard → API Keys, and see Authentication for how the counter and resets work.

Anonymous Explorer gate

The unauthenticated Explorer embed on the marketing site is capped at 5 calls per UTC day per IP. This gate exists only on the anonymous proxy and does not apply to authenticated API keys. Sign up for a free key to remove it.

Headroom headers

Every response from the authenticated API carries headroom headers, so you or your agent can self-throttle before hitting the wall.

HeaderMeaning
X-RateLimit-LimitRequest-window cap (600 per minute).
X-RateLimit-RemainingRequests still available in the current sliding window.
X-RateLimit-ResetUnix epoch seconds when the window resets. Omitted if the limiter is degraded.
X-Concurrency-LimitThe concurrency cap (50).
X-Concurrency-RemainingConcurrency slots still free as this request entered.
Retry-AfterOn a 429 only. Seconds to wait before retrying.

X-RateLimit-* and X-Concurrency-* are not aliases of each other. The first is the requests-per-window limiter, the second is the in-flight cap. A client that conflates them self-throttles against the wrong ceiling.

On RATE_LIMITED, Retry-After is the number of seconds until the window resets. On CONCURRENCY_LIMIT it is a short static hint of 1 second, because a slot can free the moment any in-flight request returns.

If the limiter's backing store is briefly unavailable we fail open: nothing is throttled, and X-RateLimit-Remaining reports the nominal cap rather than a measured figure. A missing X-RateLimit-Reset is the signal to distrust the other two on that response. The headroom headers are also absent on 401 responses, which are rejected before the limiter runs.

How to handle a 429

Read error.type

RATE_LIMITED means slow down your request rate. CONCURRENCY_LIMIT means narrow your worker pool. They need opposite fixes.

Honor Retry-After

Wait at least that many seconds before the first retry.

Then back off exponentially with jitter

If a retry also gets a 429, double the wait and add random jitter, so a burst of your own workers does not resynchronize and stampede the cap together.

Cap the retries

Give up after a handful of attempts and surface the error rather than retrying forever.

A drop-in TypeScript helper:

async function callWithBackoff(
  url: string,
  init: RequestInit,
  maxRetries = 5,
): Promise<Response> {
  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, init);

    // Only 429s are retryable here. 402 (out of credits) and other
    // 4xx client errors are not: retrying them just wastes time.
    if (res.status !== 429 || attempt >= maxRetries) return res;

    const retryAfter = Number(res.headers.get("Retry-After")) || 1;
    // Exponential backoff with full jitter, seeded by Retry-After.
    const backoff = retryAfter * 2 ** attempt;
    const jittered = backoff * (0.5 + Math.random() * 0.5);
    await new Promise((r) => setTimeout(r, jittered * 1000));
  }
}

Sizing a batch job

To drain a large account safely:

  • Keep your worker pool at or under 50 concurrent requests.
  • Stay under 600 requests per minute.
  • Watch X-Concurrency-Remaining and X-RateLimit-Remaining to stay off the wall.
  • Budget credits per page, not per job.
  • Loop your paginated endpoints until has_more is false.

The Recipes page has a full drain-an-account loop.

Next steps