# Error Handling (/docs/errors) Error Handling [#error-handling] All errors follow the same envelope format: ```json { "success": false, "error": { "type": "INSUFFICIENT_CREDITS", "message": "Your account has 0 credits remaining. This endpoint requires 1 credits.", "status": 402, "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits" }, "credits_used": 0, "credits_remaining": 0, "request_id": "req-abc123" } ``` Every error envelope carries `credits_used` — the **net** credits charged for the failed request. Error paths deduct then refund, so this is `0` in effectively every case. Read it to confirm a failure cost you nothing rather than inferring it from the refund rules below. Every response — success or error — includes an `X-Request-Id` header matching `request_id`. Include it when contacting support so we can trace the exact request in our logs. The `doc_url` on every error points at the matching section on this page. Jump straight to a code with its anchor, e.g. `#insufficient-credits` or `#upstream-error`. Error Codes [#error-codes] | Code | Status | Retryable | Description | Action | | ------------------------------------------------------------------------------- | ------ | ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------- | | [`MISSING_API_KEY`](#missing-api-key) | 401 | No | No `x-api-key` header on the request | Add the `x-api-key` header | | [`INVALID_API_KEY`](#invalid-api-key) | 401 | No | Key is malformed, not found, revoked, or expired | Check the key in Dashboard → API Keys | | [`INSUFFICIENT_CREDITS`](#insufficient-credits) | 402 | No | Balance is lower than the endpoint cost | Top up in Dashboard → Billing | | [`KEY_BUDGET_EXCEEDED`](#key-budget-exceeded) | 402 | No | This key has spent its own per-key credit limit. The account balance is untouched | Raise or reset the key's limit in Dashboard → API Keys | | [`INVALID_REQUEST`](#invalid-request) | 400 | No | Missing required params, failed format validation, or no `oneOf` member satisfied | Check the endpoint's required parameters | | [`COHORT_MEMBER_LIMIT_EXCEEDED`](#cohort-member-limit-exceeded) | 400 | No | A cohort member upload would exceed the cohort limit | Reduce the upload or remove members before retrying | | [`COHORT_IDENTITY_PLATFORM_UNSUPPORTED`](#cohort-identity-platform-unsupported) | 400 | No | The identity platform is not supported for cohort queries | Use a supported cohort identity platform | | [`COHORT_IDENTITY_CONFLICT`](#cohort-identity-conflict) | 409 | No | A normalized identity is already assigned to another external ID | Use the existing external ID or a different identity | | [`COHORT_QUERY_NOT_CANCELLABLE`](#cohort-query-not-cancellable) | 409 | No | The cohort query is terminal and cannot be cancelled | Read its status or results instead | | [`COHORT_QUERY_NOT_READY`](#cohort-query-not-ready) | 409 | No | The cohort query has not completed | Wait for a terminal status before reading results | | [`COHORT_RESULT_TOO_LARGE`](#cohort-result-too-large) | 413 | No | One cohort result exceeds the response-page byte ceiling | Reduce the stored result payload before retrying | | [`METHOD_NOT_ALLOWED`](#method-not-allowed) | 405 | No | The method is not accepted on this path. Most `/v1/*` endpoints are `GET`-only; response includes an `Allow: GET` header | Use `GET`, or the documented method for that path | | [`ENDPOINT_NOT_FOUND`](#endpoint-not-found) | 404 | No | Platform or resource is not supported | Check the [API Reference](/docs/api-reference.md) | | [`RESOURCE_NOT_FOUND`](#resource-not-found) | 404 | No | Upstream returned 404, or the resource exists but contains no usable data. Credits refunded when triggered by empty upstream | Verify the handle/URL exists | | [`IDEMPOTENCY_KEY_CONFLICT`](#idempotency-key-conflict) | 409 | No | The `Idempotency-Key` you sent is already in use by another account | Generate a fresh key (UUIDv4 recommended) | | [`IDEMPOTENCY_KEY_PAYLOAD_MISMATCH`](#idempotency-key-payload-mismatch) | 422 | No | You reused an `Idempotency-Key` with different query parameters or request body | Use a new key for the new payload | | [`PAYLOAD_TOO_LARGE`](#payload-too-large) | 413 | No | The JSON request body exceeds the 1 MB size limit, rejected before parsing | Reduce the batch size (split a large `ids`/`urls` array) | | [`RATE_LIMITED`](#rate-limited) | 429 | Yes, after backoff | More than 600 requests in a 1-minute window on the same API key | Honor `Retry-After`, then back off | | [`CONCURRENCY_LIMIT`](#concurrency-limit) | 429 | Yes, after backoff | More than 50 concurrent requests on the same API key | Honor `Retry-After`, then back off | | [`UPSTREAM_ERROR`](#upstream-error) | 502 | Yes, with backoff | Upstream platform returned an error. Credits refunded | Retry after a short backoff | | [`SERVICE_UNAVAILABLE`](#service-unavailable) | 503 | Yes, with backoff | Circuit breaker is open for this platform, or the upstream provider is rate-limiting us. Credits refunded. Response includes `Retry-After: 30`. A **withdrawn** endpoint also returns 503 — permanently, and with no `Retry-After` | Retry after `Retry-After`; if it is absent, do not retry | | [`INTERNAL_ERROR`](#internal-error) | 500 | Yes, with backoff | Unexpected error. Credits refunded | Retry; contact support with the `request_id` if it persists | Only the five transient failures (`RATE_LIMITED`, `CONCURRENCY_LIMIT`, `UPSTREAM_ERROR`, `SERVICE_UNAVAILABLE`, `INTERNAL_ERROR`) are safe to retry. Every client error (`4xx` other than `429`) is deterministic: the same request will fail the same way, so retrying wastes time and credits are never at stake because none were deducted. See [Handling retries](#handling-retries) for copy-paste backoff loops. Error Code Reference [#error-code-reference] Each code below carries a stable anchor — the exact target of the `doc_url` in the error envelope. MISSING_API_KEY [#missing-api-key] **401** — No `x-api-key` header was present on the request. Add the header with one of your keys from Dashboard → API Keys. No credits are deducted. INVALID_API_KEY [#invalid-api-key] **401** — The key is malformed, not found, revoked, or expired. Check the key in Dashboard → API Keys. No credits are deducted. INSUFFICIENT_CREDITS [#insufficient-credits] **402** — Your balance is lower than the endpoint's cost. Top up in Dashboard → Billing. No credits are deducted. `credits_remaining` reports your current balance. KEY_BUDGET_EXCEEDED [#key-budget-exceeded] **402** — The API key you used has its own **credit limit**, and this request would take it past that limit. Your account balance is untouched and nothing was deducted. This is not the same problem as [`INSUFFICIENT_CREDITS`](#insufficient-credits), and topping up will not fix it. A per-key limit is a cap you set yourself so that one key — typically a test, CI, or staging key — can never spend more than a fixed amount of the shared account balance. Hitting it means that key did its job. To resolve it, open Dashboard → API Keys and either raise the key's limit, reset its usage counter back to zero, or remove the limit entirely. You can also switch to a key with no limit set (production keys usually have none). `credits_remaining` still reports your **account** balance, so a large number there alongside this error is expected and is the clearest signal that the cap — not the balance — is what stopped the request: ```json { "success": false, "error": { "type": "KEY_BUDGET_EXCEEDED", "message": "This API key has a limit of 500 credits and has used 498. This endpoint requires 5 credits. Your account balance (12480 credits) was not charged — raise or reset this key's limit in Dashboard → API Keys, or use a key with no limit.", "status": 402, "doc_url": "https://www.socialcrawl.dev/docs/errors#key-budget-exceeded" }, "credits_used": 0, "credits_remaining": 12480, "request_id": "req-abc123" } ``` Do not retry this one on a timer. The counter is cumulative rather than windowed, so it never clears on its own — only a limit change or a usage reset will let the key spend again. See [Authentication](/docs/authentication.md#per-key-credit-limits). INVALID_REQUEST [#invalid-request] **400** — A required parameter is missing, a value failed format validation, or no `oneOf` member was satisfied. Check the endpoint's required parameters in the [API Reference](/docs/api-reference.md). No credits are deducted. Some rejections carry an `error.details.reason` you can branch on: | `error.details.reason` | What it means | How to fix it | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `invalid_utf8` | The value contains Unicode replacement characters (`U+FFFD`), which is what invalid UTF-8 decodes to. Your client almost certainly sent the value in a non-UTF-8 charset such as windows-1251 or latin-1. | Send the value as UTF-8. With cURL, pass the query via `--data-urlencode` rather than interpolating it into the URL. If your shell hands cURL non-UTF-8 bytes in the first place, set your locale (`LANG=en_US.UTF-8`, or `chcp 65001` on Windows). | | `double_encoded` | The value was percent-encoded before your HTTP client encoded it again, so it arrived as the literal text `%D1%80%D0%B0…`. Searching for that matches nothing. | Pass the raw UTF-8 value and let your HTTP client encode it. Do not call `urlencode` / `encodeURIComponent` yourself first. | Both checks run before billing, so a mangled query costs you nothing. This most often bites on non-Latin search queries. A query that is silently corrupted in transit cannot match anything, and it used to come back as an empty `200`, which looked like "this platform has no content in my language". It is now a `400` that names the fault: ```json { "success": false, "error": { "type": "INVALID_REQUEST", "message": "Parameter 'query' is not valid UTF-8. It contains Unicode replacement characters (U+FFFD), which almost always means the client sent the value in a non-UTF-8 charset (e.g. windows-1251 or latin-1). Send the value as UTF-8.", "details": { "reason": "invalid_utf8" } } } ``` A correctly-encoded non-Latin query needs no special handling. Cyrillic, Greek, Arabic, Korean, and Japanese all work as-is: ```bash curl -G https://www.socialcrawl.dev/v1/threads/search \ -H "x-api-key: YOUR_KEY" \ --data-urlencode "query=работа" ``` Getting `200` with an empty `items` array on a **correctly-encoded** query is a different thing entirely, and is not an error. Most platform search endpoints are keyword indexes rather than semantic ones, so a full sentence tends to return nothing in any language (`find a job` returns results; `i cannot find a job` returns none). Search with keywords, not sentences. `error.details.did_you_mean` — the self-correcting rejections [#errordetailsdid_you_mean--the-self-correcting-rejections] A second family of `INVALID_REQUEST` carries `error.details.did_you_mean` instead of a `reason`: the exact parameter, or `param=value` pair, to send instead. You get it when you send a foreign pagination param (`did_you_mean: "cursor"` — see [Pagination](/docs/pagination.md#sending-the-wrong-param-is-a-400-not-a-silent-bug)), when you combine two options the upstream cannot honor together, or when an option you sent needs a companion parameter you left out. A few of these also carry `remove_params`, a list of parameters to drop; apply both fields and you land on a valid request. So `error.details` has two shapes and they never appear together — branch on which key is present. These rejections are all pre-billing, so they cost 0 credits, and the hint is stable enough for an SDK to auto-correct against. METHOD_NOT_ALLOWED [#method-not-allowed] **405** — The method is not accepted on this path. Most of the `/v1/*` surface is `GET`-only, and those paths answer any other method with this error plus an `Allow: GET` header. The exceptions are the six batch endpoints listed under [Idempotency](#idempotency), which take `POST`, and the `/v1/web/*` job surface, which takes `POST` on job, monitor, and session creation and `PATCH`/`DELETE` on monitors, sessions, and jobs. No credits are deducted. ENDPOINT_NOT_FOUND [#endpoint-not-found] **404** — The platform or resource is not supported. Check the [API Reference](/docs/api-reference.md) for the correct path. No credits are deducted. RESOURCE_NOT_FOUND [#resource-not-found] **404** — The upstream returned 404, or the resource exists but contains no usable data (e.g. a nonexistent handle). Verify the handle or URL. Credits are refunded when this is triggered by an empty upstream body. Transcript endpoints (e.g. `/v1/youtube/video/transcript`) additionally set `error.details.reason` so you can tell exactly why no transcript came back: | `details.reason` | Meaning | | ------------------- | ----------------------------------------------------------------------------------- | | `captions_disabled` | The video exists, but its owner has disabled captions | | `no_captions` | The video exists, but it has no caption track (or none in the requested `language`) | | `login_required` | The video requires login to view, so its transcript cannot be retrieved | | `video_gone` | The video is deleted, private, or never existed | All four are deterministic for a given video — retrying the same URL returns the same result and never costs credits. Around 5–10% of public YouTube videos have no retrievable captions, so batch pipelines should expect and skip these rather than retry them. The batch `POST /v1/youtube/transcripts` endpoint reports these per row as `status: "not_found"` with `ext.reason`, and refunds each affected row. Its vocabulary is coarser: only `no_captions` (which also covers captions-disabled and login-gated videos) and `video_gone`. If you need to tell a captions-disabled video from a genuinely caption-less one, re-fetch that id through the singular endpoint. COHORT_MEMBER_LIMIT_EXCEEDED [#cohort-member-limit-exceeded] **400** — The member upload would exceed the cohort limit. Reduce the upload or remove members before retrying. COHORT_IDENTITY_PLATFORM_UNSUPPORTED [#cohort-identity-platform-unsupported] **400** — The supplied identity platform is not supported for cohort queries. Use a supported platform. COHORT_IDENTITY_CONFLICT [#cohort-identity-conflict] **409** — The normalized platform identity is already assigned to another external ID in this cohort. Use the existing external ID or a different identity. COHORT_QUERY_NOT_CANCELLABLE [#cohort-query-not-cancellable] **409** — The query is already terminal and cannot be cancelled. Read its status or results instead. COHORT_QUERY_NOT_READY [#cohort-query-not-ready] **409** — The query has not completed, so results are not available yet. Wait for a terminal status before reading results. COHORT_RESULT_TOO_LARGE [#cohort-result-too-large] **413** — One decrypted result exceeds the 1,000,000-byte response-page ceiling. No partial result is returned. IDEMPOTENCY_KEY_CONFLICT [#idempotency-key-conflict] **409** — The `Idempotency-Key` you sent is already in use by another account. Generate a fresh key (a UUIDv4 is recommended). No credits are deducted. IDEMPOTENCY_KEY_PAYLOAD_MISMATCH [#idempotency-key-payload-mismatch] **422** — You reused an `Idempotency-Key` with a different payload — different query parameters on a `GET`, or a different JSON body on a `POST`. Use a new key for the new payload. No credits are deducted. PAYLOAD_TOO_LARGE [#payload-too-large] **413**: The request's JSON body exceeded the 1 MB size limit and was rejected before parsing, so no credits were deducted. Reduce the batch size, for example by splitting a large `ids` or `urls` array across several requests. RATE_LIMITED [#rate-limited] **429** — More than 600 requests in a 1-minute sliding window on the same API key. Slow down and retry after `Retry-After`. Response carries `X-RateLimit-Limit`/`Remaining`/`Reset`. No credits are deducted. See [Rate Limits](/docs/rate-limits.md). CONCURRENCY_LIMIT [#concurrency-limit] **429** — More than 50 concurrent requests were in flight on the same API key. Reduce concurrency and retry. No credits are deducted. Response carries `X-Concurrency-Limit`/`Remaining` (not `X-RateLimit-*`). See [Rate Limits](/docs/rate-limits.md). UPSTREAM_ERROR [#upstream-error] **502** — The upstream platform returned an error (after our automatic retries). Retry after a short backoff. Credits are refunded. To tell an upstream outage from a transient blip, check the live [status page](/status) or `GET /v1/status` for the platform's current circuit state. SERVICE_UNAVAILABLE [#service-unavailable] **503** — Several causes. The two common ones are transient and both carry `Retry-After: 30`: the circuit breaker is open after repeated upstream failures, or the upstream data provider is momentarily rate-limiting us. Wait 30 seconds and retry. Credits are refunded. A 503 **without** a `Retry-After` header is not transient. It means the endpoint has been withdrawn because its upstream no longer serves it, and the `message` names the reason. Retrying will never succeed, so branch on the presence of `Retry-After` rather than on the status alone. Nothing is deducted in the first place on a withdrawn endpoint — the router rejects it before billing — so there is no refund to wait for. Check the [changelog](/changelog) for the replacement. INTERNAL_ERROR [#internal-error] **500** — An unexpected error occurred on our side. Retry; if it persists, contact support with the `request_id`. Credits are refunded. Idempotency [#idempotency] Send an `Idempotency-Key` header to make a charging request safe to retry. The key is honored on **every charging endpoint** — the `GET /v1/*` catch-all and all six `POST` endpoints (`/v1/prism/post-stats`, `/v1/prism/comment-lookup`, `/v1/prism/profiles`, `/v1/youtube/transcripts`, `/v1/youtube/videos`, `/v1/youtube/channels`). * A retry with the same key and the same payload **replays** the stored response verbatim, charges nothing (`X-Credits-Used: 0`, `X-Idempotent-Replay: true`), and does not re-run the upstream call — so a network retry or a cron double-fire never double-charges or double-creates. * The key's identity covers the request payload: query parameters for `GET`, and the canonical JSON body for `POST`. Reusing a key with a different payload returns [`IDEMPOTENCY_KEY_PAYLOAD_MISMATCH`](#idempotency-key-payload-mismatch) (422) rather than a false replay. * A key already claimed by another account returns [`IDEMPOTENCY_KEY_CONFLICT`](#idempotency-key-conflict) (409). * Keys are retained for 24 hours. Use a fresh UUIDv4 per logical operation. Refund Rules [#refund-rules] Credits are automatically refunded when: * Upstream returns a 5xx error (`UPSTREAM_ERROR`, 502) * The circuit breaker rejects the request, or the upstream provider rate-limits us (`SERVICE_UNAVAILABLE`, 503) * An unexpected server error occurs (`INTERNAL_ERROR`, 500) * Upstream returns 200 with an empty body — e.g. a nonexistent handle — triggering `RESOURCE_NOT_FOUND` (404) Credits are **never deducted** for these outcomes, so no refund is needed: * Cache hits (`X-Cache: HIT`, `X-Credits-Used: 0`) * `METHOD_NOT_ALLOWED` (405), `IDEMPOTENCY_KEY_CONFLICT` (409), and `IDEMPOTENCY_KEY_PAYLOAD_MISMATCH` (422) * `KEY_BUDGET_EXCEEDED` (402) — the request is refused before the charge lands, and the key's own usage counter is left where it was * Idempotent replays — the original charge already appears on your earlier request Client errors (`MISSING_API_KEY`, `INVALID_API_KEY`, `INVALID_REQUEST`, `ENDPOINT_NOT_FOUND`) are rejected before any deduction. Handling retries [#handling-retries] Retry only the statuses marked retryable above: `429` (`RATE_LIMITED` or `CONCURRENCY_LIMIT`), `500`, `502`, and `503`. When a response carries a `Retry-After` header (a `503` always sends `Retry-After: 30`; a rate-limit `429` sends seconds until the window resets; a concurrency `429` sends a short hint), wait that long before the first retry. Otherwise back off exponentially with full jitter so a pool of your own workers does not resynchronize and stampede the cap together. Cap the attempts and surface the error rather than retrying forever. Never retry a `4xx` client error other than `429`: the request is deterministic and will fail the same way. Because credits are refunded on every retryable failure, a retry loop never double-charges. Add an `Idempotency-Key` header (see [Idempotency](#idempotency)) if you also want a network-level retry of a *successful* call to replay rather than re-run. ```bash #!/usr/bin/env bash # Retry only transient failures (429, 500, 502, 503, 504). Honor Retry-After # when present, else exponential backoff with jitter. Give up after 5 tries. url="https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" max_retries=5 attempt=0 while :; do body=$(mktemp); headers=$(mktemp) status=$(curl -s -o "$body" -D "$headers" -w '%{http_code}' \ -H "x-api-key: $SOCIALCRAWL_API_KEY" "$url") if [ "$status" -lt 400 ]; then cat "$body"; rm -f "$body" "$headers"; break fi case "$status" in 429|500|502|503|504) ;; # retryable, fall through *) echo "Non-retryable $status"; cat "$body"; rm -f "$body" "$headers"; exit 1 ;; esac attempt=$((attempt + 1)) if [ "$attempt" -gt "$max_retries" ]; then echo "Gave up after $max_retries retries (last status $status)" rm -f "$body" "$headers"; exit 1 fi # Retry-After (seconds) wins; else exponential backoff 2^attempt with jitter. retry_after=$(grep -i '^retry-after:' "$headers" | tr -d '\r' | awk '{print $2}') if [ -n "$retry_after" ]; then delay="$retry_after" else delay=$(awk "BEGIN{srand(); print (2 ^ $attempt) * (0.5 + rand() / 2)}") fi rm -f "$body" "$headers" sleep "$delay" done ``` ```python import os import random import time import requests # pip install requests BASE_URL = "https://www.socialcrawl.dev/v1" RETRYABLE = {429, 500, 502, 503, 504} def get_with_retry(path, params=None, max_retries=5): headers = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]} for attempt in range(max_retries + 1): resp = requests.get(f"{BASE_URL}{path}", params=params, headers=headers) # Success, or a client error we must not retry: hand it back / raise. if resp.ok or resp.status_code not in RETRYABLE or attempt == max_retries: resp.raise_for_status() return resp.json() # Retry-After (seconds) wins; else exponential backoff with full jitter. retry_after = resp.headers.get("Retry-After") delay = float(retry_after) if retry_after else (2 ** attempt) * (0.5 + random.random() / 2) time.sleep(delay) data = get_with_retry("/tiktok/profile", {"handle": "charlidamelio"}) print(data) ``` ```js // Node 18+ (built-in fetch). No dependencies. const BASE_URL = "https://www.socialcrawl.dev/v1"; const RETRYABLE = new Set([429, 500, 502, 503, 504]); const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); async function getWithRetry(path, { params = {}, maxRetries = 5 } = {}) { const url = new URL(BASE_URL + path); for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v); const headers = { "x-api-key": process.env.SOCIALCRAWL_API_KEY }; for (let attempt = 0; ; attempt++) { const res = await fetch(url, { headers }); if (res.ok || !RETRYABLE.has(res.status)) { if (!res.ok) throw new Error(`SocialCrawl ${res.status}: ${await res.text()}`); return res.json(); } if (attempt >= maxRetries) { throw new Error( `Gave up after ${maxRetries} retries (last status ${res.status})`, ); } // Retry-After (seconds) wins; else exponential backoff with full jitter. const retryAfter = Number(res.headers.get("Retry-After")); const delayMs = Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter * 1000 : 2 ** attempt * 1000 * (0.5 + Math.random() / 2); await sleep(delayMs); } } const data = await getWithRetry("/tiktok/profile", { params: { handle: "charlidamelio" }, }); console.log(data); ``` Debugging Tips [#debugging-tips] * Every error envelope carries a `doc_url` pointing at the matching section on this page, e.g. `https://www.socialcrawl.dev/docs/errors#insufficient-credits` * `request_id` matches the `X-Request-Id` header and the `request_id` column on Dashboard → Activity Logs * Persistent 502s on a known-good input usually indicate an upstream outage — check the [status page](/status) (or `GET /v1/status`) for the platform's circuit state * A high retry count on a slow-but-successful call is surfaced in the `X-Upstream-Retries` response header