# Error Handling (/docs/errors) Every SocialCrawl error code with its status, retry verdict, and refund behavior, plus copy-paste backoff loops Every failure returns the same JSON envelope: a stable `error.type` to branch on, an HTTP `status`, an `error.retryable` flag, and a `doc_url` pointing at that code's entry on this page. Six failures are worth retrying. `credits_used` always reports the charge committed at response time; automatic compensation covers the rare case where an immediate refund cannot commit. ```json title="Response" { "success": false, "error": { "type": "INSUFFICIENT_CREDITS", "message": "Your account has 0 credits remaining. This endpoint requires 1 credits. Retrying this request will not succeed until your account balance changes.", "status": 402, "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits", "retryable": false, "details": { "reason": "balance_too_low", "retry_will_succeed": false, "next_step": "Top up your credits on the billing page (or turn on auto-recharge there), then send the request again.", "top_up_url": "https://www.socialcrawl.dev/dashboard/billing/payments", "credits_required": 1 } }, "credits_used": 0, "credits_remaining": 0, "request_id": "req-abc123" } ``` Four fields carry the debugging weight: - **`error.type`** is what you branch on. It is stable for a given failure and never changes meaning under `/v1`. - **`error.retryable`** is `true` when sending the identical request again can succeed, and `false` when it will fail the same way until you change something. It matches the Retryable column in the table below, so a client can decide without keeping its own list of codes. - **`credits_used`** is the **net committed** charge for the failed request. It is 0 when no deduction occurred or the refund committed. A non-zero value on a retryable failure means the immediate refund did not commit and automatic compensation is pending; retain `request_id`. - **`request_id`** matches the `X-Request-Id` header, which is present on every response, success or error. Include it when you contact support so we can trace the exact request in our logs. The `doc_url` is an anchor on this page, so you can jump straight to a code with `#insufficient-credits` or `#upstream-error`. ## Error codes | Code | Status | Retryable | Refund | What it means | | ------------------------------------------------------------------------------- | ------ | ------------------ | ---------------------------------- | --------------------------------------------------------------------------------- | | [`MISSING_API_KEY`](#missing-api-key) | 401 | No | Not charged | No `x-api-key` header on the request | | [`INVALID_API_KEY`](#invalid-api-key) | 401 | No | Not charged | Key is malformed, not found, revoked, or expired | | [`INSUFFICIENT_CREDITS`](#insufficient-credits) | 402 | No | Not charged | Balance is lower than the endpoint cost | | [`KEY_BUDGET_EXCEEDED`](#key-budget-exceeded) | 402 | No | Not charged | This key has spent its own per-key credit limit. The account balance is untouched | | [`INVALID_REQUEST`](#invalid-request) | 400 | No | Not charged | Missing, malformed, conflicting, or non-empty undeclared parameters | | [`METHOD_NOT_ALLOWED`](#method-not-allowed) | 405 | No | Not charged | The method is not accepted on this path | | [`ENDPOINT_NOT_FOUND`](#endpoint-not-found) | 404 | No | Not charged | Platform or resource is not supported | | [`RESOURCE_NOT_FOUND`](#resource-not-found) | 404 | No | Refunded on an empty upstream body | Upstream returned 404, or the resource exists but contains no usable data | | [`PAYLOAD_TOO_LARGE`](#payload-too-large) | 413 | No | Not charged | The JSON request body exceeds the 1 MB size limit, rejected before parsing | | [`IDEMPOTENCY_KEY_CONFLICT`](#idempotency-key-conflict) | 409 | No | Not charged | The key conflicts with an incompatible active reservation | | [`IDEMPOTENCY_IN_PROGRESS`](#idempotency-in-progress) | 409 | Yes, after 1s | Not charged | Another request with the same key and payload is still running | | [`IDEMPOTENCY_REPLAY_UNAVAILABLE`](#idempotency-replay-unavailable) | 409 | No | Not charged | The completed response exceeded the 64 KB idempotent replay limit | | [`IDEMPOTENCY_KEY_PAYLOAD_MISMATCH`](#idempotency-key-payload-mismatch) | 422 | No | Not charged | You reused an `Idempotency-Key` with different query parameters or request body | | [`COHORT_LIMIT_EXCEEDED`](#cohort-limit-exceeded) | 400 | No | Not charged | The account already holds the maximum of 100 cohorts | | [`COHORT_MEMBER_LIMIT_EXCEEDED`](#cohort-member-limit-exceeded) | 400 | No | Not charged | A cohort member upload would exceed the cohort limit | | [`COHORT_IDENTITY_PLATFORM_UNSUPPORTED`](#cohort-identity-platform-unsupported) | 400 | No | Not charged | The identity platform is not supported for cohort queries | | [`COHORT_IDENTITY_CONFLICT`](#cohort-identity-conflict) | 409 | No | Not charged | A normalized identity is already assigned to another external ID | | [`COHORT_QUERY_NOT_CANCELLABLE`](#cohort-query-not-cancellable) | 409 | No | Not charged | The cohort query is terminal and cannot be cancelled | | [`COHORT_QUERY_NOT_READY`](#cohort-query-not-ready) | 409 | No | Not charged | The cohort query has not completed | | [`COHORT_RESULT_TOO_LARGE`](#cohort-result-too-large) | 413 | No | Not charged | One cohort result exceeds the response-page byte ceiling | | [`RATE_LIMITED`](#rate-limited) | 429 | Yes, after backoff | Not charged | More than 600 requests in a 1-minute window on the same API key | | [`CONCURRENCY_LIMIT`](#concurrency-limit) | 429 | Yes, after backoff | Not charged | More than 50 concurrent requests on the same API key | | [`UPSTREAM_ERROR`](#upstream-error) | 502 | Yes, with backoff | Refunded | Upstream platform returned an error | | [`SERVICE_UNAVAILABLE`](#service-unavailable) | 503 | Yes, with backoff | Refunded | Circuit breaker open, upstream provider throttling us, or a withdrawn endpoint | | [`INTERNAL_ERROR`](#internal-error) | 500 | Yes, with backoff | Refunded | Unexpected error on our side | ## Which errors are safe to retry The retryable failures are `IDEMPOTENCY_IN_PROGRESS`, `RATE_LIMITED`, `CONCURRENCY_LIMIT`, `UPSTREAM_ERROR`, `SERVICE_UNAVAILABLE`, and `INTERNAL_ERROR`. Every error body says the same thing in `error.retryable`, so you can branch on that field instead of copying this list. The flag follows the code, so a withdrawn endpoint's `SERVICE_UNAVAILABLE` still reads `true`; tell that one apart by its missing `Retry-After` header, as described under [`SERVICE_UNAVAILABLE`](#service-unavailable). Every other client error is deterministic. `IDEMPOTENCY_IN_PROGRESS` is the one retryable 409: wait for `Retry-After`, then send the same key and payload again. See [Handling retries](#handling-retries) for copy-paste backoff loops. ## Error reference Each code below carries a stable anchor, the exact target of the `doc_url` in the error envelope. ### MISSING_API_KEY [#missing-api-key] **401.** No `x-api-key` header was present on the request. Add the header with one of your keys from Dashboard → API Keys. No credits are deducted. ### INVALID_API_KEY [#invalid-api-key] **401.** The key is malformed, not found, revoked, or expired. Check the key in Dashboard → API Keys. No credits are deducted. ### INSUFFICIENT_CREDITS [#insufficient-credits] **402.** Your balance is lower than the endpoint's cost. Top up in Dashboard → Billing. No credits are deducted. `credits_remaining` reports your current balance. Sending the same request again will not succeed until your balance changes, so keep this code out of your retry loop. The message ends with a sentence saying so, and `error.details` gives the same answer as fields: | `error.details` key | What it holds | | -------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | | `reason` | `balance_too_low` | | `retry_will_succeed` | Always `false`. The identical request fails the same way until credits are added, by a top-up or by auto-recharge. | | `next_step` | One sentence saying what to do next. | | `top_up_url` | The billing page in the dashboard, where you buy credits or turn on auto-recharge. | | `credits_required` | The credits this request needed. Present on `GET` endpoints. | | `pricing` | `"ceiling"` when the endpoint holds its worst-case cost up front and refunds whatever the call does not use. Absent on fixed-price endpoints. | | `lower_cost_with` | Sent with `pricing: "ceiling"`: the parameters that shrink the hold. | A repeat refusal within a minute of the first can carry `Retry-After: 60`. That is how long the refusal is remembered, not a sign that waiting will help: a top-up clears it straight away. **When the number is a ceiling.** `/v1/threads/search` holds 1 credit for every search window its `limit` allows, plus the query relaxation searches a multi-word query can fan out to, and refunds whatever the call does not use. A balance below that hold gets a 402 even though the finished call might have cost less. When a smaller request would hold less, the 402 says so: `details.pricing` is `"ceiling"`, `details.lower_cost_with` is `["limit", "expand"]`, and the message explains that a smaller `limit`, or `expand=false`, holds fewer credits. ```json title="Response" { "success": false, "error": { "type": "INSUFFICIENT_CREDITS", "message": "Your account has 3 credits remaining. This endpoint requires 11 credits. The 11 credits required are the most this call can cost: the credits it holds for search windows and query relaxation that it does not use are refunded afterwards. A smaller `limit`, or `expand=false`, holds fewer credits. Retrying this request will not succeed until your account balance changes.", "status": 402, "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits", "retryable": false, "details": { "reason": "balance_too_low", "retry_will_succeed": false, "next_step": "Top up your credits on the billing page (or turn on auto-recharge there), then send the request again.", "top_up_url": "https://www.socialcrawl.dev/dashboard/billing/payments", "credits_required": 11, "pricing": "ceiling", "lower_cost_with": ["limit", "expand"] } }, "credits_used": 0, "credits_remaining": 3, "request_id": "req-abc123" } ``` **Stopping a runaway client.** If one script or agent spends the balance that your other keys depend on, give its key a [per-key credit limit](/docs/authentication.md#per-key-credit-limits). That key then stops at its own cap with [`KEY_BUDGET_EXCEEDED`](#key-budget-exceeded), and the rest of the account keeps working. ### KEY_BUDGET_EXCEEDED [#key-budget-exceeded] **402.** The API key you used has its own **credit limit**, and this request would take it past that limit. Your account balance is untouched and nothing was deducted. This is not the same problem as [`INSUFFICIENT_CREDITS`](#insufficient-credits), and topping up will not fix it. A per-key limit is a cap you set yourself so that one key (typically a test, CI, or staging key) can never spend more than a fixed amount of the shared account balance. Hitting it means that key did its job. To resolve it, open Dashboard → API Keys and either raise the key's limit, reset its usage counter back to zero, or remove the limit entirely. You can also switch to a key with no limit set, which is how production keys are usually configured. `credits_remaining` still reports your **account** balance, so a large number there alongside this error is expected. It is the clearest signal that the cap, not the balance, is what stopped the request. ```json title="Response" { "success": false, "error": { "type": "KEY_BUDGET_EXCEEDED", "message": "This API key has a limit of 500 credits and has used 498. This endpoint requires 5 credits. Your account balance (12480 credits) was not charged. Raise or reset this key's limit in Dashboard → API Keys, or use a key with no limit. Retrying this request will not succeed until this key's credit limit is raised, removed or reset.", "status": 402, "doc_url": "https://www.socialcrawl.dev/docs/errors#key-budget-exceeded", "retryable": false, "details": { "reason": "key_budget_reached", "retry_will_succeed": false, "next_step": "Raise or remove this key's credit limit, or reset its usage, on the API keys page, or send the request with a key that has no limit.", "key_settings_url": "https://www.socialcrawl.dev/dashboard/api/keys", "credits_required": 5 } }, "credits_used": 0, "credits_remaining": 12480, "request_id": "req-abc123" } ``` `error.details` has the same keys as [`INSUFFICIENT_CREDITS`](#insufficient-credits), with two differences: `reason` is `key_budget_reached`, and `key_settings_url` (the API keys page) replaces `top_up_url`, because buying credits does not lift a key's own limit. Do not retry this one on a timer. The counter is cumulative rather than windowed, so it never clears on its own. Only a limit change or a usage reset will let the key spend again. See [Authentication](/docs/authentication.md#per-key-credit-limits). ### INVALID_REQUEST [#invalid-request] **400.** A required parameter is missing, a value failed format validation, or no `oneOf` member was satisfied. Check the endpoint's required parameters in the [API Reference](/docs/api-reference.md). A rejected request costs 0 credits. Most rejections run before billing, so nothing is deducted. A few can only be detected by the data source after the call has been charged, such as asking for a page past the end of a search. Those are refunded in full before the response is sent, so `credits_used` is still 0. Some rejections carry `error.details` you can branch on. It has two shapes and they never appear together, so branch on which key is present. | `error.details.reason` | What it means | How to fix it | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `invalid_utf8` | The value contains Unicode replacement characters (`U+FFFD`), which is what invalid UTF-8 decodes to. Your client almost certainly sent the value in a non-UTF-8 charset such as windows-1251 or latin-1. | Send the value as UTF-8. With cURL, pass the query via `--data-urlencode` rather than interpolating it into the URL. If your shell hands cURL non-UTF-8 bytes in the first place, set your locale (`LANG=en_US.UTF-8`, or `chcp 65001` on Windows). | | `double_encoded` | The value was percent-encoded before your HTTP client encoded it again, so it arrived as the literal text `%D1%80%D0%B0…`. Searching for that matches nothing. | Pass the raw UTF-8 value and let your HTTP client encode it. Do not call `urlencode` / `encodeURIComponent` yourself first. | This most often bites on non-Latin search queries. A corrupted query cannot match anything, and it used to come back as an empty `200`, which looked like "this platform has no content in my language". It is now a `400` that names the fault: ```json title="Response" { "success": false, "error": { "type": "INVALID_REQUEST", "message": "Parameter 'query' is not valid UTF-8. It contains Unicode replacement characters (U+FFFD), which almost always means the client sent the value in a non-UTF-8 charset (e.g. windows-1251 or latin-1). Send the value as UTF-8.", "details": { "reason": "invalid_utf8" } } } ``` A correctly encoded non-Latin query needs no special handling. Cyrillic, Greek, Arabic, Korean, and Japanese all work as-is: ```bash title="cURL" curl -G https://www.socialcrawl.dev/v1/threads/search \ -H "x-api-key: YOUR_KEY" \ --data-urlencode "query=работа" ``` A second family of `INVALID_REQUEST` carries `error.details.did_you_mean` instead of a `reason`: the exact parameter, or `param=value` pair, to send instead. You get it when you send a foreign pagination param (`did_you_mean: "cursor"`, see [Pagination](/docs/pagination.md#sending-the-wrong-param-is-a-400-not-a-silent-bug)), when you combine two options the upstream cannot honor together, or when an option you sent needs a companion parameter you left out. A few of these also carry `remove_params`, a list of parameters to drop. Apply both fields and you land on a valid request. The hint is stable enough for an SDK to auto-correct against. A value outside a parameter's accepted set is rejected the same way, before any credit is spent, with `details.invalid_parameters`, `details.allowed_values` and `details.example` (one valid `param=value`). A one-letter slip (`label=spma`) adds `did_you_mean: "spam"`. A search term sent to `relevance` or `label` (`relevance=traumeel`, `label=alberto`) adds `did_you_mean: "relevant_to"` on a search that judges relevance, or `"query"` elsewhere, and the message shows the request to send instead. From the 100th identical rejection on one API key in a UTC day, `details.repeated_today` carries the count and the message says so: the same request will keep failing the same way, so change it rather than retrying it. Only a rejection with a specific reason is counted (a named limit, parameter or value), so two different mistakes on one endpoint are never added together. Some searches stop at a fixed page number. `/v1/instagram/search/reels` stops at page 11 when `date_posted` is set. Asking for a later page returns `INVALID_REQUEST` with `details.reason` set to `page_limit`, refunded in full: | Key | What it holds | | ---------------- | ------------------------------------------------------------------ | | `max_page` | The last page this search serves | | `requested_page` | The page you asked for | | `narrow_with` | Parameters that narrow the search, for example `["query", "date_posted"]` | | `example` | A valid request: your own narrowing parameters with the last page | ```json title="Response" { "success": false, "error": { "type": "INVALID_REQUEST", "message": "This search stops at page 11 when `date_posted` is set, so page 12 is past its last page and there are no more pages for this query and date window. To reach different reels, use a more specific `query` or a narrower `date_posted` window (`last-week`, `last-month` or `last-year`), or leave `date_posted` out to page further without a date filter. Your credits have been refunded.", "status": 400, "doc_url": "https://www.socialcrawl.dev/docs/errors#invalid-request", "retryable": false, "details": { "reason": "page_limit", "max_page": 11, "requested_page": 12, "narrow_with": ["query", "date_posted"], "example": "/v1/instagram/search/reels?query=top%20restaurants&date_posted=last-month&page=11" } }, "credits_used": 0, "credits_remaining": 12480, "request_id": "req-abc123" } ``` Treat it as the end of the walk. Paging further will not return more results for the same query and date window. A more specific query or a narrower date window returns a different set of reels, and the same query without `date_posted` pages past 11. Getting `200` with an empty `items` array on a **correctly-encoded** query is a different thing entirely, and is not an error. Most platform search endpoints are keyword indexes rather than semantic ones, so a full sentence tends to return nothing in any language (`find a job` returns results; `i cannot find a job` returns none). Search with keywords, not sentences. ### METHOD_NOT_ALLOWED [#method-not-allowed] **405.** The method is not accepted on this path. Registry read paths advertise `Allow: GET, HEAD`; POST-only routes advertise `Allow: POST`. The JSON body and `X-Request-Id` are still canonical. No credits are deducted. The exceptions are the six batch endpoints listed under [Idempotency](#idempotency), which take `POST`, and the `/v1/web/*` job surface, which takes `POST` on job, monitor, and session creation and `PATCH` or `DELETE` on monitors, sessions, and jobs. ### ENDPOINT_NOT_FOUND [#endpoint-not-found] **404.** The platform or resource is not supported. Check the [API Reference](/docs/api-reference.md) for the correct path. No credits are deducted. ### RESOURCE_NOT_FOUND [#resource-not-found] **404.** The upstream returned 404, or the resource exists but contains no usable data, for example a nonexistent handle. Verify the handle or URL. Credits are refunded when this is triggered by an empty upstream body. Three endpoint families narrow the cause with `error.details.reason`, so you can tell a permanent gap from a fixable mistake. Transcript endpoints such as `/v1/youtube/video/transcript` set `error.details.reason`: | `details.reason` | Meaning | | --------------------- | ------------------------------------------------------------------------------------------------------------------ | | `captions_disabled` | The video exists, but its owner has disabled captions | | `no_captions` | The video exists, but it has no caption track (or none in the requested `language`) | | `no_captions_or_gone` | No caption track, or the post is unavailable. Reddit answers both the same way, so we do not say which | | `no_speech` | Instagram: the video exists, but no speech was detected in it, so there is nothing to transcribe | | `no_video_or_gone` | Instagram: the URL is a photo or carousel post, or the post is unavailable. Instagram answers both the same way | | `post_not_public` | LinkedIn: LinkedIn does not show this post to logged-out viewers and it could not be resolved to an original post | | `login_required` | The video requires login to view, so its transcript cannot be retrieved | | `video_gone` | The video is deleted, private, or never existed | Every reason is deterministic for a given video. Retrying the same URL returns the same result and never costs credits. Around 5 to 10% of public YouTube videos have no retrievable captions, so batch pipelines should expect and skip these rather than retry them. `/v1/instagram/media/transcript` transcribes the audio of videos up to 2 minutes long. A longer video returns **400** `INVALID_REQUEST` with a message giving its length, also at no charge. The batch `POST /v1/youtube/transcripts` endpoint reports these per row as `status: "not_found"` with `ext.reason`, and refunds each affected row. Its vocabulary is coarser: only `no_captions` (which also covers captions-disabled and login-gated videos) and `video_gone`. If you need to tell a captions-disabled video from a genuinely caption-less one, re-fetch that id through the singular endpoint. `/v1/web/scrape` sets `error.details.reason` to `site_not_supported` when it cannot fetch the site at all. This is deterministic: `retry_will_succeed` is `false`, and nothing is charged. When the URL is on a platform SocialCrawl reads directly, `error.details.suggestion` names the endpoint that reads it, such as `/v1/instagram/post` or `/v1/linkedin/profile`. Profile endpoints such as `/v1/tiktok/profile` set `error.details.reason` for the same purpose, because a handle that stops resolving has causes that need opposite responses from you. Account-keyed list endpoints carry it too, whenever a source explained itself. That covers the post, reel and roster lanes you poll a tracked list through, such as `/v1/instagram/profile/posts`, `/v1/instagram/profile/reels` and `/v1/instagram/followers`. A list 404 with no explanation from any source carries no `details`, so the field's presence is itself the signal that a source gave a verdict. | `details.reason` | Meaning | | ------------------- | ------------------------------------------------------------------------------------------------------------------ | | `account_gone` | The platform reports no account under this identifier: removed, deactivated, renamed, or it never existed | | `account_private` | The account exists, but the platform does not serve this data publicly (audience controls, or a private or limited account) | | `handle_unresolved` | No source could resolve the account and none reported a reason | `account_private` is the one that matters most for a tracked-handle list. The account is still live and usually still publishing, so pruning it on the 404 drops a creator you wanted. On TikTok its posts remain readable through `/v1/tiktok/profile/videos` with the same handle, and the response says so in `details.suggestion`. On Instagram a private account's posts are not readable by anyone, so there is no lane that still serves them, but you can confirm the handle is worth keeping. `/v1/instagram/profile` returns `200` for a private account, with `author.private` set to `true`, while a handle Instagram no longer carries returns `404` at no cost: ```bash title="cURL" curl -H "x-api-key: $SOCIALCRAWL_API_KEY" "https://www.socialcrawl.dev/v1/instagram/profile?handle=aleecapalbo" ``` That call costs one credit when the account exists and nothing when it does not, so sweeping the handles that failed is priced by how many are still alive. `account_gone` covers a rename as well as a removal, because the platform answers both the same way: the username is simply not in use. TikTok numeric user ids are permanent, so you can separate the two yourself. Keep the `author.id` from any earlier successful call and re-run the lookup with it: ```bash title="cURL" curl -H "x-api-key: $SOCIALCRAWL_API_KEY" "https://www.socialcrawl.dev/v1/tiktok/profile?user_id=5831967" ``` A `200` means the account was renamed, and `author.username` in that response is the current handle. A second `404` means it is genuinely gone. Both the handle call and the id call cost nothing when they miss. YouTube works the same way. A channel id (the 24-character id starting with `UC`) never changes, while a handle can. When `/v1/youtube/channel` misses on a `handle` with `account_gone`, `details.suggestion` says to re-run it with `channelId` set to the `author.id` you kept from an earlier successful call: ```bash title="cURL" curl -H "x-api-key: $SOCIALCRAWL_API_KEY" "https://www.socialcrawl.dev/v1/youtube/channel?channelId=UCX6OQ3DkcsbYNE6H8uQQuVA" ``` A `200` means the channel changed its handle, and `author.username` in that response is the current one. A second `404` means the channel is gone. All three reasons are deterministic for a given identifier and never cost credits, so a pipeline should record them and move on rather than retry. ### PAYLOAD_TOO_LARGE [#payload-too-large] **413.** The request's JSON body exceeded the 1 MB size limit and was rejected before parsing, so no credits were deducted. Reduce the batch size, for example by splitting a large `ids` or `urls` array across several requests. ### IDEMPOTENCY_KEY_CONFLICT [#idempotency-key-conflict] **409.** The `Idempotency-Key` conflicts with an incompatible active reservation. Generate a fresh key, ideally a UUIDv4. Keys are scoped per account; this error does not reveal or depend on another account's keys. No credits are deducted. ### IDEMPOTENCY_IN_PROGRESS [#idempotency-in-progress] **409.** Another request with this key and the same payload is still running. This retry was not dispatched or charged. Wait for `Retry-After: 1`, then retry with the same key. Once the original finishes, the retry replays its stored result at 0 credits. ### IDEMPOTENCY_REPLAY_UNAVAILABLE [#idempotency-replay-unavailable] **409.** The original request completed and its billing outcome is known, but its response exceeded the 64 KB idempotent replay storage limit. This retry was not dispatched or charged. The same key cannot reconstruct the original body. Use a new key only if you intend to make and pay for a new request. ### IDEMPOTENCY_KEY_PAYLOAD_MISMATCH [#idempotency-key-payload-mismatch] **422.** You reused an `Idempotency-Key` with a different payload, meaning different query parameters on a `GET` or a different JSON body on a `POST`. Use a new key for the new payload. No credits are deducted. ### COHORT_LIMIT_EXCEEDED [#cohort-limit-exceeded] **400.** The account already holds the maximum of 100 cohorts under this API key. Delete an unused cohort (`DELETE /v1/cohorts/{cohortId}`) before creating another. ### COHORT_MEMBER_LIMIT_EXCEEDED [#cohort-member-limit-exceeded] **400.** The member upload would exceed the cohort limit. Reduce the upload or remove members before retrying. ### COHORT_IDENTITY_PLATFORM_UNSUPPORTED [#cohort-identity-platform-unsupported] **400.** The supplied identity platform is not supported for cohort queries. Use a supported platform. ### COHORT_IDENTITY_CONFLICT [#cohort-identity-conflict] **409.** The normalized platform identity is already assigned to another external ID in this cohort. Use the existing external ID or a different identity. ### COHORT_QUERY_NOT_CANCELLABLE [#cohort-query-not-cancellable] **409.** The query is already terminal and cannot be cancelled. Read its status or results instead. ### COHORT_QUERY_NOT_READY [#cohort-query-not-ready] **409.** The query has not completed, so results are not available yet. Wait for a terminal status before reading results. ### COHORT_RESULT_TOO_LARGE [#cohort-result-too-large] **413.** One decrypted result exceeds the 1,000,000-byte response-page ceiling. No partial result is returned. ### RATE_LIMITED [#rate-limited] **429.** More than 600 requests in a 1-minute sliding window on the same API key. Slow down and retry after `Retry-After`. The response carries `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset`. No credits are deducted. See [Rate Limits](/docs/rate-limits.md). ### CONCURRENCY_LIMIT [#concurrency-limit] **429.** More than 50 concurrent requests were in flight on the same API key. Reduce concurrency and retry. No credits are deducted. The response carries `X-Concurrency-Limit` and `X-Concurrency-Remaining`, not `X-RateLimit-*`. See [Rate Limits](/docs/rate-limits.md). ### UPSTREAM_ERROR [#upstream-error] **502.** The upstream platform returned an error, after our own automatic retries. Retry after a short backoff. Credits are refunded. To tell an upstream outage from a transient blip, check the live [status page](/status) or `GET /v1/status` for the platform's current circuit state. ### SERVICE_UNAVAILABLE [#service-unavailable] **503.** Several causes. An open circuit breaker or an upstream provider throttle carries `Retry-After: 30`. If the idempotency claim store is temporarily unavailable, a keyed request fails closed before billing with `Retry-After: 1`; retry with the same key. No balance lookup is attempted on that path, so `credits_remaining` is `null` and `X-Credits-Remaining` is omitted. A 503 **without** a `Retry-After` header is not transient. It means the endpoint has been withdrawn because its upstream no longer serves it, and the `message` names the reason. Retrying will never succeed, so branch on the presence of `Retry-After` rather than on the status alone. Nothing is deducted in the first place on a withdrawn endpoint, since the router rejects it before billing, so there is no refund to wait for. Check the [changelog](/changelog) for the replacement. ### INTERNAL_ERROR [#internal-error] **500.** An unexpected error occurred on our side. Retry, and if it persists, contact support with the `request_id`. The server attempts the refund before responding. If it cannot commit, `credits_used` remains non-zero and the exact outstanding remainder is queued for automatic compensation rather than falsely reported as refunded. ## Idempotency Send an `Idempotency-Key` header to make a charging request safe to retry. The key is honored on **every charging endpoint**: the `GET /v1/*` catch-all and all six `POST` endpoints (`/v1/prism/post-stats`, `/v1/prism/comment-lookup`, `/v1/prism/profiles`, `/v1/youtube/transcripts`, `/v1/youtube/videos`, `/v1/youtube/channels`). - A retry with the same key and the same payload replays the stored terminal status and payload, charges nothing (`X-Credits-Used: 0`, `X-Idempotent-Replay: true`), and does not re-run the upstream call. Replay billing metadata is refreshed: the current balance is returned when known. - The key's identity covers the request payload: query parameters for `GET`, and the canonical JSON body for `POST`. Reusing a key with a different payload returns [`IDEMPOTENCY_KEY_PAYLOAD_MISMATCH`](#idempotency-key-payload-mismatch) (422) rather than a false replay. - Keys are scoped per account. Two accounts can use the same opaque key independently. - A same-key request that is still running returns [`IDEMPOTENCY_IN_PROGRESS`](#idempotency-in-progress) (409) with `Retry-After: 1`. It is not dispatched or charged; retry with the same key. - An unfinished claim expires after 5 minutes so a crashed request cannot lock the key indefinitely. - A completed outcome whose body exceeded 64 KB returns [`IDEMPOTENCY_REPLAY_UNAVAILABLE`](#idempotency-replay-unavailable) (409). Use a new key only to make a new billable request. - If the claim store is unavailable, the keyed request fails closed at 503 with `Retry-After: 1` before billing. - Completed outcomes are retained for 24 hours. Use a fresh UUIDv4 per logical operation. - GET SSE cannot be combined with `Idempotency-Key`; the unsupported combination returns an unbilled `400 INVALID_REQUEST` before dispatch. POST batch streams keep sync-equivalent replay. ## Refund rules Credits are refunded automatically when: - Upstream returns a 5xx error (`UPSTREAM_ERROR`, 502). - The circuit breaker rejects the request, or the upstream provider rate-limits us (`SERVICE_UNAVAILABLE`, 503). - An unexpected server error occurs (`INTERNAL_ERROR`, 500). - Upstream returns 200 with an empty body, for example a nonexistent handle, triggering `RESOURCE_NOT_FOUND` (404). Credits are **never deducted** for these outcomes, so no refund is needed: - Cache hits (`X-Cache: HIT`, `X-Credits-Used: 0`). - `METHOD_NOT_ALLOWED` (405), `IDEMPOTENCY_KEY_CONFLICT` (409), `IDEMPOTENCY_IN_PROGRESS` (409), `IDEMPOTENCY_REPLAY_UNAVAILABLE` (409), and `IDEMPOTENCY_KEY_PAYLOAD_MISMATCH` (422). - `KEY_BUDGET_EXCEEDED` (402). The request is refused before the charge lands, and the key's own usage counter is left where it was. - Idempotent replays. The original charge already appears on your earlier request. Client errors (`MISSING_API_KEY`, `INVALID_API_KEY`, `INVALID_REQUEST`, `ENDPOINT_NOT_FOUND`) are rejected before any deduction. The one exception is an `INVALID_REQUEST` that only the data source can detect, such as a page past the end of a search: that call is charged and then refunded in full before the response is sent. See [Credits](/docs/credits.md#when-are-credits-refunded) for the full refund matrix. ## Handling retries Retry only the errors marked retryable above: `IDEMPOTENCY_IN_PROGRESS` (409), `RATE_LIMITED` or `CONCURRENCY_LIMIT` (429), and transient `500`, `502`, or `503` responses. Honor `Retry-After`: idempotency contention and claim-store outages send `1`, circuit/provider 503s send `30`, rate-limit 429s send the seconds until reset, and concurrency 429s send a short hint. Otherwise back off exponentially with full jitter, so a pool of your own workers does not resynchronize and stampede the cap together. Cap the attempts and surface the error rather than retrying forever. Never retry a `4xx` client error other than `429`: the request is deterministic and will fail the same way. Retryable failures attempt an immediate refund; if that database mutation fails, exact compensation is queued and the response keeps a non-zero committed `credits_used`. Add an `Idempotency-Key` header (see [Idempotency](#idempotency)) so a network-level retry of a successful sync call replays rather than re-runs. Each loop below retries only the transient statuses (429, 500, 502, 503, 504), honors `Retry-After` when present, falls back to exponential backoff with jitter, and gives up after five attempts. ```bash title="Terminal" #!/usr/bin/env bash url="https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" max_retries=5 attempt=0 while :; do body=$(mktemp); headers=$(mktemp) status=$(curl -s -o "$body" -D "$headers" -w '%{http_code}' \ -H "x-api-key: $SOCIALCRAWL_API_KEY" "$url") if [ "$status" -lt 400 ]; then cat "$body"; rm -f "$body" "$headers"; break fi case "$status" in 429|500|502|503|504) ;; # retryable, fall through *) echo "Non-retryable $status"; cat "$body"; rm -f "$body" "$headers"; exit 1 ;; esac attempt=$((attempt + 1)) if [ "$attempt" -gt "$max_retries" ]; then echo "Gave up after $max_retries retries (last status $status)" rm -f "$body" "$headers"; exit 1 fi # Retry-After (seconds) wins; else exponential backoff 2^attempt with jitter. retry_after=$(grep -i '^retry-after:' "$headers" | tr -d '\r' | awk '{print $2}') if [ -n "$retry_after" ]; then delay="$retry_after" else delay=$(awk "BEGIN{srand(); print (2 ^ $attempt) * (0.5 + rand() / 2)}") fi rm -f "$body" "$headers" sleep "$delay" done ``` ```python title="Python" import os import random import time import requests # pip install requests BASE_URL = "https://www.socialcrawl.dev/v1" RETRYABLE = {429, 500, 502, 503, 504} def get_with_retry(path, params=None, max_retries=5): headers = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]} for attempt in range(max_retries + 1): resp = requests.get(f"{BASE_URL}{path}", params=params, headers=headers) # Success, or a client error we must not retry: hand it back / raise. if resp.ok or resp.status_code not in RETRYABLE or attempt == max_retries: resp.raise_for_status() return resp.json() # Retry-After (seconds) wins; else exponential backoff with full jitter. retry_after = resp.headers.get("Retry-After") delay = float(retry_after) if retry_after else (2 ** attempt) * (0.5 + random.random() / 2) time.sleep(delay) data = get_with_retry("/tiktok/profile", {"handle": "charlidamelio"}) print(data) ``` ```js title="JavaScript" // Node 18+ (built-in fetch). No dependencies. const BASE_URL = "https://www.socialcrawl.dev/v1"; const RETRYABLE = new Set([429, 500, 502, 503, 504]); const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); async function getWithRetry(path, { params = {}, maxRetries = 5 } = {}) { const url = new URL(BASE_URL + path); for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v); const headers = { "x-api-key": process.env.SOCIALCRAWL_API_KEY }; for (let attempt = 0; ; attempt++) { const res = await fetch(url, { headers }); if (res.ok || !RETRYABLE.has(res.status)) { if (!res.ok) throw new Error(`SocialCrawl ${res.status}: ${await res.text()}`); return res.json(); } if (attempt >= maxRetries) { throw new Error( `Gave up after ${maxRetries} retries (last status ${res.status})`, ); } // Retry-After (seconds) wins; else exponential backoff with full jitter. const retryAfter = Number(res.headers.get("Retry-After")); const delayMs = Number.isFinite(retryAfter) && retryAfter > 0 ? retryAfter * 1000 : 2 ** attempt * 1000 * (0.5 + Math.random() / 2); await sleep(delayMs); } } const data = await getWithRetry("/tiktok/profile", { params: { handle: "charlidamelio" }, }); console.log(data); ``` ## Debugging tips - Every error envelope carries a `doc_url` pointing at the matching section on this page, for example `https://www.socialcrawl.dev/docs/errors#insufficient-credits`. - `request_id` matches the `X-Request-Id` header and the `request_id` column on Dashboard → Activity Logs. - Persistent 502s on a known-good input usually indicate an upstream outage. Check the [status page](/status), or `GET /v1/status`, for the platform's circuit state. - A high retry count on a slow-but-successful call is surfaced in the `X-Upstream-Retries` response header. ## Next steps - [Rate limits](/docs/rate-limits.md): The two 429s, the headroom headers, and how to size a batch job. - [Credits](/docs/credits.md): What each request costs, and the full refund matrix. - [Response schema](/docs/response-schema.md): The success envelope and every response header. - [Versioning](/docs/versioning.md): Why an error