Error Handling

All errors follow the same envelope format:

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 0 credits remaining. This endpoint requires 1 credits.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits"
  },
  "credits_remaining": 0,
  "request_id": "req-abc123"
}

Every response — success or error — includes an X-Request-Id header matching request_id. Include it when contacting support so we can trace the exact request in our logs.

The doc_url on every error points at the matching section on this page. Jump straight to a code with its anchor, e.g. #insufficient-credits or #upstream-error.

Error Codes

Only the four transient failures (CONCURRENCY_LIMIT, UPSTREAM_ERROR, SERVICE_UNAVAILABLE, INTERNAL_ERROR) are safe to retry. Every client error (4xx other than 429) is deterministic: the same request will fail the same way, so retrying wastes time and credits are never at stake because none were deducted. See Handling retries for copy-paste backoff loops.

Error Code Reference

Each code below carries a stable anchor — the exact target of the doc_url in the error envelope.

MISSING_API_KEY

401 — No x-api-key header was present on the request. Add the header with one of your keys from Dashboard → API Keys. No credits are deducted.

INVALID_API_KEY

401 — The key is malformed, not found, revoked, or expired. Check the key in Dashboard → API Keys. No credits are deducted.

INSUFFICIENT_CREDITS

402 — Your balance is lower than the endpoint's cost. Top up in Dashboard → Billing. No credits are deducted. credits_remaining reports your current balance.

INVALID_REQUEST

400 — A required parameter is missing, a value failed format validation, or no oneOf member was satisfied. Check the endpoint's required parameters in the API Reference. No credits are deducted.

METHOD_NOT_ALLOWED

405 — A non-GET method hit a GET-only /v1/* endpoint. The response includes an Allow: GET header. Switch to GET. No credits are deducted.

ENDPOINT_NOT_FOUND

404 — The platform or resource is not supported. Check the API Reference for the correct path. No credits are deducted.

RESOURCE_NOT_FOUND

404 — The upstream returned 404, or the resource exists but contains no usable data (e.g. a nonexistent handle). Verify the handle or URL. Credits are refunded when this is triggered by an empty upstream body.

IDEMPOTENCY_KEY_CONFLICT

409 — The Idempotency-Key you sent is already in use by another account. Generate a fresh key (a UUIDv4 is recommended). No credits are deducted.

IDEMPOTENCY_KEY_PAYLOAD_MISMATCH

422 — You reused an Idempotency-Key with a different payload — different query parameters on a GET, or a different JSON body on a POST. Use a new key for the new payload. No credits are deducted.

PAYLOAD_TOO_LARGE

413: The request's JSON body exceeded the 1 MB size limit and was rejected before parsing, so no credits were deducted. Reduce the batch size, for example by splitting a large ids or urls array across several requests.

CONCURRENCY_LIMIT

429 — More than 50 concurrent requests were in flight on the same API key. Reduce concurrency and retry. No credits are deducted.

UPSTREAM_ERROR

502 — The upstream platform returned an error (after our automatic retries). Retry after a short backoff. Credits are refunded. To tell an upstream outage from a transient blip, check the live status page or GET /v1/status for the platform's current circuit state.

SERVICE_UNAVAILABLE

503 — The circuit breaker is open for this platform after repeated upstream failures. The response includes Retry-After: 30. Wait 30 seconds and retry. Credits are refunded.

INTERNAL_ERROR

500 — An unexpected error occurred on our side. Retry; if it persists, contact support with the request_id. Credits are refunded.

Idempotency

Send an Idempotency-Key header to make a charging request safe to retry. The key is honored on every charging endpoint — the GET /v1/* catch-all and all four POST endpoints (/v1/prism/post-stats, /v1/prism/comment-lookup, /v1/youtube/videos, /v1/youtube/channels).

• A retry with the same key and the same payload replays the stored response verbatim, charges nothing (X-Credits-Used: 0, X-Idempotent-Replay: true), and does not re-run the upstream call — so a network retry or a cron double-fire never double-charges or double-creates.
• The key's identity covers the request payload: query parameters for GET, and the canonical JSON body for POST. Reusing a key with a different payload returns IDEMPOTENCY_KEY_PAYLOAD_MISMATCH (422) rather than a false replay.
• A key already claimed by another account returns IDEMPOTENCY_KEY_CONFLICT (409).
• Keys are retained for 24 hours. Use a fresh UUIDv4 per logical operation.

Refund Rules

Credits are automatically refunded when:

• Upstream returns a 5xx error (UPSTREAM_ERROR, 502)
• The circuit breaker rejects the request (SERVICE_UNAVAILABLE, 503)
• An unexpected server error occurs (INTERNAL_ERROR, 500)
• Upstream returns 200 with an empty body — e.g. a nonexistent handle — triggering RESOURCE_NOT_FOUND (404)

Credits are never deducted for these outcomes, so no refund is needed:

• Cache hits (X-Cache: HIT, X-Credits-Used: 0)
• METHOD_NOT_ALLOWED (405), IDEMPOTENCY_KEY_CONFLICT (409), and IDEMPOTENCY_KEY_PAYLOAD_MISMATCH (422)
• Idempotent replays — the original charge already appears on your earlier request

Client errors (MISSING_API_KEY, INVALID_API_KEY, INVALID_REQUEST, ENDPOINT_NOT_FOUND) are rejected before any deduction.

Handling retries

Retry only the four statuses marked retryable above: 429 (CONCURRENCY_LIMIT), 500, 502, and 503. When a response carries a Retry-After header (a 503 always sends Retry-After: 30; a 429 sends a short hint), wait that long before the first retry. Otherwise back off exponentially with full jitter so a pool of your own workers does not resynchronize and stampede the cap together. Cap the attempts and surface the error rather than retrying forever. Never retry a 4xx client error other than 429: the request is deterministic and will fail the same way.

Because credits are refunded on every retryable failure, a retry loop never double-charges. Add an Idempotency-Key header (see Idempotency) if you also want a network-level retry of a successful call to replay rather than re-run.

#!/usr/bin/env bash
# Retry only transient failures (429, 500, 502, 503, 504). Honor Retry-After
# when present, else exponential backoff with jitter. Give up after 5 tries.
url="https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio"
max_retries=5
attempt=0

while :; do
  body=$(mktemp); headers=$(mktemp)
  status=$(curl -s -o "$body" -D "$headers" -w '%{http_code}' \
    -H "x-api-key: $SOCIALCRAWL_API_KEY" "$url")

  if [ "$status" -lt 400 ]; then
    cat "$body"; rm -f "$body" "$headers"; break
  fi

  case "$status" in
    429|500|502|503|504) ;;  # retryable, fall through
    *) echo "Non-retryable $status"; cat "$body"; rm -f "$body" "$headers"; exit 1 ;;
  esac

  attempt=$((attempt + 1))
  if [ "$attempt" -gt "$max_retries" ]; then
    echo "Gave up after $max_retries retries (last status $status)"
    rm -f "$body" "$headers"; exit 1
  fi

  # Retry-After (seconds) wins; else exponential backoff 2^attempt with jitter.
  retry_after=$(grep -i '^retry-after:' "$headers" | tr -d '\r' | awk '{print $2}')
  if [ -n "$retry_after" ]; then
    delay="$retry_after"
  else
    delay=$(awk "BEGIN{srand(); print (2 ^ $attempt) * (0.5 + rand() / 2)}")
  fi
  rm -f "$body" "$headers"
  sleep "$delay"
done

import os
import random
import time

import requests  # pip install requests

BASE_URL = "https://www.socialcrawl.dev/v1"
RETRYABLE = {429, 500, 502, 503, 504}


def get_with_retry(path, params=None, max_retries=5):
    headers = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]}
    for attempt in range(max_retries + 1):
        resp = requests.get(f"{BASE_URL}{path}", params=params, headers=headers)
        # Success, or a client error we must not retry: hand it back / raise.
        if resp.ok or resp.status_code not in RETRYABLE or attempt == max_retries:
            resp.raise_for_status()
            return resp.json()
        # Retry-After (seconds) wins; else exponential backoff with full jitter.
        retry_after = resp.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else (2 ** attempt) * (0.5 + random.random() / 2)
        time.sleep(delay)


data = get_with_retry("/tiktok/profile", {"handle": "charlidamelio"})
print(data)

// Node 18+ (built-in fetch). No dependencies.
const BASE_URL = "https://www.socialcrawl.dev/v1";
const RETRYABLE = new Set([429, 500, 502, 503, 504]);
const sleep = (ms) => new Promise((r) => setTimeout(r, ms));

async function getWithRetry(path, { params = {}, maxRetries = 5 } = {}) {
  const url = new URL(BASE_URL + path);
  for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
  const headers = { "x-api-key": process.env.SOCIALCRAWL_API_KEY };

  for (let attempt = 0; ; attempt++) {
    const res = await fetch(url, { headers });
    if (res.ok || !RETRYABLE.has(res.status)) {
      if (!res.ok) throw new Error(`SocialCrawl ${res.status}: ${await res.text()}`);
      return res.json();
    }
    if (attempt >= maxRetries) {
      throw new Error(`Gave up after ${maxRetries} retries (last status ${res.status})`);
    }
    // Retry-After (seconds) wins; else exponential backoff with full jitter.
    const retryAfter = Number(res.headers.get("Retry-After"));
    const delayMs =
      Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000
        : 2 ** attempt * 1000 * (0.5 + Math.random() / 2);
    await sleep(delayMs);
  }
}

const data = await getWithRetry("/tiktok/profile", { params: { handle: "charlidamelio" } });
console.log(data);

Debugging Tips

• Every error envelope carries a doc_url pointing at the matching section on this page, e.g. https://www.socialcrawl.dev/docs/errors#insufficient-credits
• request_id matches the X-Request-Id header and the request_id column on Dashboard → Activity Logs
• Persistent 502s on a known-good input usually indicate an upstream outage — check the status page (or GET /v1/status) for the platform's circuit state
• A high retry count on a slow-but-successful call is surfaced in the X-Upstream-Retries response header