SocialCrawl

Response Schema

The unified SocialCrawl response envelope, headers, and partial-data warnings

Response Schema

Every SocialCrawl response — success or error — uses the same envelope. Write one parser, handle every platform.

Successful Response

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile",
  "data": {
    "author": { "username": "charlidamelio", "followers": 156800000 },
    "computed": {
      "engagement_rate": 0.082,
      "language": "en",
      "content_category": "entertainment",
      "estimated_reach": 12857600
    }
  },
  "credits_used": 1,
  "credits_remaining": 4999,
  "request_id": "req-XXXXX",
  "cached": false
}

Envelope Fields

FieldTypeDescription
successbooleantrue for successful responses, false for errors
platformstringPlatform identifier (tiktok, instagram, etc.) — meta for account endpoints like /v1/credits/balance
endpointstringThe full endpoint path that was called
dataobjectPlatform-specific, normalised payload. Lists are returned as { items, next_cursor?, total? }. The computed sub-object is attached to every Author and Post — see Computed fields
credits_usedintegerCredits consumed by this request (0 on cache hits and idempotent replays)
credits_remainingintegerYour balance after this request
request_idstringUnique identifier (req-XXXXX) for debugging and log lookup
droppedintegerList endpoints only. Count of upstream items omitted because they could not be repaired to the endpoint schema. Healthy list responses include 0
cachedbooleantrue when served from cache — cache hits cost 0 credits
paginationobjectList endpoints only. The uniform pagination contract { next_cursor, has_more, page_size } — identical on every platform. See List endpoints below and Pagination

data._warnings[] — Partial-data channel

When the field-map or computed-field pipeline hits ambiguous upstream data, it attaches a human-readable notice to data._warnings: string[]:

{
  "data": {
    "author": { "username": "..." },
    "computed": { "engagement_rate": 1.0 },
    "_warnings": ["engagement_rate clamped from 1.42 to 1.0"]
  }
}

Treat entries as advisory — the response is still valid. Empty arrays are omitted from the envelope, so _warnings is only present when there is something to report.

Error Response

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 0 credits remaining. This endpoint requires 1 credits.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors/insufficient-credits"
  },
  "credits_remaining": 0,
  "request_id": "req-abc123"
}

See Error Handling for the full error-code table and refund rules.

Response Headers

HeaderValue
X-Request-IdMatches request_id in the body — use it to correlate logs
X-Credits-UsedCredits consumed (0 on cache hits, empty-upstream 404s, 405/409/422, and idempotent replays)
X-Credits-RemainingBalance after this request
X-CacheHIT (served from cache, 0 credits) or MISS. Force a MISS with Cache-Control: no-cache (see Caching)
X-Idempotent-Replay"true" on idempotent replays (only present when the response was replayed). See Idempotent requests
Retry-After"30" (seconds) — only on 503 circuit-breaker responses
Allow"GET" — only on 405 METHOD_NOT_ALLOWED responses

List endpoints

List archetypes (PostList, CommentList, SearchResult) are normalised to a consistent shape regardless of upstream key names:

{
  "data": {
    "items": [
      /* ... */
    ],
    "next_cursor": "eyJwYWdlIjoyfQ==",
    "total": 1247
  }
}

The data-level next_cursor and total are legacy per-payload fields, only included when upstream provides them.

Use the top-level pagination block for pagination. Every list response also carries a uniform pagination object at the envelope root — identical on all platforms and all three pagination styles (cursor, page, offset):

{
  "data": {
    "items": [
      /* ... */
    ]
  },
  "dropped": 0,
  "pagination": {
    "next_cursor": "sc.eyJ2IjoyLCJjIjoiMTc3NTM5NDk1MDAwMCJ9",
    "has_more": true,
    "page_size": 30
  }
}

The loop is the same everywhere: read pagination.next_cursor, send it back as the cursor query parameter, and stop when pagination.has_more is false. You never handle a per-platform input-param name yourself — the API rewrites the sc. cursor to the endpoint's native param (max_cursor, after, etc.) on the way in. For the full drain recipe, see Pagination.

Idempotent requests

Any /v1/* request can be made safely retriable by sending an Idempotency-Key header (UUIDv4 recommended):

GET /v1/tiktok/profile?handle=charlidamelio HTTP/1.1
x-api-key: sc_...
Idempotency-Key: 7a5e1b4c-2d8f-4a3b-9c1e-6e8b4d2a1f3c

Replayed calls return the original response verbatim, deduct 0 new credits, and add X-Idempotent-Replay: true. Keys are scoped per account with a 24-hour TTL. See the Error Handling page for the 409/422 outcomes when a key is reused incorrectly.

Force a fresh fetch

Responses are cached and shared across your account, so repeat calls return instantly and cost 0 credits (X-Cache: HIT). To bypass the cache and force a guaranteed-live fetch for a single call, send the standard Cache-Control: no-cache request header:

GET /v1/tiktok/profile?handle=charlidamelio HTTP/1.1
x-api-key: sc_...
Cache-Control: no-cache

The request fetches live from the source and is billed at the normal endpoint cost (a forced MISS, so X-Cache: MISS and a non-zero X-Credits-Used). The fresh result is written back, so your next plain call gets a free HIT. Only the no-cache directive triggers it; Cache-Control: no-store on its own does not.

For the freshness windows per data type, the shared-cache model, and guidance on when to bypass the cache, see the Caching page.

Response Schema | SocialCrawl