Response Schema

Every SocialCrawl response uses the same envelope, on success and on error. Write one parser and it handles every platform.

Successful response

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile",
  "data": {
    "author": { "username": "charlidamelio", "followers": 156800000 },
    "computed": {
      "engagement_rate": null,
      "language": null,
      "content_category": null,
      "estimated_reach": null
    }
  },
  "credits_used": 1,
  "credits_remaining": 4999,
  "request_id": "req-XXXXX",
  "cached": false
}

The computed leaves are null here because this abridged author carries no likes_count and no bio, and because estimated_reach is always null on a profile. Every computed field is a real value or an honest null, never a filled-in guess, so type your parser for T | null on all four. Computed fields states the exact rule per field.

Envelope fields

• successbooleanrequired

  true for successful responses, false for errors.

• platformstringrequired

  Platform identifier such as tiktok or instagram. Account endpoints like /v1/credits/balance report meta.

• endpointstringrequired

  The full endpoint path that was called.

• dataobjectrequired

  Platform-specific, normalised payload. Lists are returned as items plus optional next_cursor and total. The computed sub-object is attached to every Author and Post.

• credits_usedintegerrequired

  Credits consumed by this request. 0 on cache hits and idempotent replays.

• credits_remaininginteger | nullrequired

  Your current committed balance when known. null when a response cannot establish it without an auxiliary balance lookup.

• request_idstringrequired

  Unique identifier in the form req-XXXXX, for debugging and log lookup.

• cachedbooleanrequired

  true when served from cache. Cache hits cost 0 credits.

• paginationobject

  List endpoints only. The uniform pagination contract next_cursor, has_more, page_size, identical on every platform.

• idempotent_replayboolean

  Present and true only when this response is an idempotency replay. Absent on a normal response.

Two more fields live inside data rather than at the envelope root: data.dropped and data._warnings[].

data.dropped: Discarded-row counter

On list endpoints, data.dropped counts upstream items omitted because they could not be repaired to the endpoint schema. A healthy list response carries "dropped": 0. It is the only signal that a page came back short, so read it on every page of a drain rather than inferring completeness from items.length.

data._warnings[]: Partial-data channel

When the field-map or computed-field pipeline hits ambiguous upstream data, it attaches a human-readable notice to data._warnings: string[]:

{
  "data": {
    "author": { "username": "..." },
    "computed": { "engagement_rate": 1.0 },
    "_warnings": [
      "computed.engagement_rate: value exceeded 1.0 (raw: 1.42); clamped"
    ]
  }
}

Treat entries as advisory. The response is still valid. Empty arrays are omitted, so _warnings is present only when there is something to report.

Error response

{
  "success": false,
  "error": {
    "type": "INSUFFICIENT_CREDITS",
    "message": "Your account has 0 credits remaining. This endpoint requires 1 credits.",
    "status": 402,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits"
  },
  "credits_used": 0,
  "credits_remaining": 0,
  "request_id": "req-abc123"
}

credits_used is the net committed charge for the failed request. It is 0 when no deduction occurred or the refund committed. If an immediate refund fails, it remains non-zero while exact compensation is queued; keep the request_id for reconciliation.

doc_url is always the anchor form /docs/errors#<code-slug>, never a per-code sub-path. See Errors for the full error-code table and refund rules.

Response headers

List endpoints

List archetypes (PostList, CommentList, SearchResult) are normalised to a consistent shape regardless of upstream key names:

{
  "data": {
    "items": [
      /* ... */
    ],
    "next_cursor": "eyJwYWdlIjoyfQ==",
    "total": 1247
  }
}

The data-level next_cursor and total are legacy per-payload fields, included only when upstream provides them.

Use the top-level pagination block for pagination. Every list response also carries a uniform pagination object at the envelope root. It is identical on all platforms and all three pagination styles (cursor, page, offset):

{
  "data": {
    "items": [
      /* ... */
    ],
    "dropped": 0
  },
  "pagination": {
    "next_cursor": "sc.eyJ2IjoyLCJjIjoiMTc3NTM5NDk1MDAwMCJ9",
    "has_more": true,
    "page_size": 30
  }
}

The loop is the same everywhere. Read pagination.next_cursor, send it back as the cursor query parameter, and stop when pagination.has_more is false. You never handle a per-platform input-param name yourself. The API rewrites the sc. cursor to the endpoint's native param, max_cursor or after for example, on the way in. For the full drain recipe, see Pagination.

Idempotent requests

Any /v1/* request can be made safely retriable by sending an Idempotency-Key header. A UUIDv4 is recommended.

curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Idempotency-Key: 7a5e1b4c-2d8f-4a3b-9c1e-6e8b4d2a1f3c"

const res = await fetch(
  "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio",
  {
    headers: {
      "x-api-key": process.env.SOCIALCRAWL_API_KEY!,
      "Idempotency-Key": crypto.randomUUID(),
    },
  },
);

const body = await res.json();
console.log(body.idempotent_replay, body.credits_used);

import os
import uuid

import requests

res = requests.get(
    "https://www.socialcrawl.dev/v1/tiktok/profile",
    params={"handle": "charlidamelio"},
    headers={
        "x-api-key": os.environ["SOCIALCRAWL_API_KEY"],
        "Idempotency-Key": str(uuid.uuid4()),
    },
)

body = res.json()
print(body.get("idempotent_replay"), body["credits_used"])

Replayed calls keep the cached payload immutable except for billing metadata: credits_used becomes 0 and idempotent_replay becomes true. A known current balance refreshes body credits_remaining and the X-Credits-Remaining header; no balance row resolves to 0. On a transient balance lookup failure, body credits_remaining is null and X-Credits-Remaining is omitted.

GET endpoints cannot combine Idempotency-Key with Accept: text/event-stream, because an interrupted stream has no complete response body to replay. Request Accept: application/json when you need idempotent replay. The unsupported combination returns an unbilled 400 INVALID_REQUEST. POST batch streams keep their documented batch replay behavior.

Keys are scoped per account. A new request holds a 5-minute execution claim; settled outcomes remain replayable for 24 hours, subject to the 64 KB body cap. See Errors for contention, replay-limit, claim-store, and payload-mismatch outcomes.

Force a fresh fetch

Responses from cache-enabled endpoints are cached and the cache is shared, so a repeat call returns instantly and costs 0 credits (X-Cache: HIT). To bypass the cache for a single call, send the standard Cache-Control: no-cache request header:

curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Cache-Control: no-cache"

The request fetches live from the source and is billed at the normal endpoint cost, a forced MISS, so X-Cache: MISS and a non-zero X-Credits-Used. The fresh result is written back, so your next plain call gets a free HIT. Only the no-cache directive triggers it. Cache-Control: no-store on its own does not. Registry-marked uncached endpoints skip this cache contract.

For the freshness windows per data type, the shared-cache model, and guidance on when to bypass, see Caching.

Next steps