# Response Schema (/docs/response-schema) The unified SocialCrawl response envelope, headers, and partial-data warnings Every SocialCrawl response uses the same envelope, on success and on error. Write one parser and it handles every platform. ## Successful response ```json title="Response" { "success": true, "platform": "tiktok", "endpoint": "/v1/tiktok/profile", "data": { "author": { "username": "charlidamelio", "followers": 156800000 }, "computed": { "engagement_rate": null, "language": null, "content_category": null, "estimated_reach": null } }, "credits_used": 1, "credits_remaining": 4999, "request_id": "req-XXXXX", "cached": false } ``` The `computed` leaves are `null` here because this abridged author carries no `likes_count` and no `bio`, and because `estimated_reach` is always `null` on a profile. Every computed field is a real value or an honest `null`, never a filled-in guess, so type your parser for `T | null` on all four. [Computed fields](/docs/computed-fields.md) states the exact rule per field. ### Envelope fields | Field | Type | Required | Description | | --- | --- | --- | --- | | `success` | `boolean` | yes | true for successful responses, false for errors. | | `platform` | `string` | yes | Platform identifier such as tiktok or instagram. Account endpoints like /v1/credits/balance report meta. | | `endpoint` | `string` | yes | The full endpoint path that was called. | | `data` | `object` | yes | Platform-specific, normalised payload. Lists are returned as items plus optional next_cursor and total. The computed sub-object is attached to every Author and Post. | | `credits_used` | `integer` | yes | Credits consumed by this request. 0 on cache hits and idempotent replays. | | `credits_remaining` | `integer \| null` | yes | Your current committed balance when known. null when a response cannot establish it without an auxiliary balance lookup. | | `request_id` | `string` | yes | Unique identifier in the form req-XXXXX, for debugging and log lookup. | | `cached` | `boolean` | yes | true when served from cache. Cache hits cost 0 credits. | | `pagination` | `object` | no | List endpoints only. The uniform pagination contract next_cursor, has_more, page_size, identical on every platform. | | `idempotent_replay` | `boolean` | no | Present and true only when this response is an idempotency replay. Absent on a normal response. | Two more fields live inside `data` rather than at the envelope root: `data.dropped` and `data._warnings[]`. ### `data.dropped`: Discarded-row counter On list endpoints, `data.dropped` counts upstream items omitted because they could not be repaired to the endpoint schema. A healthy list response carries `"dropped": 0`. It is the only signal that a page came back short, so read it on every page of a drain rather than inferring completeness from `items.length`. `dropped` sits next to `items` inside `data`, while `pagination` sits at the envelope root. Reading `response.dropped` returns `undefined` and silently looks like zero. ### `data._warnings[]`: Partial-data channel When the field-map or computed-field pipeline hits ambiguous upstream data, it attaches a human-readable notice to `data._warnings: string[]`: ```json title="Response" { "data": { "author": { "username": "..." }, "computed": { "engagement_rate": 1.0 }, "_warnings": [ "computed.engagement_rate: value exceeded 1.0 (raw: 1.42); clamped" ] } } ``` Treat entries as advisory. The response is still valid. Empty arrays are omitted, so `_warnings` is present only when there is something to report. ## Error response ```json title="Response" { "success": false, "error": { "type": "INSUFFICIENT_CREDITS", "message": "Your account has 0 credits remaining. This endpoint requires 1 credits.", "status": 402, "doc_url": "https://www.socialcrawl.dev/docs/errors#insufficient-credits" }, "credits_used": 0, "credits_remaining": 0, "request_id": "req-abc123" } ``` `credits_used` is the **net committed** charge for the failed request. It is `0` when no deduction occurred or the refund committed. If an immediate refund fails, it remains non-zero while exact compensation is queued; keep the `request_id` for reconciliation. `doc_url` is always the anchor form `/docs/errors#`, never a per-code sub-path. See [Errors](/docs/errors.md) for the full error-code table and refund rules. ## Response headers | Header | Value | | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `X-Request-Id` | Matches `request_id` in the body. Use it to correlate logs | | `X-Credits-Used` | Credits consumed. `0` on cache hits, empty-upstream 404s, 405/409/422, and idempotent replays | | `X-Credits-Remaining` | Current balance when known. On an idempotency replay, omitted when the balance lookup fails; body `credits_remaining` is `null` instead | | `X-Cache` | `HIT` (served from cache, 0 credits) or `MISS`. Force a `MISS` with `Cache-Control: no-cache` | | `X-Idempotent-Replay` | `"true"` on idempotent replays. Only present when the response was replayed | | `X-RateLimit-Limit` / `-Remaining` / `-Reset` | Request-window headroom, on every authenticated response. See [Rate limits](/docs/rate-limits.md) | | `X-Concurrency-Limit` / `-Remaining` | In-flight headroom, on every authenticated response. Not an alias of `X-RateLimit-*` | | `X-Upstream-Retries` | How many upstream retries this call needed. Omitted when there were none | | `Retry-After` | Seconds to wait. Sent on both `429`s (seconds until the window resets for `RATE_LIMITED`, a static `1` for `CONCURRENCY_LIMIT`), on a transient `503` (`30`), and on a `402` served from the credit cooldown (`60`). Absent otherwise | | `Allow` | Accepted method set on 405 responses: `"GET, HEAD"` for registry reads or `"POST"` for POST-only routes | ## List endpoints List archetypes (`PostList`, `CommentList`, `SearchResult`) are normalised to a consistent shape regardless of upstream key names: ```json title="Response" { "data": { "items": [ /* ... */ ], "next_cursor": "eyJwYWdlIjoyfQ==", "total": 1247 } } ``` The `data`-level `next_cursor` and `total` are legacy per-payload fields, included only when upstream provides them. **Use the top-level `pagination` block for pagination.** Every list response also carries a uniform `pagination` object at the envelope root. It is identical on all platforms and all three pagination styles (cursor, page, offset): ```json title="Response" { "data": { "items": [ /* ... */ ], "dropped": 0 }, "pagination": { "next_cursor": "sc.eyJ2IjoyLCJjIjoiMTc3NTM5NDk1MDAwMCJ9", "has_more": true, "page_size": 30 } } ``` The loop is the same everywhere. Read `pagination.next_cursor`, send it back as the `cursor` query parameter, and stop when `pagination.has_more` is `false`. You never handle a per-platform input-param name yourself. The API rewrites the `sc.` cursor to the endpoint's native param, `max_cursor` or `after` for example, on the way in. For the full drain recipe, see [Pagination](/docs/pagination.md). ## Idempotent requests Any `/v1/*` request can be made safely retriable by sending an `Idempotency-Key` header. A UUIDv4 is recommended. ```bash title="cURL" curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \ -H "x-api-key: YOUR_API_KEY" \ -H "Idempotency-Key: 7a5e1b4c-2d8f-4a3b-9c1e-6e8b4d2a1f3c" ``` ```ts title="TypeScript" const res = await fetch( "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio", { headers: { "x-api-key": process.env.SOCIALCRAWL_API_KEY!, "Idempotency-Key": crypto.randomUUID(), }, }, ); const body = await res.json(); console.log(body.idempotent_replay, body.credits_used); ``` ```python title="Python" import os import uuid import requests res = requests.get( "https://www.socialcrawl.dev/v1/tiktok/profile", params={"handle": "charlidamelio"}, headers={ "x-api-key": os.environ["SOCIALCRAWL_API_KEY"], "Idempotency-Key": str(uuid.uuid4()), }, ) body = res.json() print(body.get("idempotent_replay"), body["credits_used"]) ``` Replayed calls keep the cached payload immutable except for billing metadata: `credits_used` becomes 0 and `idempotent_replay` becomes `true`. A known current balance refreshes body `credits_remaining` and the `X-Credits-Remaining` header; no balance row resolves to 0. On a transient balance lookup failure, body `credits_remaining` is `null` and `X-Credits-Remaining` is omitted. GET endpoints cannot combine `Idempotency-Key` with `Accept: text/event-stream`, because an interrupted stream has no complete response body to replay. Request `Accept: application/json` when you need idempotent replay. The unsupported combination returns an unbilled `400 INVALID_REQUEST`. POST batch streams keep their documented batch replay behavior. Keys are scoped per account. A new request holds a 5-minute execution claim; settled outcomes remain replayable for 24 hours, subject to the 64 KB body cap. See [Errors](/docs/errors.md) for contention, replay-limit, claim-store, and payload-mismatch outcomes. ## Force a fresh fetch Responses from cache-enabled endpoints are cached and the cache is shared, so a repeat call returns instantly and costs 0 credits (`X-Cache: HIT`). To bypass the cache for a single call, send the standard `Cache-Control: no-cache` request header: ```bash title="cURL" curl 'https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio' \ -H "x-api-key: YOUR_API_KEY" \ -H "Cache-Control: no-cache" ``` The request fetches live from the source and is billed at the normal endpoint cost, a forced `MISS`, so `X-Cache: MISS` and a non-zero `X-Credits-Used`. The fresh result is written back, so your next plain call gets a free `HIT`. Only the `no-cache` directive triggers it. `Cache-Control: no-store` on its own does not. Registry-marked uncached endpoints skip this cache contract. For the freshness windows per data type, the shared-cache model, and guidance on when to bypass, see [Caching](/docs/caching.md). ## Next steps - [Pagination](/docs/pagination.md): The cursor drain loop, identical on every list endpoint. - [Computed fields](/docs/computed-fields.md): How engagement_rate, language, category, and reach are derived. - [Errors](/docs/errors.md): Every error type, its status, and whether it refunds. - [Caching](/docs/caching.md): Freshness windows, free hits, and forcing a live fetch.