Batch endpoints

Six endpoints take a whole array in one POST and return one row per input item. Use them when you need the same lookup for a list: a morning refresh of 400 post URLs, 60 handles to vet, 100 transcripts for a corpus.

Batch is not a different product. Each row is byte-identical to what the matching single endpoint returns. You trade N requests for one, and you pay for the rows that actually resolved.

The six endpoints

Authentication is the same x-api-key header as every other endpoint. These responses are never cached, so cached is always false. A batch refresh exists to read the live number.

However many items it carries, a batch counts as one request against the 600 per minute and 50 concurrent budgets. See Rate limits.

Making a request

Send a JSON body. The request body ceiling is 1 MB. A larger body is rejected with 413 PAYLOAD_TOO_LARGE before anything is parsed or charged.

curl -X POST 'https://www.socialcrawl.dev/v1/prism/profiles' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: vetting-batch-1" \
  -d '{
    "items": [
      { "platform": "tiktok", "handle": "@scout2015" },
      { "platform": "linkedin", "handle": "williamhgates", "custom_id": "vet-1" },
      { "platform": "instagram", "handle": "@nosuchaccount" }
    ]
  }'

const BASE = "https://www.socialcrawl.dev";

const res = await fetch(`${BASE}/v1/prism/profiles`, {
  method: "POST",
  headers: {
    "x-api-key": process.env.SOCIALCRAWL_API_KEY!,
    "Content-Type": "application/json",
    "Idempotency-Key": "vetting-batch-1",
  },
  body: JSON.stringify({
    items: [
      { platform: "tiktok", handle: "@scout2015" },
      { platform: "linkedin", handle: "williamhgates", custom_id: "vet-1" },
      { platform: "instagram", handle: "@nosuchaccount" },
    ],
  }),
});

const { data } = await res.json();
console.log(data.summary.ok, data.summary.credits_charged);

import os

import requests

res = requests.post(
    "https://www.socialcrawl.dev/v1/prism/profiles",
    headers={
        "x-api-key": os.environ["SOCIALCRAWL_API_KEY"],
        "Idempotency-Key": "vetting-batch-1",
    },
    json={
        "items": [
            {"platform": "tiktok", "handle": "@scout2015"},
            {"platform": "linkedin", "handle": "williamhgates", "custom_id": "vet-1"},
            {"platform": "instagram", "handle": "@nosuchaccount"},
        ]
    },
)

summary = res.json()["data"]["summary"]
print(summary["ok"], summary["credits_charged"])

The response

The four fan-out endpoints (post-stats, comment-lookup, profiles, transcripts) return per-row results plus a summary:

{
  "success": true,
  "platform": "prism",
  "endpoint": "/v1/prism/profiles",
  "data": {
    "results": [
      {
        "index": 0,
        "target": { "platform": "tiktok", "handle": "@scout2015" },
        "platform": "tiktok",
        "status": "ok",
        "data": { "author": { "username": "scout2015", "followers": 5300 } },
        "cost": 1,
        "fetched_at": "2026-07-04T09:00:03.128Z"
      },
      {
        "index": 1,
        "custom_id": "vet-1",
        "target": { "platform": "linkedin", "handle": "williamhgates" },
        "platform": "linkedin",
        "status": "ok",
        "data": {
          "author": { "username": "williamhgates", "followers": 36000000 }
        },
        "cost": 5,
        "fetched_at": "2026-07-04T09:00:03.771Z"
      },
      {
        "index": 2,
        "target": { "platform": "instagram", "handle": "@nosuchaccount" },
        "platform": "instagram",
        "status": "not_found",
        "cost": 0,
        "ext": { "reason": "empty_or_private" },
        "error": {
          "type": "NOT_FOUND",
          "message": "The profile returned no data (private, suspended, or deleted)."
        },
        "fetched_at": "2026-07-04T09:00:02.904Z"
      }
    ],
    "summary": {
      "total": 3,
      "ok": 2,
      "not_found": 1,
      "unsupported": 0,
      "error": 0,
      "deferred": 0,
      "coverage": 0.6666666666666666,
      "credits_charged": 6,
      "credits_refunded": 1
    },
    "legs": []
  },
  "credits_used": 6,
  "credits_remaining": 4521,
  "request_id": "req-abc123",
  "cached": false
}

Rows come back in input order, always all N of them, never paginated. custom_id is echoed verbatim when you supply it, so you can join rows back to your own records without relying on position. legs carries per-upstream call telemetry where the endpoint fans out. comment-lookup returns results and summary only.

Row statuses

Per-item isolation is the point: one dead link never fails the batch. Endpoint-specific detail arrives in ext rather than as new statuses. A caption-less video, for example, is not_found with ext.reason: "no_captions".

The summary

summary is the same nine fields on every fan-out batch, and the same object appears in the SSE done event, so the two can never disagree. coverage is the row ratio ok / total, not a credit ratio. credits_charged always equals the envelope's credits_used.

The YouTube id batches

POST /v1/youtube/videos and POST /v1/youtube/channels are chunk-metered rather than per-row, so they return the ordinary list shape: data.items and data.total, each item identical to the matching single endpoint, with no results or summary block.

They are billed by chunk: 5 credits per 50 ids, charged in full on a 200, because the upstream chunk is consumed whether or not every id in it resolves. A batch that resolves nothing returns 200 with { "items": [], "total": 0 } and is fully refunded, and a total upstream failure is fully refunded too.

Billing, worked

POST /v1/youtube/videos with 1000 ids: ceil(1000 / 50) = 20 chunks, 20 × 5 = 100 credits held upfront. If the upstream returns anything at all, that is the settled charge. If it resolves zero items, all 100 come back.

POST /v1/prism/profiles with 40 TikTok handles and 10 LinkedIn handles: the hold is 40 × 1 + 10 × 5 = 90 credits. If 3 TikTok handles are private and 1 LinkedIn handle is gone, the settled charge is 37 × 1 + 9 × 5 = 82, and 8 credits are refunded, visible as credits_refunded: 8 in the summary.

The pattern is the same everywhere: hold the worst case, settle to the rows that succeeded, refund the difference. You never pay for a not_found, unsupported, error, or deferred row.

HTTP status codes

A 502 still carries the per-row detail, so you can see which items failed and why rather than getting a bare error.

Idempotency and retrying the tail

Every batch endpoint honours an optional Idempotency-Key request header, checked before the credit hold. Keys live for 24 hours.

That makes the natural retry workflow safe:

The 45-second deadline

The fan-out endpoints run under a 45-second budget. Items still queued when it expires come back deferred at 0 credits, and their upstream was never touched. This is a pressure valve, not an error. Re-request the deferred items and they will run.

If you routinely see deferred rows, either lower the item count per call or stream the response so you can start processing rows the moment they land.

Streaming a batch

post-stats, comment-lookup, profiles, and transcripts stream when you send Accept: text/event-stream:

curl -N -X POST 'https://www.socialcrawl.dev/v1/prism/post-stats' \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{"urls": ["https://www.youtube.com/watch?v=dQw4w9WgXcQ"]}'

Each row is emitted the moment it settles, as a result event keyed by row type (stat, item, profile, or transcript), followed by a terminal done event carrying the same nine-field summary as the sync response. Billing is identical either way. An SSE batch is stored for idempotent replay as its sync-JSON equivalent, so a retry with the same key replays whichever wire format it asks for. See Streaming (SSE) for the full frame contract.

The upfront hold is settled through one serialized lifecycle shared by normal completion, timeout, error, and disconnect recovery. If an immediate refund fails, the terminal state keeps the outstanding committed charge and queues that exact remainder for automatic compensation; a late executor cannot race the recovery path into a duplicate refund.

POST /v1/youtube/videos and POST /v1/youtube/channels do not stream.

When to use batch instead

Next steps