# Async requests (/docs/async)

Run any GET endpoint in the background with Prefer respond-async or async=true, get a 202 and a job to poll, and read the exact response later. Also how to run up to 1,000 calls to one GET endpoint as a single job.

Some calls take longer than your client waits. `youtube/channel/about`, the transcript endpoints and `twitter/ai-search` can each run for 30 seconds or more, while the MCP server and many HTTP clients stop waiting at 30 seconds. Any GET endpoint can run in the background instead: you get a job straight away, and you read the finished response when it is ready.

The call is the same call. It is validated before the job is created, then served, cached, billed and refunded exactly as it would be if you had waited for it.

## Run one GET in the background

Add the header `Prefer: respond-async` ([RFC 7240](https://www.rfc-editor.org/rfc/rfc7240)) to the call you already make:

```bash title="cURL"
curl "https://www.socialcrawl.dev/v1/youtube/channel/about?handle=@mkbhd" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Prefer: respond-async"
```

If your client cannot set a header (an agent's fetch tool, a no-code HTTP step), add `async=true` to the query instead:

```bash title="cURL"
curl "https://www.socialcrawl.dev/v1/youtube/channel/about?handle=@mkbhd&async=true" \
  -H "x-api-key: YOUR_API_KEY"
```

The answer is `202 Accepted` with the job. The `Location` header is the URL to poll, `Retry-After` says how many seconds to wait first, and `Preference-Applied: respond-async` confirms the header was honoured:

```json title="Response"
{
  "success": true,
  "platform": "prism",
  "endpoint": "/v1/prism/jobs",
  "data": {
    "job_id": "bj_Vn3kQ8sLx2",
    "endpoint": "youtube/channel/about",
    "billing": "per_call",
    "status": "queued",
    "item_count": 1,
    "chunk_count": 1,
    "chunks_done": 0,
    "credits_hold": 0,
    "credits_charged": 0,
    "credits_max": 25,
    "summary": null,
    "webhook_url": null,
    "results_url": "https://www.socialcrawl.dev/v1/prism/jobs/bj_Vn3kQ8sLx2",
    "created_at": "2026-10-24T09:00:00.000Z",
    "completed_at": null,
    "expires_at": "2026-10-31T09:00:00.000Z"
  },
  "credits_used": 0,
  "credits_remaining": 4521,
  "request_id": "req-abc123",
  "cached": false
}
```

`async=false` or `async=0` is an ordinary call. Any other value is a free `400`, so a typo never runs a different call. `dry_run=1` still wins: the call is validated and quoted for 0 credits and no job is created.

## Read the result

Poll `GET /v1/prism/jobs/{job_id}` (the `Location` URL). Reading a job is free:

```bash title="cURL"
curl "https://www.socialcrawl.dev/v1/prism/jobs/bj_Vn3kQ8sLx2" \
  -H "x-api-key: YOUR_API_KEY"
```

While the job runs, `status` is `queued` or `running` and the answer carries `Retry-After`. Once it is `completed`, `results[0].response` is exactly the body the GET would have returned, envelope and all:

```json title="Response"
{
  "success": true,
  "data": {
    "job_id": "bj_Vn3kQ8sLx2",
    "endpoint": "youtube/channel/about",
    "billing": "per_call",
    "status": "completed",
    "credits_charged": 25,
    "summary": { "total": 1, "ok": 1, "not_found": 0, "unsupported": 0, "error": 0, "deferred": 0, "coverage": 1, "credits_charged": 25, "credits_refunded": 0 },
    "results": [
      {
        "index": 0,
        "status": "ok",
        "http_status": 200,
        "params": { "handle": "@mkbhd" },
        "credits_used": 25,
        "response": { "success": true, "data": { "...": "the channel, as the GET returns it" }, "credits_used": 25 }
      }
    ],
    "next_cursor": null,
    "results_complete": true
  },
  "credits_used": 0
}
```

| Row field | Meaning |
| --------- | ------- |
| `index` | The item's position in the job (0 for a single GET). |
| `status` | `ok` (2xx), `not_found` (404) or `error` (anything else). |
| `http_status` | The status the GET answered with. |
| `params` | The query the GET ran with. |
| `credits_used` | What that call charged. A cached, failed or not-found call charges what it would have charged directly, usually 0. |
| `response` | The GET's full body, including an error envelope when it failed, so you can handle it exactly as you handle a direct call. |

A minimal poll loop:

```ts title="TypeScript"
const headers = { "x-api-key": process.env.SOCIALCRAWL_API_KEY! };

const submit = await fetch(
  "https://www.socialcrawl.dev/v1/youtube/channel/about?handle=@mkbhd",
  { headers: { ...headers, Prefer: "respond-async" } },
);
const jobUrl = submit.headers.get("location")!;

for (;;) {
  const res = await fetch(jobUrl, { headers });
  const { data } = await res.json();
  if (data.status === "completed" || data.status === "failed") {
    console.log(data.results[0].response);
    break;
  }
  const wait = Number(res.headers.get("retry-after") ?? "5");
  await new Promise((r) => setTimeout(r, wait * 1000));
}
```

## Run many calls to one GET as a job

To run the same endpoint for a list, send the queries to `POST /v1/prism/jobs` with the endpoint's path. Each item is one call's query parameters:

```bash title="cURL"
curl -X POST "https://www.socialcrawl.dev/v1/prism/jobs" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "endpoint": "youtube/channel/about",
    "items": [
      { "handle": "@mkbhd" },
      { "handle": "@veritasium" },
      { "channelId": "UCX6OQ3DkcsbYNE6H8uQQuVA" }
    ],
    "webhook": { "url": "https://hooks.yourapp.com/socialcrawl" }
  }'
```

- `endpoint` is any GET endpoint's path, with or without the leading `/v1/`.
- `items` holds 1 to 1,000 objects. A number or boolean value is sent as text, and a list is sent comma-separated.
- Every item is validated as that GET would be before the job is stored. A bad item is a free `400` that names its position, for example `Item 2: Missing required parameter: handle`, and nothing is created.
- Items run 3 at a time, each under the same time limit as a direct call, and results page five chunks at a time through `next_cursor`.
- `webhook` works as it does for [batch jobs](/docs/batch.md#the-completion-webhook): one signed `batch.completed` call with the summary and the results URL.

## How an async call is billed

An async GET holds nothing when you submit it, and `credits_used` on the `202` is 0. Each item is billed as that GET when it runs, at the same price, with the same cache and the same refunds, and `credits_charged` on the job adds them up. `credits_max` on the `202` is the most the job can cost.

If your balance cannot cover one call, the submit is the same `402 INSUFFICIENT_CREDITS` the direct call would give, and no job is created. A job whose every call failed ends `failed`.

Each item carries a fixed idempotency key, so if a worker restarts mid-job, an item that already ran is replayed, not billed twice. Send your own `Idempotency-Key` on the submit and a retry returns the same job instead of creating a second one.

## When to use it

| You want | Use |
| -------- | --- |
| A slow call that can outlast your client's timeout | `Prefer: respond-async` or `async=true` on that GET |
| The same GET for a list of handles, URLs or ids | `POST /v1/prism/jobs` with `endpoint` set to that GET |
| Thousands of profiles or post stats at batch prices | `POST /v1/prism/jobs` with `prism/profiles` or `prism/post-stats` ([batch jobs](/docs/batch.md#background-jobs-for-thousands-of-items)) |
| Results as they arrive, on an open connection | [Streaming](/docs/streaming.md) |

Two endpoints work differently. `prism/discover` has its own background runs: send its own `async=true`, and the router leaves both the parameter and `Prefer` to it. `prism/answers` only answers as a stream, so it cannot run as a job: call it with `Accept: text/event-stream`.

Jobs and their results are kept for 7 days. `GET /v1/prism/jobs` lists your key's jobs, newest first.