Pagination

Every list endpoint on every platform paginates the same way. Learn it once and it works on all 123 paginatable endpoints: TikTok, Instagram, YouTube, Reddit, LinkedIn, Naver, all of them.

The one rule

Read pagination.next_cursor from the response, send it back as the cursor query parameter, and stop when pagination.has_more is false.

That is the entire contract. You never construct a cursor, decode a token, or look up a platform's native pagination param. The API hands you an opaque token and takes it straight back.

The response block

Every list response carries a top-level pagination block:

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile/videos",
  "data": {
    "items": [ /* … this page of results … */ ],
    "total": 1247
  },
  "credits_used": 1,
  "credits_remaining": 98,
  "request_id": "req-abc123",
  "cached": false,
  "pagination": {
    "next_cursor": "sc.eyJ2IjoyLCJjIjoiNzM4…",
    "has_more": true,
    "page_size": 30
  }
}

The token is a single sc. string that works identically for cursor-, page-, and offset-style upstreams, so your loop is byte-for-byte the same everywhere. (data.total, when present, is the upstream's own result count, useful for a progress bar, but never use it to decide when to stop; has_more is the only always-correct signal.)

A worked round-trip

Request page 1 (no cursor):

curl "https://www.socialcrawl.dev/v1/tiktok/profile/videos?handle=mrbeast" \
  -H "x-api-key: YOUR_API_KEY"

The response comes back with "pagination": { "next_cursor": "sc.eyJ2Ijoy…", "has_more": true, "page_size": 30 }.

Request page 2 (pass that next_cursor back as cursor):

curl "https://www.socialcrawl.dev/v1/tiktok/profile/videos?handle=mrbeast&cursor=sc.eyJ2Ijoy…" \
  -H "x-api-key: YOUR_API_KEY"

Keep going until a response returns "has_more": false (and "next_cursor": null). Swap tiktok/profile/videos for any list endpoint in the table below and this is unchanged.

The loop

Identical for every list endpoint on every platform:

async function* paginate(path: string, params: Record<string, string>) {
  let cursor: string | null = null;
  do {
    const url = new URL(`https://www.socialcrawl.dev/v1/${path}`);
    for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
    if (cursor) url.searchParams.set("cursor", cursor); // <-- always "cursor"

    const res = await fetch(url, {
      headers: { "x-api-key": process.env.SOCIALCRAWL_KEY! },
    });
    const json = await res.json();

    for (const item of json.data.items) yield item;

    cursor = json.pagination.next_cursor; // <-- send back verbatim next round
  } while (cursor); // <-- pagination.has_more === false ends the loop
}

// Usage: same call shape for TikTok, Instagram, YouTube, Reddit, …
for await (const video of paginate("tiktok/profile/videos", { handle: "mrbeast" })) {
  // … handle each item
}

Two things to get right:

1. Send next_cursor back as the EXACT string you received, under the key cursor. Don't decode, trim, or re-encode it; it's an opaque token.
2. Stop when has_more is false (equivalently, when next_cursor is null). Don't infer the end from an empty page or from total.

Recipe: drain a full account safely

The real-world loop is "pull every post/comment/follower an account has" without burning credits or tripping limits. Walk pages until has_more is false, and keep three things in mind:

// Drain every video for a handle, then stop.
async function drain(path: string, params: Record<string, string>) {
  const all: unknown[] = [];
  let cursor: string | null = null;
  let pages = 0;

  do {
    const url = new URL(`https://www.socialcrawl.dev/v1/${path}`);
    for (const [k, v] of Object.entries(params)) url.searchParams.set(k, v);
    if (cursor) url.searchParams.set("cursor", cursor);

    const res = await fetch(url, {
      headers: { "x-api-key": process.env.SOCIALCRAWL_KEY! },
    });

    // Respect backpressure: on 429 (CONCURRENCY_LIMIT) or 503 back off and retry
    // the SAME cursor; never advance past a page you didn't successfully read.
    if (res.status === 429 || res.status === 503) {
      const wait = Number(res.headers.get("retry-after") ?? "2") * 1000;
      await new Promise((r) => setTimeout(r, wait));
      continue;
    }

    const json = await res.json();
    all.push(...json.data.items);
    cursor = json.pagination.next_cursor;
    pages += 1;
  } while (cursor);

  return { items: all, pages };
}

• Concurrency. You can drain many accounts in parallel, but a single API key allows at most 50 concurrent requests. Exceed it and you get a 429 CONCURRENCY_LIMIT (0 credits charged). Cap your worker pool well under 50; within one account, pages are inherently sequential (each needs the previous next_cursor).
• Credit budget. Each page is a normal billed request: one page of a 1-credit endpoint costs 1 credit, so draining a 50-page feed costs about 50 credits. Multiply pages by the endpoint's per-call cost (see Endpoint pricing) before you loop, and cap pages if you only need the most recent N results. Cache hits on re-runs cost 0 credits.
• Bounded depth. Page- and offset-style endpoints (Naver, LinkedIn, the app stores) have a finite maximum depth; has_more flips to false when you reach it. Cursor-style feeds end when the upstream runs out of items.

Sending the wrong param is a 400, not a silent bug

You never need a platform's native pagination param name: always send cursor. If you do send a foreign cursor param (say you copy max_id off an Instagram response onto a TikTok endpoint), you get a 400 INVALID_REQUEST with a did_you_mean: "cursor" hint and zero credits charged, instead of silently receiving page 1 again.

Stale or malformed cursors

If you pass a cursor the upstream rejects (expired, hand-edited, copied across endpoints), you'll get a 502 UPSTREAM_ERROR or 400 INVALID_PARAMETER envelope. Credits are refunded on upstream errors (see Credits → When are credits refunded?). Start over with no cursor.

Page size

Most endpoints let the upstream decide: you'll typically get 10 to 30 items per page (pagination.page_size tells you exactly). Where a page-size control exists (see the Page size column below), pass a universal limit and the API maps it to the upstream's native name (Naver display, GitHub per_page, and so on), capped at that endpoint's maximum. A — in that column means no page-size control: to get fewer items, slice client-side; to get more, paginate another page.

Every paginatable endpoint

123 endpoints paginate today (83 cursor-based, 29 page-based, 11 offset-based). Every one of them takes the universal cursor input and returns the same pagination block (next_cursor / has_more / page_size): no per-platform special cases. The Style and Page size columns are informational only; you always send cursor.

Apple App Store

Content Analysis

Facebook

GitHub

Google

Google Play

Hacker News

Instagram

Kwai

LinkedIn

Naver

Pinterest

Prism

Reddit

Rumble

Spotify

TikTok

TikTok Shop

Truth Social

Twitter/X

Web Scraping

YouTube

See also

• Response schema → List endpoints — the envelope shape in one sentence.
• Quickstart — your first request in under a minute.
• Endpoint pricing — every endpoint and what each call costs.
• Error handling — refund rules for stale-cursor errors, and the concurrency limit.