Pagination

Every list endpoint (PostList, CommentList, SearchResult) returns the same envelope shape:

{
  "data": {
    "items": [ /* ... */ ],
    "next_cursor": "eyJwYWdlIjoyfQ==",
    "total": 1247
  }
}

You read data.next_cursor and pass it back as a query parameter on the next request. When next_cursor is missing or null, you've reached the end.

The catch: the name of the query parameter you pass it back as is different per platform. TikTok wants max_cursor, Instagram wants cursor or next_max_id, Reddit wants after. We normalise the response field name so your code stays simple, but the input param name still has to match the upstream. The table at the bottom of this page lists the exact param for every endpoint.

There's one exception: Naver is offset-based (start + display), not cursor-based. You increment start yourself.

How to iterate

The general loop, in pseudocode:

let cursor: string | undefined;
while (true) {
  const url = new URL("https://api.socialcrawl.dev/v1/tiktok/profile/videos");
  url.searchParams.set("handle", "mrbeast");
  if (cursor) url.searchParams.set("max_cursor", cursor); // <-- platform-specific param name

  const res = await fetch(url, { headers: { "x-api-key": process.env.SOCIALCRAWL_KEY! } });
  const json = await res.json();

  for (const item of json.data.items) {
    // ... handle item
  }

  if (!json.data.next_cursor) break;
  cursor = json.data.next_cursor; // <-- pass back verbatim
}

Two rules:

1. Pass next_cursor back as the EXACT string you received. Don't decode, trim, or re-encode it — these are upstream tokens.
2. Stop when next_cursor is missing. The field is omitted from the envelope when there are no more pages. Don't check for empty string.

Naver (offset-based)

Naver doesn't return a cursor. Increment start by display each iteration, and stop when items comes back shorter than display or when start + display > 1000 (Naver's hard cap):

const PAGE_SIZE = 100;          // Naver `display` — cap 100
let start = 1;                  // 1-indexed offset — cap 1000
while (start <= 1000) {
  const url = new URL("https://api.socialcrawl.dev/v1/naver/search/news");
  url.searchParams.set("query", "삼성전자");
  url.searchParams.set("display", String(PAGE_SIZE));
  url.searchParams.set("start", String(start));

  const res = await fetch(url, { headers: { "x-api-key": process.env.SOCIALCRAWL_KEY! } });
  const json = await res.json();
  for (const item of json.data.items) { /* ... */ }

  if (json.data.items.length < PAGE_SIZE) break;
  start += PAGE_SIZE;
}

What about page size?

For every platform except Naver, upstream decides. You'll typically get 10–30 items per page. There is no limit or page_size query parameter — passing one is silently ignored. If you need fewer items, slice the array client-side; if you need more, iterate more pages.

Stale or malformed cursors

If you pass a cursor that the upstream rejects (expired, hand-edited, copied across endpoints), you'll get a 502 UPSTREAM_ERROR or 400 INVALID_PARAMETER envelope. Credits are refunded on upstream errors — see Credits → When are credits refunded?. Start over with no cursor.

What total means

data.total is included when the upstream returns a result count. It's the upstream's count, not ours, and can be:

• Exact (Reddit, Naver) — the precise number of items in the result set.
• Estimated (Instagram, TikTok search) — an upstream estimate that may not match the number you can actually paginate to.
• Missing — the upstream didn't tell us, so we don't either.

Don't use total to decide when to stop iterating. Use next_cursor for that — it's the only signal that's always correct.

Every paginatable endpoint

42 endpoints paginate today (31 cursor-based, 11 offset-based), grouped by platform.

Facebook

Google

Instagram

LinkedIn

Naver

Pinterest

Reddit

TikTok

Truth Social

YouTube

See also

• Response schema → List endpoints — the envelope shape in one sentence.
• Endpoint pricing — every endpoint and what each call costs.
• Error handling — refund rules for stale-cursor errors.