SocialCrawl
Cookbooks

Get a whole account, safely

Walk every video, comment or follower an account has with a cursor loop that retries a failed page, stops on a cap and never pays twice for a page

Send one handle, get back every video the account has, one page at a time, and stop on a cap you set. The same loop works on any list that returns a cursor, so the example also reads the comments of one video. You never skip a page, and you never pay twice for the one you already hold.

Measured on 22/11/2026: a walk of 3 pages of one news publisher's TikTok videos read 30 videos for 3 credits. Reading the second page again straight after cost 0 credits.

Start in five minutes

Set your key once. Every block on this page reads it.

Terminal
export SOCIALCRAWL_API_KEY=sc_your_key_here

Paste this prompt into your coding agent:

Prompt
Read https://www.socialcrawl.dev/docs/recipes/get-a-whole-account.md and follow it to list the TikTok videos of the handle washingtonpost. Use the key in SOCIALCRAWL_API_KEY. Price one page with dry_run=true first and show me the estimate. Then walk the pages with max_items=30 and max_credits=5, and never raise either number without asking me.

Or price the first page yourself. A dry run quotes it for 0 credits:

cURL
curl -G "https://www.socialcrawl.dev/v1/tiktok/profile/videos" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  --data-urlencode "handle=washingtonpost" \
  --data-urlencode "dry_run=true"

On 22/11/2026 it answered valid: true and a hold of 1 credit for one page.

How it works

  • Each answer holds one page of rows and a next_cursor that points at the next page.
  • You send that cursor back as cursor, exactly as you got it, and the walk moves on.
  • The walk ends when has_more is false. You never use data.total as the stop signal, because a list can leave it empty.
  • A page that fails with 429 or 503 is sent again with the same cursor, after the wait the answer asks for.
  • A cap such as max_items or max_credits keeps a long account from running up the bill.

What you need

CallWhat it givesCredits
GET /v1/tiktok/profile/videosOne page of an account's videos and the cursor for the nextBilled per page, see the roster
GET /v1/tiktok/post/commentsOne page of a video's comments, the same cursor ruleBilled per page, see the roster
GET /v1/utility/estimate or dry_run=trueThe price of a request or a capped walk0

Steps

Step 1: read one page and keep the cursor

cURL
curl -G "https://www.socialcrawl.dev/v1/tiktok/profile/videos" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  --data-urlencode "handle=washingtonpost"
Python
import json, os, urllib.parse, urllib.request

url = "https://www.socialcrawl.dev/v1/tiktok/profile/videos?" + urllib.parse.urlencode({"handle": "washingtonpost"})
req = urllib.request.Request(url, headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]})
page = json.load(urllib.request.urlopen(req, timeout=120))
print(len(page["data"]["items"]), page["pagination"]["has_more"], page["pagination"]["next_cursor"][:24])
  • pagination.next_cursor is an opaque token. Do not decode it, trim it or build one.
  • On 22/11/2026 a page held 10 videos, has_more was true and the page cost 1 credit.
Response (trimmed)
{
  "data": { "items": [ { "post": { "id": "7681829627085507870" } } ] },
  "credits_used": 1,
  "pagination": { "next_cursor": "sc.eyJ2Ijoy...", "has_more": true, "page_size": 10 }
}

Step 2: send the cursor back until has_more is false

Add cursor=<next_cursor> to the same request and keep every other parameter the same. Stop when pagination.has_more is false.

  • A page can be empty and still bill. On 22/11/2026 a walk bounded by a date ended on an empty page with has_more: false, and that page cost 1 credit.
  • data.total is not the stop signal. On the video list it was null, and on the comment list it was a number the walk never needed.
  • Pages of one account run one after another, because each needs the cursor of the one before.

Step 3: retry a failed page with the same cursor

On 429 or 503, wait for the Retry-After header and send the same URL again. Never advance past a page you did not read, and never restart from page 1. A failed page costs nothing, so the retry is free.

The program at the bottom of the page does this for you.

Step 4: cap the walk

Send max_items and max_credits and the API walks the cursor for you in one call. A page starts only while its price still fits under max_credits.

cURL
curl -G "https://www.socialcrawl.dev/v1/tiktok/profile/videos" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  --data-urlencode "handle=cnn" \
  --data-urlencode "max_items=20" \
  --data-urlencode "max_credits=2"
Python
import json, os, urllib.parse, urllib.request

query = {"handle": "cnn", "max_items": "20", "max_credits": "2"}
url = "https://www.socialcrawl.dev/v1/tiktok/profile/videos?" + urllib.parse.urlencode(query)
req = urllib.request.Request(url, headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]})
answer = json.load(urllib.request.urlopen(req, timeout=120))
print(len(answer["data"]["items"]), answer["data"]["walk"]["stopped"], answer["credits_used"])
  • data.walk.stopped says why the walk ended: end, max_items, max_credits, max_pages or time_budget.
  • pagination.next_cursor still resumes the walk right after the last row you received.
  • GET /v1/utility/estimate quotes the capped walk for free. On 22/11/2026 it said a request with max_items=30 and max_credits=5 charges up to 5 credits.
  • On 22/11/2026 the capped walk above returned 20 videos in 2 pages for 2 credits and stopped on max_items.

Step 5: read a second list the same way

Comments use the same cursor rule. Pass the video link from step 1.

cURL
curl -G "https://www.socialcrawl.dev/v1/tiktok/post/comments" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  --data-urlencode "url=https://www.tiktok.com/@washingtonpost/video/7681829627085507870" \
  --data-urlencode "max_items=30" \
  --data-urlencode "max_credits=5"

On 22/11/2026 one page held 50 comments for 1 credit, and the walk stopped on max_items with has_more still true.

Step 6: hand a long walk to a background job

For many accounts, or a walk that outlasts one request, send the same call as an item of POST /v1/prism/jobs. The job resumes from its own cursor and keeps your limits. The Async page has the body and how to read the result.

Store only what is new

Key each video on post.id. Save the last next_cursor you read next to your rows, so a walk that stopped on a cap carries on from there instead of from page 1. A row you already hold is not new, even when a later walk meets it again. To read only what was posted since your last run, use Poll an account daily.

What it costs

Each page costs what one call to that endpoint costs, and a capped walk bills each page it reads. The roster lists the price of every endpoint.

What the runs measured on 22/11/2026:

RunRowsCredits
The dry run of one page00
A walk of 3 pages by cursor30 videos3
The second page read again straight after10 videos0 (served from cache)
A capped walk, max_items=20 and max_credits=220 videos2
One page of comments, max_items=3050 comments1
The full program below, capped at 3 pages30 videos3

Tips, and what not to use

  • Do not walk with a guessed page number. The cursor is the only safe way to the next page.
  • Do not restart from page 1 after a failure. Send the same cursor again.
  • Do not read data.total as the stop signal. Read has_more.
  • A handle that returns no rows ends on the first page with stopped: end and costs 0 credits.
  • A page emptied by since or until is still billed. To pay only when something is new, use Listening or Monitors.
  • Within one account the pages run in order. To read many accounts at once, run them in parallel and keep the number of requests under your rate limits.

Full program and agent brief

Next steps