Poll an account daily without paying twice
Fetch only the posts an account published since your last run, bounded by a date on the first run and by the newest post you hold on every run after
Send a handle, and get back only the posts published since your last run. The first run stops at a date. Every run after it stops at the newest post you already hold, so a quiet day reads one page and ends there.
Measured on 22/11/2026: a first run bounded by date read 36 videos of one news publisher in 4 pages for 4 credits. The next run, stopped at the newest video it held, read 1 page for 1 credit and found no new video.
Start in five minutes
Set your key once. Every block on this page reads it.
export SOCIALCRAWL_API_KEY=sc_your_key_herePaste this prompt into your coding agent:
Read https://www.socialcrawl.dev/docs/recipes/poll-an-account-daily.md and follow it to poll the TikTok handle washingtonpost for new videos. Use the key in SOCIALCRAWL_API_KEY. Price one page with dry_run=true first and show me the estimate. On the first run use since=2026-11-20. On later runs use stop_at_id with the newest video you hold, and never raise the page cap without asking me.Or price the first page yourself. A dry run quotes it for 0 credits:
curl -G "https://www.socialcrawl.dev/v1/tiktok/profile/videos" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
--data-urlencode "handle=washingtonpost" \
--data-urlencode "since=2026-11-20" \
--data-urlencode "dry_run=true"How it works
- The first run has no post to stop at, so it sends
sincewith a date and walks back to it. - You keep the newest post you received and save its id.
- Every later run sends that id as
stop_at_id. The page stops just before that post. - The same boundary goes on every page of one run, and the run ends when
has_moreisfalse. - Pinned posts sit out of date order, so they never count as the newest post you hold.
What you need
| Call | What it gives | Credits |
|---|---|---|
GET /v1/tiktok/profile/videos with since or stop_at_id | The posts newer than your boundary, one page at a time | Billed per page, see the roster |
GET /v1/utility/estimate or dry_run=true | The price of the request | 0 |
The same two parameters work on the Facebook, Instagram, Threads, X and YouTube post lists. The Pagination page names each endpoint.
Steps
Step 1: run once, bounded by a date
curl -G "https://www.socialcrawl.dev/v1/tiktok/profile/videos" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
--data-urlencode "handle=washingtonpost" \
--data-urlencode "since=2026-11-15"import json, os, urllib.parse, urllib.request
query = {"handle": "washingtonpost", "since": "2026-11-15"}
url = "https://www.socialcrawl.dev/v1/tiktok/profile/videos?" + urllib.parse.urlencode(query)
req = urllib.request.Request(url, headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]})
page = json.load(urllib.request.urlopen(req, timeout=120))
print(len(page["data"]["items"]), page["pagination"]["has_more"], page["pagination"].get("stopped_at"))sincetakes a date such as2026-11-15or an ISO 8601 timestamp. A date is read as midnight UTC.- While
has_moreistrue, sendnext_cursorback ascursorwith the samesince. - The page that reaches an older post ends the walk with
pagination.stopped_atset tosince. - On 22/11/2026 the four pages read 7, 10, 10 and 9 videos, and the last ended with
stopped_at: sinceandhas_more: false.
Step 2: save the newest post you hold
Keep the id of the first row that is not pinned. A profile opens with the posts its owner pinned, and those can be years old.
{
"data": {
"items": [
{ "post": { "id": "7710747895028337934", "published_at": "2026-11-21T22:00:06.000Z", "flags": { "pinned": false } } }
]
},
"credits_used": 1,
"pagination": { "next_cursor": "sc.eyJ2Ijoy...", "has_more": true, "page_size": 7 }
}Step 3: poll with stop_at_id
curl -G "https://www.socialcrawl.dev/v1/tiktok/profile/videos" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
--data-urlencode "handle=washingtonpost" \
--data-urlencode "stop_at_id=7710747895028337934"import json, os, urllib.parse, urllib.request
query = {"handle": "washingtonpost", "stop_at_id": "7710747895028337934"}
url = "https://www.socialcrawl.dev/v1/tiktok/profile/videos?" + urllib.parse.urlencode(query)
req = urllib.request.Request(url, headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]})
page = json.load(urllib.request.urlopen(req, timeout=120))
print(len(page["data"]["items"]), page["pagination"].get("stopped_at"))- The page stops just before the post you named. That post and every older one are left off.
pagination.stopped_atisknown_idandnext_cursorisnull, so the run is over.- If the post is not on the page, the page comes back whole with its cursor. Keep walking with the same
stop_at_id. stop_at_idtakes a post id or a post link. One longer than 512 characters is a free400.- On 22/11/2026 this run came back with 3 pinned videos you already hold and no new video,
stopped_atwasknown_id, and it cost 1 credit.
Step 4: schedule it
Run the program below once a day from cron or any scheduler. It keeps the newest post id and the ids it has seen in a small file. A bad date is a 400 that costs nothing: on 22/11/2026 a since that was not a date answered 400 INVALID_REQUEST for 0 credits.
Store only what is new
Send the same stop_at_id or since on every page of the walk, then keep the newest post you received for the next run.
- Key each post on
post.id. - Skip a row whose
flags.pinnedistrue. It is not a new post, and it can come back on every run. - Save the newest id only after the whole run finishes. A run that fails halfway then starts again from the old boundary.
- On YouTube, send
stop_at_idrather than a date. YouTube list dates come from labels such as "4 months ago", so a date stop there is an estimate.
What it costs
Each page costs what a page of that endpoint costs. A poll that finds your newest post on page 1 pays for page 1 and stops there. The roster lists the price of every endpoint.
What the runs measured on 22/11/2026:
| Run | Pages | Credits |
|---|---|---|
| The dry run | 0 | 0 |
First run, since=2026-11-15, 36 videos | 4 | 4 |
Second run, stop_at_id, no new video | 1 | 1 |
A since that is not a date | 0 | 0 |
The program below, first run with since=2026-11-20, 7 videos, served from cache | 1 | 0 |
Tips, and what not to use
- A page emptied by
sinceis still billed. If a daily check costs you more than the new posts are worth, Listening and Monitors bill when something is new. - Do not keep sending a date after the first run. The newest post you hold is a tighter boundary, so each poll reads fewer pages.
- Do not poll more often than you need. Each poll that finds nothing still reads one page.
- Start a Monitor when you want no cron job at all. A Monitor runs the check for you on a schedule you choose.
Full program and agent brief
Next steps
Get a whole account
Walk every video, comment or follower an account has with a cursor loop that retries a failed page, stops on a cap and never pays twice for a page
Every review, then new ones
Pull a business's review history once, then collect only the reviews written since, with a bulk job that refreshes itself from the earlier job
