Track topics and hashtags

Polling each hashtag on each platform returns the same posts again on every poll, posts that use the hashtag for something else, and a different format per platform. One scheduled discover request replaces those polls. Each topic is a group with its keywords and hashtags, every post is judged against the topic, repeats are removed across platforms, and each run returns only the posts that no earlier run returned.

The request

{
  "groups": [
    {
      "id": "georgia",
      "keywords": ["Georgia General Assembly", "Georgia election 2026", "Georgia cost of living", "#gapol", "#georgiapolitics"],
      "context": "Posts about politics in the US state of Georgia: the General Assembly, the 2026 election, abortion law and the cost of living in Georgia"
    },
    {
      "id": "groceries",
      "keywords": ["grocery prices", "shrinkflation", "SNAP benefits", "#groceryprices", "#foodstamps"],
      "context": "Posts about rising grocery prices, shrinkflation, and SNAP or food stamp benefits in the United States"
    },
    {
      "id": "maga",
      "keywords": ["MAGA", "Trump administration", "America First", "#maga", "#americafirst"],
      "context": "Posts about President Trump, his administration or the MAGA movement, from supporters or critics"
    }
  ],
  "platforms": ["tiktok", "instagram", "youtube", "twitter"],
  "period": "2d",
  "label": ["stance"],
  "max_credits": 1200,
  "schedule": { "every": "6h", "max_credits_per_run": 1200, "max_credits_per_month": 30000, "name": "Topics" }
}

# request.json is the body above: "groups": [...], "schedule": {...}, "max_credits": 1200
curl -X POST "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: topics-6h" \
  --data @request.json

Send it once with "dry_run": true and without schedule to see every search and the estimate, free. The real request answers with data.schedule (its schedule_id, next_fire_at and links.runs_url) and starts the first run at once.

What each part does

• Write a hashtag as a #tag keyword inside the group. On Instagram, TikTok and YouTube it runs both the platform's hashtag search and its keyword search, and on X it searches the hashtag as text. A group has no hashtags field.
• context is what a kept post must be about. A post that uses #maga or #gapol for something else is dropped as off_topic, so one group's hashtag noise never reaches your data.
• period counts calendar days in tz (UTC by default), so 1d is today only and a run just after midnight would see minutes of posts. 2d covers the gap, and the watermark below stops repeats.
• label: ["stance"] labels each kept post positive, negative, neutral or no_opinion toward its group's keywords, and null when the judge is unsure. summary.by_group.<id>.stance counts them per topic.
• schedule.every takes 1h, 6h, 12h, 1d, 7d or 30d. Sending the same request again from the same key returns the schedule that already exists (deduplicated: true) and starts nothing.

Only new posts on every run

Each scheduled run starts from the previous run's watermark, so a post that an earlier run returned is dropped before it is judged and never billed again. summary.dropped.by_filter["filter:watermark"] counts them. A scheduled request also continues from a one-off run of the same request, so a test run you made first is not delivered twice.

Measured on 20/10/2026 with the three groups above and "period": "1d":

On the one-off run the searches returned 2,232 posts. Of those, 299 were exact repeats across searches, 977 were outside the window and 497 were off-topic, so 440 were kept. The one-off run took about 6 minutes.

What a run costs

The searches cost the same on every run, and judging is paid only for posts not judged before. Per platform, on the one-off run:

Instagram's hashtag search costs 5 credits a page and its reels search returned few posts inside one day, so Instagram cost about 60 of each run's 115 search credits and kept 1 of the 124 new posts on the first scheduled run and 13 of the 162 on the second. Keep Instagram for the topics where you need it, or put it in a group of its own on a slower schedule. summary.by_platform.<platform>.credits_per_kept shows this for your own topics after every run.

max_credits_per_run must cover the dry run's worst case (estimate.credits.max, 1,143 for this request), so that no run stops halfway; a lower cap is a free 400 whose suggested_value is the number to send. The runs above spent 139 to 209. max_credits_per_month is the budget. A run that could take the month past it is skipped and the schedule pauses until the next month.

Reading the posts

• GET {links.runs_url} lists the schedule's runs, newest first, each with status, credits.spent and status_url.
• GET {status_url}/results.jsonl streams a settled run's kept posts, one per line, free. Each post carries group, platform, url, published_at, author.key, engagement, matched_keywords, hashtags, judgments.relevance.p and judgments.stance.label. Store posts by id, which stays the same across runs.
• Split the export on \n only. Bios and captions can contain the characters U+2028 and U+0085, and a split that treats them as line ends (Python's splitlines()) cuts a JSON line in half.
• Each run that keeps posts sends discover.new_rows with the count and the first 100 new ids. Register a URL once with POST /v1/webhook-endpoints and "types": ["discover.*"] (Event webhooks), or read the same events with GET /v1/events?types=discover.new_rows&cursor=oldest (Events).
• An account runs two background runs at once. A scheduled run that starts while two others are running waits as queued and starts by itself.

Hashtags you are not searching yet

view=items on a week of a topic's posts lists the hashtags those posts use, with their count per platform and their growth over the week:

curl -G "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: YOUR_API_KEY" \
  --data-urlencode "keywords=Georgia General Assembly,Georgia election 2026,Georgia cost of living,#gapol,#georgiapolitics" \
  --data-urlencode "context=Posts about politics in the US state of Georgia: the General Assembly, the 2026 election, abortion law and the cost of living in Georgia" \
  --data-urlencode "view=items" \
  --data-urlencode "platforms=tiktok,instagram,youtube,twitter" \
  --data-urlencode "period=7d" \
  --data-urlencode "max_credits=150"

On 20/10/2026 it returned 268 items for 45 credits. The hashtags the group did not search included #JonOssoff (19 posts on four platforms, growth 5.33), #election2026 (14), #KeishaLanceBottoms (8, growth 7), #earlyvoting (7) and #GeorgiaSenate (3). Add the ones that belong to the topic as #tag keywords; sending the schedule's request with new keywords creates a new schedule, so delete the old one with DELETE /v1/prism/discover/schedules/{schedule_id}.

Manage the schedule

All four are free. See Schedules for skipped slots and pauses.

Next steps