# Track topics and hashtags (/docs/prism/discover-topics)

Search several topics and their hashtags on TikTok, Instagram, YouTube and X every few hours, judge every post against each topic and get only the posts no earlier run returned

Polling each hashtag on each platform returns the same posts again on every poll, posts that use the hashtag for something else, and a different format per platform. One scheduled discover request replaces those polls. Each topic is a group with its keywords and hashtags, every post is judged against the topic, repeats are removed across platforms, and each run returns only the posts that no earlier run returned.

## The request

```json title="Request body"
{
  "groups": [
    {
      "id": "georgia",
      "keywords": ["Georgia General Assembly", "Georgia election 2026", "Georgia cost of living", "#gapol", "#georgiapolitics"],
      "context": "Posts about politics in the US state of Georgia: the General Assembly, the 2026 election, abortion law and the cost of living in Georgia"
    },
    {
      "id": "groceries",
      "keywords": ["grocery prices", "shrinkflation", "SNAP benefits", "#groceryprices", "#foodstamps"],
      "context": "Posts about rising grocery prices, shrinkflation, and SNAP or food stamp benefits in the United States"
    },
    {
      "id": "maga",
      "keywords": ["MAGA", "Trump administration", "America First", "#maga", "#americafirst"],
      "context": "Posts about President Trump, his administration or the MAGA movement, from supporters or critics"
    }
  ],
  "platforms": ["tiktok", "instagram", "youtube", "twitter"],
  "period": "2d",
  "label": ["stance"],
  "max_credits": 1200,
  "schedule": { "every": "6h", "max_credits_per_run": 1200, "max_credits_per_month": 30000, "name": "Topics" }
}
```

```bash title="cURL"
# request.json is the body above: "groups": [...], "schedule": {...}, "max_credits": 1200
curl -X POST "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: topics-6h" \
  --data @request.json
```

Send it once with `"dry_run": true` and without `schedule` to see every search and the estimate, free. The real request answers with `data.schedule` (its `schedule_id`, `next_fire_at` and `links.runs_url`) and starts the first run at once.

## What each part does

- Write a hashtag as a `#tag` keyword inside the group. On Instagram, TikTok and YouTube it runs both the platform's hashtag search and its keyword search, and on X it searches the hashtag as text. A group has no `hashtags` field.
- `context` is what a kept post must be about. A post that uses `#maga` or `#gapol` for something else is dropped as `off_topic`, so one group's hashtag noise never reaches your data.
- `period` counts calendar days in `tz` (UTC by default), so `1d` is today only and a run just after midnight would see minutes of posts. `2d` covers the gap, and the watermark below stops repeats.
- `label: ["stance"]` labels each kept post `positive`, `negative`, `neutral` or `no_opinion` toward its group's keywords, and `null` when the judge is unsure. `summary.by_group.<id>.stance` counts them per topic.
- `schedule.every` takes `1h`, `6h`, `12h`, `1d`, `7d` or `30d`. Sending the same request again from the same key returns the schedule that already exists (`deduplicated: true`) and starts nothing.

## Only new posts on every run

Each scheduled run starts from the previous run's watermark, so a post that an earlier run returned is dropped before it is judged and never billed again. `summary.dropped.by_filter["filter:watermark"]` counts them. A scheduled request also continues from a one-off run of the same request, so a test run you made first is not delivered twice.

Measured on 20/10/2026 with the three groups above and `"period": "1d"`:

| Run | Posts returned by the searches | Kept | Dropped as already delivered | Credits |
| --- | --- | --- | --- | --- |
| One-off run, 16:21 UTC | 2,232 | 440 | 0 | 209 |
| First scheduled run, 16:31 UTC | 2,175 | 124 | 1,006 | 139 |
| Second scheduled run, 17:32 UTC | 2,274 | 162 | 961 | 151 |

On the one-off run the searches returned 2,232 posts. Of those, 299 were exact repeats across searches, 977 were outside the window and 497 were off-topic, so 440 were kept. The one-off run took about 6 minutes.

## What a run costs

The searches cost the same on every run, and judging is paid only for posts not judged before. Per platform, on the one-off run:

| Platform | Search credits | Kept posts | Credits per kept post |
| --- | --- | --- | --- |
| YouTube | 21 | 213 | 0.10 |
| X | 14 | 127 | 0.11 |
| TikTok | 20 | 64 | 0.31 |
| Instagram | 59 | 36 | 1.64 |

Instagram's hashtag search costs 5 credits a page and its reels search returned few posts inside one day, so Instagram cost about 60 of each run's 115 search credits and kept 1 of the 124 new posts on the first scheduled run and 13 of the 162 on the second. Keep Instagram for the topics where you need it, or put it in a group of its own on a slower schedule. `summary.by_platform.<platform>.credits_per_kept` shows this for your own topics after every run.

`max_credits_per_run` must cover the dry run's worst case (`estimate.credits.max`, 1,143 for this request), so that no run stops halfway; a lower cap is a free 400 whose `suggested_value` is the number to send. The runs above spent 139 to 209. `max_credits_per_month` is the budget. A run that could take the month past it is skipped and the schedule pauses until the next month.

## Reading the posts

- `GET {links.runs_url}` lists the schedule's runs, newest first, each with `status`, `credits.spent` and `status_url`.
- `GET {status_url}/results.jsonl` streams a settled run's kept posts, one per line, free. Each post carries `group`, `platform`, `url`, `published_at`, `author.key`, `engagement`, `matched_keywords`, `hashtags`, `judgments.relevance.p` and `judgments.stance.label`. Store posts by `id`, which stays the same across runs.
- Split the export on `\n` only. Bios and captions can contain the characters U+2028 and U+0085, and a split that treats them as line ends (Python's `splitlines()`) cuts a JSON line in half.
- Each run that keeps posts sends `discover.new_rows` with the count and the first 100 new ids. Register a URL once with `POST /v1/webhook-endpoints` and `"types": ["discover.*"]` ([Event webhooks](/docs/webhook-endpoints.md)), or read the same events with `GET /v1/events?types=discover.new_rows&cursor=oldest` ([Events](/docs/events.md)).
- An account runs two background runs at once. A scheduled run that starts while two others are running waits as `queued` and starts by itself.

## Hashtags you are not searching yet

`view=items` on a week of a topic's posts lists the hashtags those posts use, with their count per platform and their growth over the week:

```bash title="cURL"
curl -G "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: YOUR_API_KEY" \
  --data-urlencode "keywords=Georgia General Assembly,Georgia election 2026,Georgia cost of living,#gapol,#georgiapolitics" \
  --data-urlencode "context=Posts about politics in the US state of Georgia: the General Assembly, the 2026 election, abortion law and the cost of living in Georgia" \
  --data-urlencode "view=items" \
  --data-urlencode "platforms=tiktok,instagram,youtube,twitter" \
  --data-urlencode "period=7d" \
  --data-urlencode "max_credits=150"
```

On 20/10/2026 it returned 268 items for 45 credits. The hashtags the group did not search included `#JonOssoff` (19 posts on four platforms, growth 5.33), `#election2026` (14), `#KeishaLanceBottoms` (8, growth 7), `#earlyvoting` (7) and `#GeorgiaSenate` (3). Add the ones that belong to the topic as `#tag` keywords; sending the schedule's request with new keywords creates a new schedule, so delete the old one with `DELETE /v1/prism/discover/schedules/{schedule_id}`.

## Manage the schedule

| Call | What it does |
| --- | --- |
| `GET /v1/prism/discover/schedules/{schedule_id}` | The schedule, its last 10 runs and this month's spend. |
| `POST /v1/prism/discover/schedules/{schedule_id}/pause` | Stops new runs until you resume it. |
| `POST /v1/prism/discover/schedules/{schedule_id}/resume` | Starts it again, optionally with new caps. |
| `DELETE /v1/prism/discover/schedules/{schedule_id}` | Deletes it; its runs keep their posts until their retention ends. |

All four are free. See [Schedules](/docs/prism/discover.md#schedules) for skipped slots and pauses.

## Next steps

- [Discover](/docs/prism/discover.md): Every field, view and price of discover.
- [Find accounts](/docs/prism/discover-accounts.md): The accounts behind a list of keywords, with their posts and comments.
- [Event webhooks](/docs/webhook-endpoints.md): Register one URL for every run