SocialCrawl
EnginesDiscover

Groups, schedules and lists

Run many topics in one request, repeat it on a schedule, and keep lists of authors and posts you already hold

A POST /v1/prism/discover body can hold several groups, each with its own keywords, question and platforms. The same body can run again on a schedule, and every run continues from where the last one stopped. Every other request field is on Discover request fields.

Groups

Each group in groups is a search of its own: its keywords, its context, its platforms, its country and its labels. Rows keep the group they came from in group_tags, and a row found by two groups is kept once, with the second group in also_in_groups. The groups row of the request fields lists what a group takes. Market research and the monthly brand report show a full POST body with groups, and dry_run: true in the body prices it for free.

Lists you hold

A long author list

POST /v1/prism/discover/inputs with { "kind": "authors", "authors": ["nasa", "@natgeo", "twitter:spacex"] } stores 1 to 10,000 handles (each at most 100 characters) and answers 201 with author_list_id, which any run takes in place of authors. A bare handle matches on any platform and platform:handle on that platform only. A list is free, belongs to your account, and expires after 30 days; an account holds at most 500 live lists at once, and one more is a free 400 input_limit that names the routes below. authors itself takes at most 100 handles on the GET and 1,000 on the POST.

With authors_match=walk, discover reads each listed author's own posts in the window (Instagram, TikTok, YouTube, X, Facebook, Threads, LinkedIn and Reddit) and keeps the ones about your keywords or context, judged like any search row. Name each author with its platform (x:handle, instagram:handle) or its profile URL; a LinkedIn author needs its profile or company URL. A bare handle is read on every platform of the request that has a feed. An author on a platform the request does not search is listed in _warnings. Each author costs one page of its platform's feed, or pages_per_platform pages, and dry_run shows the total; a list larger than the call's budget continues in the background. On the platforms it reads, walk replaces the keyword search. A platform of the request with no account feed, such as news in the default preset, searches nothing and costs nothing when every listed author names another platform, and _warnings names it. When the list holds a bare handle, or an author named on that platform, the platform still searches your keywords and keeps only rows by the listed authors. A request left with nothing to search is a free 400 naming authors.

Posts you already hold

exclude drops the rows you name, but in a URL it fits only a few hundred ids. To leave out every post you already hold, store them once: POST /v1/prism/discover/inputs with { "kind": "exclude", "exclude": ["rd_0123456789ab", "https://www.reddit.com/r/PersonalFinanceCanada/comments/1abcde/", "reddit:handle"] } stores 1 to 50,000 entries (each at most 200 characters, in a body under 4 MB) and answers 201 with exclude_list_id. Then send exclude_list_id on any GET or POST of /v1/prism/discover, from any API key of your account. Its entries join exclude: an evidence id drops that row, a post URL drops the row with that post, however the URL is written (www., a share link's tracking parameters), and an account key drops every row by that account. Dropped rows are never judged or billed; with include=dropped they carry drop_reason filter:exclude. Evidence ids are the shortest entries, so 50,000 of them fit in one body. To add posts as you collect them, append to the same list instead of storing a new one: POST /v1/prism/discover/inputs/{exclude_list_id}/entries with { "exclude": [...] } adds the entries it does not already hold and keeps the id, so a run or schedule that names it reads the new entries on its next run. It is free and answers 200 with item_count, added and the new expires_at; past 50,000 entries it is a free 400 exclude_list_full that names the cap and the list's current count, and nothing is added. An exclude list counts toward the same 500 lists as author lists, and expires 30 days after it was stored, or 31 days after the last run that read it, so a list a schedule uses stays live; an unknown or expired id, or one of another account, is a free 400 invalid_param naming exclude_list_id.

Managing stored lists

Every route below is free and answers only to your account, from any of its API keys. An id that is unknown, expired or another account's is the same free 404 input_not_found.

  • GET /v1/prism/discover/inputs lists your live lists, newest first, each with id, kind, item_count, created_at and expires_at, plus live and limit (500). It never returns the entries.
  • POST /v1/prism/discover/inputs/{id}/entries with { "exclude": [...] } adds entries to an exclude list, as above. An author list takes no entries; store a new one.
  • DELETE /v1/prism/discover/inputs/{id} deletes a list of either kind and frees its slot. A run or schedule that names it afterwards gets the same free 400 as an unknown id.

Only new rows: since_watermark

Every run answers data.next.watermark, an opaque token bound to your account and to the request without its resolved dates, so period=7d keeps working day after day. A GET answers it in the call; a background run answers it on its run route once it settles. Send it back as since_watermark on the same request and you get only rows that earlier runs of that request did not deliver. Rows already delivered are dropped before judging, so they are never billed again, while the platform searches are paid as usual. summary.new_since_last_run counts the kept rows the previous run had not delivered, and is null on the first run.

A request with since_watermark always fetches: it never returns a completed run, and two identical ones sent while the first is running share that run. A token from another account or another request is a free 400 watermark_mismatch, and when the watermark cannot be read the call answers a free, retryable 503 (store_unavailable, or watermark_unavailable when it cannot be checked at all) rather than running without it. A platform's watermark moves only when all its searches in the run finished, never past the run's start; rows a paused, canceled or revoked-key run already made readable count as delivered. Kept rows still waiting for the judge (on a partial run with judging left, or a run paused at max_credits) do not count, and their platform's watermark does not move, so the next run returns them once they are judged. Right after a background run settles, data.next.watermark can read null for a moment until the watermark is written; reading the run again a second later shows it. Ids delivered within 3 days of the watermark are remembered, so a late-indexed post still comes through once.

Schedules

Add schedule to a POST and the request runs again on its own:

{
  "brands": ["acme"],
  "platforms": ["reddit", "twitter", "news"],
  "period": "7d",
  "schedule": { "every": "1d", "max_credits_per_run": 50, "max_credits_per_month": 1200, "name": "Acme daily" }
}
  • The first run starts at once (or at start_at, at most 90 days ahead), and the answer carries data.schedule beside data.run.
  • Price a schedule before you create it: send the same POST with dry_run=1 on the URL or "dry_run": true in the body. It is free, creates nothing, ignores Idempotency-Key (so the real POST can send the same key), and its data.schedule adds the month's figures: slots_in_month (the runs this UTC calendar month holds on the schedule's grid, counted for the whole month even when the schedule starts later in it), credits_per_month (every run at its estimated maximum, so an upper bound), money_per_credit (amount in currency GBP; basis is active_pack for the pack you bought, or starter_list before you buy one), money_per_month, and recommended_max_credits_per_month (a monthly cap that lets every run of the longest month start, with each run at its estimated maximum and the last one at your max_credits_per_run, plus 10%; it is never below max_credits_per_run, which the real POST requires). next_step says how to set max_credits_per_month from them. For a budget in money, divide it by money_per_credit.amount.
  • Each later run starts from the previous run's watermark (see Only new rows), so it delivers only rows not delivered before.
  • max_credits_per_run caps each run and defaults to the request's max_credits. With neither, or with a cap below what one run is estimated to cost, the answer is a free 400 on schedule.max_credits_per_run with a suggested_value from the estimate; the same check runs for a schedule with start_at, as does the balance check. max_credits_per_month pauses the schedule when one more run could take this UTC month's spend past it: that slot is skipped (discover.run_skipped, reason max_credits_per_month) and the pause sends discover.run_failed. It resumes by itself the next month, unless you paused it yourself.
  • A run fires only when the previous one has finished. If it is still running or paused, that slot is skipped and recorded (previous_running, previous_paused), as is a slot with too little balance or key budget for max_credits_per_run. Three skips in a row pause the schedule and say why. A late fire runs once, never once per missed slot.
  • Sending the same scheduled request again from the same key never makes a second schedule: the answer is 200 with the existing schedule and deduplicated: true (differs names the caps or every you sent differently; change them with resume). Send an Idempotency-Key too, so a retry after a timeout replays the first answer.
  • A schedule with authors or author_list_id needs schedule.purpose (business, brand or creator); a key holds at most 20 of them.
  • Sending the same request with schedule again from the same key answers the schedule that already exists (200, data.deduplicated: true) and starts nothing; send Idempotency-Key to replay the first answer exactly (Idempotency). A skipped slot whose run was refused records the refusal's own reason (for example max_credits_below_estimate) and the schedule's next_step says how to fix it.

Manage schedules with GET /v1/prism/discover/schedules (yours, newest first), GET /v1/prism/discover/schedules/{schedule_id} (with its last 10 runs and this month's spend), POST .../pause, POST .../resume (optionally with new caps) and DELETE /v1/prism/discover/schedules/{schedule_id}; all free. GET /v1/prism/discover/runs?schedule_id= lists one schedule's runs. When a scheduled run settles with rows, its schedule sends discover.new_rows (or discover.new_accounts and discover.new_items for those views) with the count and the first 100 new ids, after the suppression list (the count can include rows suppressed past those first 100), and links.changes, the run read that returns exactly those rows with their text and judgments (changes_since set to the event's since_version, which is 0 on a run's first event); a skipped slot sends discover.run_skipped, and a failed run or a schedule that pauses itself sends discover.run_failed. A pause you send yourself sticks: the schedule stays paused, sends nothing, and starts no run until you resume it. A universal search watch is a schedule too, so it is listed here and these routes pause and resume it.

Events and webhooks

A background run tells you when it moves: discover.progress (every 10%), discover.accounts_ready, and one settle event, discover.completed, discover.partial, discover.paused or discover.failed. Schedules add the events above. Each event carries counts and ids only, never row text, and links to the run's routes. A run answered in the call sends nothing.

Read them two ways, both free:

  • Push. Register your URL once with POST /v1/webhook-endpoints { "url": "https://example.com/hooks", "types": ["discover.*"] }. Every run on your account then posts its events there, signed with Standard Webhooks and retried until you answer 2xx. The answer shows the signing secret once. See Event webhooks for verifying, testing and rotating.
  • Pull. GET /v1/events?types=discover.* reads the same events in order from a cursor, and wait_s holds the read open until one arrives. See Events.

There is no webhook field on discover itself: one registration covers every run (see What is not offered).

Next steps