# Watch a topic (/docs/recipes/watch-a-topic) Search several platforms for a topic, judge each post's stance, read the comments under the busiest ones, and repeat on a schedule that returns only new posts Send your topic once and get back the last day's posts from several platforms. Each post carries its stance, and the busiest ones carry their comments. Then put the same request on a schedule, and every run returns only the posts the last one did not. The example watches what people say about oat milk in coffee. Measured on 18/11/2026: one run kept 39 posts from the last day and returned 20, for 17 credits. That paid for 8 credits of searches, 2 of stance labels and 7 for 26 comments under 3 posts. ## Start in five minutes Set your key once. Every block on this page reads it. ```bash title="Terminal" export SOCIALCRAWL_API_KEY=YOUR_API_KEY ``` Paste this prompt into your coding agent: ```text title="Prompt" Read https://www.socialcrawl.dev/docs/recipes/watch-a-topic.md and follow it to watch posts about oat milk and oat lattes on X, Reddit, Threads and news. Use the key in SOCIALCRAWL_API_KEY. Price the request with a dry run first (dry_run=true, which is free), and show me the estimate. Send max_credits=60 on every run, and ask me before you create a schedule. ``` Or price it yourself. A dry run plans the searches and quotes them for 0 credits: ```bash title="cURL" curl -G "https://www.socialcrawl.dev/v1/prism/discover" \ -H "x-api-key: $SOCIALCRAWL_API_KEY" \ --data-urlencode "keywords=oat milk,oat latte" \ --data-urlencode "platforms=twitter,reddit,threads,news" \ --data-urlencode "period=1d" \ --data-urlencode "label=stance" \ --data-urlencode "comments=inline" \ --data-urlencode "comments.top_posts=3" \ --data-urlencode "limit=20" \ --data-urlencode "max_credits=60" \ --data-urlencode "dry_run=true" ``` On 18/11/2026 it quoted about 26 credits, at most 59. ## How it works - Each keyword runs as one search on each platform you list. - Posts outside your window are dropped, and the same post found twice is kept once. - Each kept post is judged for its stance toward the topic. - The comments under the busiest posts are read into the post's row. - A schedule repeats the request and starts each run where the last one stopped. ## What you need | Call | What it gives | Credits | | --- | --- | --- | | `GET /v1/prism/discover` | One run now, answered in the call | Each search at its platform's price, plus labels and comments, up to `max_credits` | | `POST /v1/prism/discover` with `schedule` | The same request on a schedule | 0 to create, each run billed like the call above | | `GET /v1/events` or a webhook endpoint | Word that a scheduled run has ended | 0 | ## Steps ### Step 1: run it once **cURL** ```bash title="cURL" curl -G "https://www.socialcrawl.dev/v1/prism/discover" \ -H "x-api-key: $SOCIALCRAWL_API_KEY" \ --data-urlencode "keywords=oat milk,oat latte" \ --data-urlencode "platforms=twitter,reddit,threads,news" \ --data-urlencode "period=1d" \ --data-urlencode "label=stance" \ --data-urlencode "comments=inline" \ --data-urlencode "comments.top_posts=3" \ --data-urlencode "limit=20" \ --data-urlencode "max_credits=60" ``` **Python** ```python title="Python" import json, os, urllib.parse, urllib.request params = {"keywords": "oat milk,oat latte", "platforms": "twitter,reddit,threads,news", "period": "1d", "label": "stance", "comments": "inline", "comments.top_posts": "3", "limit": "20", "max_credits": "60"} url = "https://www.socialcrawl.dev/v1/prism/discover?" + urllib.parse.urlencode(params) req = urllib.request.Request(url, headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]}) data = json.load(urllib.request.urlopen(req, timeout=120))["data"] for post in data["items"]: stance = post["judgments"].get("stance") or {} comments = (post.get("comments") or {}).get("items") or [] print(post["platform"], stance.get("label"), len(comments), post["url"]) print("save for next time:", data["next"]["watermark"]) ``` - `period=1d` keeps the posts of the last day. A watch that runs every six hours still sends `1d`, and the watermark removes what it has already seen. - `label=stance` adds `judgments.stance` to each post, a label with its confidence. It costs 1 credit per started 25 posts. - `comments=inline` reads the comments under the busiest 3 posts into a `comments` block on each of them. News rows have no comments to read. ```json title="Response (trimmed)" { "data": { "items": [ { "platform": "twitter", "url": "https://x.com/VeganRecovering/status/2123032624772120643", "published_at": "2026-11-18T12:36:36.000Z", "text": "The end of oat milk is, apparently, upon us.", "judgments": { "stance": { "label": "negative", "confidence": 0.73 } } } ], "summary": { "stance": { "positive": 17, "negative": 3, "no_opinion": 4, "uncertain": 15 } }, "next": { "watermark": "eyJ2IjoxLCJ3Ijoi..." } } } ``` `summary.stance` counts every kept post, not only the 20 returned. A post whose stance could not be told has a `null` label and counts as `uncertain`. ### Step 2: put it on a schedule Send the same fields as JSON with a `schedule`. The example keeps `dry_run`, so it prices the schedule and creates nothing. Remove `dry_run` to create it. **cURL** ```bash title="cURL" curl -X POST "https://www.socialcrawl.dev/v1/prism/discover" \ -H "x-api-key: $SOCIALCRAWL_API_KEY" \ -H "content-type: application/json" \ -d '{"keywords":["oat milk","oat latte"],"platforms":["twitter","reddit","threads","news"],"period":"1d","label":["stance"],"comments":"inline","limit":20,"schedule":{"every":"6h","max_credits_per_run":60,"max_credits_per_month":3000},"dry_run":true}' ``` **Python** ```python title="Python" import json, os, urllib.request body = {"keywords": ["oat milk", "oat latte"], "platforms": ["twitter", "reddit", "threads", "news"], "period": "1d", "label": ["stance"], "comments": "inline", "limit": 20, "schedule": {"every": "6h", "max_credits_per_run": 60, "max_credits_per_month": 3000}, "dry_run": True} # remove to create the schedule req = urllib.request.Request("https://www.socialcrawl.dev/v1/prism/discover", data=json.dumps(body).encode(), headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"], "content-type": "application/json"}) data = json.load(urllib.request.urlopen(req, timeout=120))["data"] print(data["estimate"]["credits"], data["schedule"]["credits_per_month"], "credits a month at most") ``` - `every` takes `1h`, `6h`, `12h`, `1d`, `7d` or `30d`. - `max_credits_per_run` caps each run, and `max_credits_per_month` pauses the schedule until the next month once a run could pass it. - The dry run answers what a month costs at most. Creating the schedule is free. ### Step 3: hear when a run ends Each scheduled run ends with an event. Read them from the [event feed](/docs/events.md), or register a [webhook endpoint](/docs/webhook-endpoints.md) for types such as `discover.*` and receive them as signed pushes. Each event names the run, and the run's rows are a free read. ## Store only what is new A schedule does this for you: each run starts from the previous run's watermark, so it returns only newer posts. When you run the request yourself instead, save `data.next.watermark` from each answer. Send it back as `since_watermark` next time. Key every post you store on its `url`, so a post that two keywords both found stays one row. ## What it costs | Part | Credits | | --- | --- | | The dry run | 0 | | Each search that returns posts | That platform's search price | | Judging against your keywords | 0 | | Judging against a different `context` | 1 per started 15 posts | | `label=stance` | 1 per started 25 posts | | Comments under the busiest posts | Each platform's comment page price | | Creating a schedule | 0 | What the example runs measured on 18/11/2026: | Run | Posts | Credits | | --- | --- | --- | | The dry run | 0 | 0 | | The first run | 20 returned, 39 kept | 17 | | The schedule's dry run, every 6 hours | 0 | 0, and at most 7,080 a month | | The full program a minute later, with the watermark | 0 new | 0 | ## Tips, and what not to use - Leave `context` out until you need it. On this request the dry run priced it at 18 of about 37 credits, and judging against the keywords is free. - Set `max_credits_per_month` from the dry run's `recommended_max_credits_per_month`, so every run of the month can start. - Do not start a watch with `GET /v1/search/everywhere`. It costs 20 credits a search whatever it finds. In one measured watch, the same keywords cost 420 credits on it against 211 on Discover. Keep it for a one-off pulse check. ## Full program and agent brief **Full program (Python)** ```python title="Python" # watch_a_topic.py: new posts about a topic, with stance and top comments, stored once. # Run every few hours: SOCIALCRAWL_API_KEY=... python3 watch_a_topic.py (Python 3.9+, nothing to install) import json import os import sqlite3 import urllib.parse import urllib.request from collections import Counter KEY = os.environ["SOCIALCRAWL_API_KEY"] WATCH = { "keywords": "oat milk,oat latte", "platforms": "twitter,reddit,threads,news", "period": "1d", "label": "stance", "comments": "inline", "comments.top_posts": "3", "limit": "20", "max_credits": "60", } db = sqlite3.connect("watch.db") db.execute("CREATE TABLE IF NOT EXISTS posts (url TEXT PRIMARY KEY, platform TEXT, published_at TEXT, stance TEXT, text TEXT, comments TEXT)") db.execute("CREATE TABLE IF NOT EXISTS state (k TEXT PRIMARY KEY, v TEXT)") row = db.execute("SELECT v FROM state WHERE k = 'watermark'").fetchone() params = dict(WATCH, **({"since_watermark": row[0]} if row else {})) url = "https://www.socialcrawl.dev/v1/prism/discover?" + urllib.parse.urlencode(params) req = urllib.request.Request(url, headers={"x-api-key": KEY}) with urllib.request.urlopen(req, timeout=120) as res: answer = json.load(res) data = answer["data"] new, stances = 0, Counter() for post in data["items"]: stance = ((post.get("judgments") or {}).get("stance") or {}).get("label") or "uncertain" comments = [c.get("text") for c in (post.get("comments") or {}).get("items") or []] cur = db.execute("INSERT OR IGNORE INTO posts VALUES (?, ?, ?, ?, ?, ?)", (post["url"], post["platform"], post.get("published_at"), stance, post.get("text"), json.dumps(comments))) if cur.rowcount: new += 1 stances[stance] += 1 print(f"{post['platform']:8} {stance:10} {len(comments):>3} comments {post['url']}") watermark = (data.get("next") or {}).get("watermark") if watermark: db.execute("INSERT OR REPLACE INTO state VALUES ('watermark', ?)", (watermark,)) db.commit() print(f"{new} new posts {dict(stances)}; run {data['run']['status']}, {data['run']['credits']['spent']} credits") ``` **Agent brief** ```text title="Agent brief" Goal: every new post about oat milk and oat lattes on X, Reddit, Threads and news, with its stance and the comments under the busiest posts. Read https://www.socialcrawl.dev/docs/recipes/watch-a-topic.md first. 1. Price GET /v1/prism/discover with the page's parameters and dry_run=true. Show me the estimate. 2. Run it with max_credits=60. Store each post by url with its stance and comments. 3. Save data.next.watermark and send it as since_watermark on the next run. 4. To schedule it, send the POST with schedule and dry_run true first, show me the monthly cost, and create it only when I say so. ``` ## Next steps - [Monthly report](/docs/recipes/monthly-report.md): Last month - [Discover](/docs/engines/discover.md): How Discover searches, judges and schedules. - [Event feed](/docs/events.md): Read what your scheduled runs produced.