SocialCrawl
Cookbooks

A monthly report on a category and its rivals

Collect last month's posts about several products or brands in one run, with each post's stance, and turn them into a summary table each month

Send one request with a group for each product or brand you compare. You get back last month's posts for each group, with every post's stance, and a count per group. Put the same request on a monthly schedule and the report builds itself. The example compares oat milk, almond milk and soy milk.

Measured on 18/11/2026: one run for October 2026 kept 171 posts across the three groups, each with its stance, for 18 credits. Half of that paid for the searches and half for the stance labels.

Start in five minutes

Set your key once. Every block on this page reads it.

Terminal
export SOCIALCRAWL_API_KEY=YOUR_API_KEY

Paste this prompt into your coding agent:

Prompt
Read https://www.socialcrawl.dev/docs/recipes/monthly-report.md and follow it to build last month's report on oat milk, almond milk and soy milk. Use the key in SOCIALCRAWL_API_KEY. Price the request with a dry run first ("dry_run": true, which is free), and show me the estimate. Send max_credits=60, and ask me before you create a schedule.

Or price it yourself. A dry run plans the searches and quotes them for 0 credits:

cURL
curl -X POST "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  -H "content-type: application/json" \
  -d '{"groups":[{"id":"oat-milk","keywords":["oat milk"]},{"id":"almond-milk","keywords":["almond milk"]},{"id":"soy-milk","keywords":["soy milk"]}],"platforms":["reddit","youtube","news"],"period":"last_month","label":["stance"],"limit":50,"max_credits":60,"dry_run":true}'

On 18/11/2026 it quoted about 18 credits, at most 45.

How it works

  • Each group is one product or brand, searched on every platform you list.
  • period=last_month keeps the posts of the last calendar month, so every report covers the same span.
  • Each kept post is judged for its stance toward its group's keywords.
  • A post that two groups both found is kept once, under the first group.
  • The run works in the background. You follow it, then download every row in one file.

What you need

CallWhat it givesCredits
POST /v1/prism/discover with dry_runThe plan and the price0
POST /v1/prism/discoverThe run, in the backgroundEach search at its platform's price, plus 1 per started 25 posts for stance, up to max_credits
GET /v1/prism/discover/runs/{run_id}/results.jsonlEvery row, one per line0

Steps

Step 1: start the run

Send the same body without dry_run. It answers 202 at once, with the run and its status_url.

cURL
curl -X POST "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  -H "content-type: application/json" \
  -d '{"groups":[{"id":"oat-milk","keywords":["oat milk"]},{"id":"almond-milk","keywords":["almond milk"]},{"id":"soy-milk","keywords":["soy milk"]}],"platforms":["reddit","youtube","news"],"period":"last_month","label":["stance"],"limit":50,"max_credits":60}'
Python
import json, os, urllib.request

body = {"groups": [{"id": "oat-milk", "keywords": ["oat milk"]},
                   {"id": "almond-milk", "keywords": ["almond milk"]},
                   {"id": "soy-milk", "keywords": ["soy milk"]}],
        "platforms": ["reddit", "youtube", "news"], "period": "last_month",
        "label": ["stance"], "limit": 50, "max_credits": 60}
req = urllib.request.Request("https://www.socialcrawl.dev/v1/prism/discover", data=json.dumps(body).encode(),
                             headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"], "content-type": "application/json"})
answer = json.load(urllib.request.urlopen(req, timeout=120))
print(answer["data"]["run"]["run_id"], answer["data"]["next"]["status_url"])
  • Each group takes its own keywords, and can take its own platforms, country and languages.
  • limit is the number of posts kept per group. The POST keeps up to 500 by default.
  • max_credits is required on the POST. The run pauses there and never overspends.

Step 2: follow it and download the rows

Call status_url until data.run.status is no longer queued or running. Then download results.jsonl. Both reads are free.

Python
import json, os, time, urllib.request

KEY = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]}
STATUS_URL = os.environ["STATUS_URL"]  # data.next.status_url from step 1
answer = json.load(urllib.request.urlopen(urllib.request.Request(STATUS_URL, headers=KEY), timeout=120))
while answer["data"]["run"]["status"] in ("queued", "running"):
    time.sleep(10)
    answer = json.load(urllib.request.urlopen(urllib.request.Request(STATUS_URL, headers=KEY), timeout=120))
run_id = answer["data"]["run"]["run_id"]
url = f"https://www.socialcrawl.dev/v1/prism/discover/runs/{run_id}/results.jsonl"
with open("rows.jsonl", "wb") as f:
    f.write(urllib.request.urlopen(urllib.request.Request(url, headers=KEY), timeout=120).read())
for group, g in answer["data"]["summary"]["by_group"].items():
    print(group, g["kept"]["delivered"], "posts", g["stance"])
Terminal
curl -sS "https://www.socialcrawl.dev/v1/prism/discover/runs/$RUN_ID/results.jsonl" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" > rows.jsonl

Each line of results.jsonl is one post, with its group, and the last line is {"complete":true}. The run's summary.by_group holds the counts for your table. On 18/11/2026 it read:

GroupPostsPositiveNegativeNeutralNo opinionUncertain
oat-milk7210722726
almond-milk5814424412
soy-milk5413020417

The three groups add up to more than 171, because a post that two groups found is counted in both. In results.jsonl it appears once, under the first group, with the other in also_in_groups.

Step 3: repeat it every month

Add a schedule to the same body. The example keeps dry_run, so it prices the schedule and creates nothing. Remove dry_run to create it.

cURL
curl -X POST "https://www.socialcrawl.dev/v1/prism/discover" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  -H "content-type: application/json" \
  -d '{"groups":[{"id":"oat-milk","keywords":["oat milk"]},{"id":"almond-milk","keywords":["almond milk"]},{"id":"soy-milk","keywords":["soy milk"]}],"platforms":["reddit","youtube","news"],"period":"last_month","label":["stance"],"limit":50,"schedule":{"every":"30d","max_credits_per_run":60},"dry_run":true}'
  • Send start_at with the first day of next month, so each run covers a whole month.
  • Each run ends with a discover.completed event in the event feed.

Store only what is new

Each monthly run covers a different month, so every row is new to that month's report. Store the rows by month and group, keyed on each post's url. To compare months, keep the summary of each month beside the last. A weekly run with period=7d on a schedule starts from the last run's watermark, so it returns only posts it has not delivered.

What it costs

PartCredits
The dry run0
Each search that returns postsThat platform's search price
label=stance1 per started 25 posts
Following the run and downloading the rows0
Creating a schedule0

What the example runs measured on 18/11/2026:

RunPostsCredits
The dry run00
The run for October 202617118
The schedule's dry run, every 30 days00
The full program straight after, the same bodythe same 1710

The second request found the first run of the same body and returned it, so it cost nothing.

Tips, and what not to use

  • Use product categories or large public brands as groups. A brand name that is also an ordinary word needs a context that says which meaning you want.
  • For brands, send brands instead of groups. Each brand becomes a group and gets the mention judgment, which tells a post about the brand from one that only shares its name.
  • Add comments=inline to read the comments under each group's busiest posts. Price it first: it is billed at each platform's comment page price.
  • Do not run one request per group. One request with groups plans every search together and keeps a post two groups found once.

Full program and agent brief

Next steps