SocialCrawl

Monitor Webhooks

Signed, retried webhooks that fire when a scheduled Monitor run completes, with the run result, fired alerts, and deltas

A Monitor runs a saved request (a recipe and its inputs) on a schedule and sends each run to your webhook. Each time a run completes, SocialCrawl POSTs a signed JSON body to the webhook URL you registered, carrying the run result, any alerts that fired, and the deltas against the previous run. Add track and a Monitor keeps only the numbers you pick, per row, and can deliver them as spreadsheet rows or skip the webhook entirely (see Numbers-only Monitors).

Billing webhooks and Monitor webhooks share the exact same signing scheme, so one verification routine covers both. If you already verify Billing Webhooks, you are done.

This page is about Monitors. Discover runs and discover schedules send their events (discover.completed, discover.new_rows and the rest) to event webhooks instead, one registration for every run. Webhooks and events compares every watch and delivery channel.

Prerequisites

  • An API key from socialcrawl.dev, under Dashboard → API Keys.
  • An HTTPS endpoint on a public host that can read a raw request body. Loopback, private, and link-local addresses are rejected at registration.
  • Enough credits to cover the recipe: each run bills the recipe's own cost plus 1 credit for scheduling.

How do I receive Monitor webhooks?

Create the Monitor

Monitors are a stateful resource under /v1/monitors, authenticated with the same x-api-key header as the rest of the API.

cURL
curl -X POST 'https://www.socialcrawl.dev/v1/monitors' \
  -H 'x-api-key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "recipe": "search/everywhere",
    "params": { "query": "acme corp" },
    "cadence": "daily",
    "webhook_url": "https://example.com/hooks/socialcrawl",
    "alert_rules": [{ "metric": "coverage", "op": "lt", "value": 0.6 }],
    "suppress_webhook_unless_alert": false
  }'
FieldRequiredNotes
recipeYesAny endpoint path without the /v1/ prefix, for example search/everywhere or tiktok/profile.
paramsNoThe recipe's own parameters. Required parameters are validated at create time, not at first run.
cadenceYes"hourly", "daily", "weekly", or { "cron": "0 9 * * 1" }.
webhook_urlYes, unless track is setHTTPS, public host, 2048 characters or fewer. A Monitor with track and no webhook_url is download-only.
alert_rulesNoArray of { metric, op, value }. op is gt, lt, gte, lte, abs_change_gt, pct_change_gt, or pct_change_lt.
suppress_webhook_unless_alertNoDefault false. When true, only runs with at least one fired alert deliver.
webhook_secretNoSupply your own signing secret (8 to 200 characters), or let SocialCrawl generate a whsec_... value.
trackNo{ "metrics": [...], "row_key"?, "max_rows"? }. Makes a numbers-only Monitor. See Numbers-only Monitors.
webhook_formatNo"full" (default) or "rows". "rows" needs track.
webhook_rowsNo"all" (default) or "new". "new" sends only the rows this monitor has never seen, so a row that comes back is not sent again, and needs track with an items[]. metric and a webhook_url. See Only the new rows.
nameNoDefaults to "<recipe> monitor".

Each account can keep a limited number of Monitors active at once, set by the largest credit pack it has bought: Free 3, Starter 10, Growth 25, Pro 100, Enterprise 500. Paused Monitors do not count.

Store the signing secret

The 201 response returns the monitor plus webhook_secret. That is the only time the plaintext secret is ever shown: SocialCrawl keeps an encrypted copy and cannot display it again.

JSON
{
  "monitor": {
    "id": "mon_7Qk2R9xLpV",
    "name": "search/everywhere monitor",
    "recipe": "search/everywhere",
    "cadence": "daily",
    "status": "active",
    "alert_rules": [{ "metric": "coverage", "op": "lt", "value": 0.6 }],
    "estimated_cost_per_run": 21,
    "estimated_monthly_cost": 630,
    "next_run_at": "2026-07-05T09:00:00.000Z",
    "warnings": []
  },
  "webhook_secret": "whsec_..."
}

Each run bills the recipe's own cost plus 1 credit for scheduling, which is why a 20-credit search/everywhere recipe estimates at 21 per run.

Verify every delivery before you trust it

Check the x-socialcrawl-signature header against the raw request body, using the routine in Verifying signatures below. Reject anything that fails.

Respond 2xx promptly

A non-2xx response or a timeout counts as a failed attempt. Acknowledge first and do slow work afterwards, out of band.

The rest of the family: GET /v1/monitors lists them, GET /v1/monitors/{monitor_id} reads one, GET /v1/monitors/{monitor_id}/runs is the run history, GET /v1/monitors/{monitor_id}/timeseries projects stored runs into a metric series, GET /v1/monitors/{monitor_id}/export downloads the stored numbers as CSV or NDJSON, PATCH /v1/monitors/{monitor_id} with {"status": "paused"} or {"status": "active"} pauses and resumes (and sets max_credits_per_month), and DELETE /v1/monitors/{monitor_id} unschedules it.

Event catalogue

A Monitor webhook fires once per completed run. The run's own status decides whether a delivery happens at all.

Run statusDelivers?Meaning
okYesThe recipe returned full coverage. result holds the run's data (see Payload for its shape).
partialYesThe recipe ran but some legs did not return. You were refunded for the uncovered portion.
failedNoThe recipe returned no usable data. The run is fully refunded and no webhook is sent.
skippedNoThe run was skipped because your balance could not cover it. Nothing is charged and no webhook is sent.

One more filter sits on top: with suppress_webhook_unless_alert: true, deliveries are limited to runs where at least one alert rule fired. A quiet run sends nothing.

Payload

Every delivery is a JSON object with this shape.

JSON
{
  "monitor_id": "mon_7Qk2R9xLpV",
  "run_id": "run_9Fh1Ab3Cd7",
  "recipe": "search/everywhere",
  "status": "ok",
  "scheduled_for": "2026-07-04T09:00:00.000Z",
  "alerts_fired": [
    {
      "metric": "coverage",
      "op": "lt",
      "from": null,
      "to": 0.53,
      "delta": null,
      "pct_change": null
    }
  ],
  "result": {
    "...": "the recipe body for this run"
  },
  "deltas": {
    "coverage": -0.18
  }
}
FieldTypeDescription
monitor_idstringThe Monitor that produced this run.
run_idstringThe run row. Use it to fetch the stored run via GET /v1/monitors/{monitor_id}/runs.
recipestringThe recipe the Monitor runs.
status"ok" or "partial"The run outcome. failed and skipped runs never deliver.
scheduled_forstring (ISO 8601)The scheduled slot this run filled.
alerts_firedarray of alert objectsAlert rules that matched this run. Empty when nothing fired.
resultobjectThe recipe body for this run, which is also what the run stores. For a Prism composite or search/everywhere it is the unified response. For a single-platform endpoint such as tiktok/profile it is the upstream supplier's body, before SocialCrawl maps it to the unified shape.
deltasobject (string to number)Per-metric change versus the previous ok or partial run. Empty on the first run (no prior to diff).

A numbers-only Monitor sends a different body. See Numbers-only Monitors.

Fired alert objects

Each entry in alerts_fired describes one rule that matched.

JSON
{
  "metric": "coverage",
  "op": "pct_change_lt",
  "from": 0.71,
  "to": 0.53,
  "delta": -0.18,
  "pct_change": -25.35
}
FieldTypeDescription
metricstringThe dot-path into result that the rule watches (see below).
opstringThe matched operator: gt, lt, gte, lte, abs_change_gt, pct_change_gt, or pct_change_lt.
fromnumber or nullThe previous run's value. null for absolute-threshold ops (gt, lt, gte, lte).
tonumberThe current run's value.
deltanumber or nullto - from for delta ops. null for absolute-threshold ops.
pct_changenumber or nullPercent change versus the previous value. null for absolute ops, or when the previous value was 0.

Metric paths

metric and every deltas key is a dot-path into result (the recipe body), not into the envelope. For recipe: "search/everywhere" the top-level numeric leaf is coverage, so you write coverage, never result.coverage or data.coverage. For a single-platform endpoint, result is the supplier's body, so the paths follow that supplier's field names rather than the unified shape you get from a /v1 call. To watch unified paths such as author.followers or items[].post.engagement.views, create the Monitor with track.

Paths are discovered by walking the result for finite numeric leaves, up to 4 levels deep; arrays are never metric paths (with track they are, through items[].). A rule whose path does not resolve to a number is skipped with a warning. It never fires, and deltas stays empty for it.

Because the run still delivers a normal-looking webhook, a typo in metric is silent, and with suppress_webhook_unless_alert: true it makes the Monitor go permanently quiet. Call GET /v1/monitors/{monitor_id}/timeseries after the first run to see the exact metric keys your recipe emits, and use one of those.

Numbers-only Monitors

Pass track to keep only the numbers you care about. Each run reads them off the unified response (the same shape a /v1 call returns, whichever supplier answered), stores them per row with the change since the previous run, and stores nothing else.

cURL
curl -X POST 'https://www.socialcrawl.dev/v1/monitors' \
  -H 'x-api-key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "recipe": "tiktok/profile/videos",
    "params": { "handle": "khaby.lame" },
    "cadence": "daily",
    "track": { "metrics": ["items[].post.engagement.views", "items[].post.engagement.likes"] }
  }'
track fieldNotes
metrics1 to 20 paths. items[].post.engagement.views reads one number per row of a list. author.followers or total reads one number off the page, stored under the row id _.
row_keyPath inside a row to its id. Defaults to post.id, comment.id, or author.id by the endpoint's row type. Other list types must set it.
max_rowsRows kept per run, in page order. Default 100, maximum 200.

Paths are checked when you create the Monitor, for free: each one has to land on a number in that recipe's unified response. items[].engagement.views on a post list returns 400 with the corrected path, items[].post.engagement.views, and a typo such as totl returns 400 with Did you mean 'total'?. A path to a text or true/false field returns 400 naming its type. On a recipe with no fixed response shape to check against, such as most Prism composites, the paths are accepted and listed under warnings in the response, so check the numbers of the first run. A Prism endpoint that returns a plain comment or post list, such as prism/profile/comments, is checked like that list, with rows keyed on comment.id. Billing is the same as any Monitor: the recipe's cost plus 1 credit per run, refunded in full when the run fails. The active-monitor limit is the same one above: Free 3, Starter 10, Growth 25, Pro 100, Enterprise 500, from the largest credit pack the account has bought. Paused Monitors do not count. A Monitor created without track is unchanged: webhook_url stays required, and each run still stores the recipe body.

With track:

  • webhook_url is optional. Without it the Monitor is download-only: read it with /export or /timeseries, and the create response returns webhook_secret: null.
  • Every alert_rules metric must be one of track.metrics. An items[]. rule is checked once per row against that row's previous number, and each fired alert carries row_id.
  • Each run's legs holds one entry saying which source in the endpoint's chain answered: rung is primary or fallback, and rung_index is its position (0 is the primary). A fallback source can leave a number empty that the primary fills.
  • A run on which none of the tracked metrics has a value is not charged. It is marked failed, its skip_reason says why, its credits are refunded in full, and it is never the run the next one is compared against, so an alert cannot fire from it.
  • A run whose page comes back with no results is not charged either, the same as a call that returns an empty page. Its total of 0 would describe the empty page rather than the resource, so nothing is recorded: the run is marked failed with a skip_reason, refunded in full, and no alert fires from it.
  • GET /v1/monitors/{monitor_id}/runs?include=numbers returns each run's numbers and deltas. result is always null. The numbers are what is stored. With webhook_format: "full" the page is in that delivery and is not kept. A rows delivery sends only the numbers. A Monitor with no webhook delivers nothing; the numbers are on /export and /timeseries.

Full payload

With the default webhook_format: "full" the delivery keeps the keys shown in Payload. result is the unified response for the run, and two keys change meaning:

JSON
{
  "numbers": {
    "7412233": { "items[].post.engagement.views": 184000 },
    "7419981": { "items[].post.engagement.views": 9200 }
  },
  "deltas": {
    "rows": {
      "7412233": {
        "items[].post.engagement.views": { "prev": 171500, "cur": 184000, "abs": 12500, "pct": 7.29 }
      }
    },
    "rows_new": ["7419981"],
    "rows_gone": []
  }
}

deltas is null on the first run. pct is null when the previous value was 0.

Rows payload for a spreadsheet

webhook_format: "rows" sends one flat entry per row and metric, signed the same way:

JSON
{
  "monitor_id": "mon_7Qk2R9xLpV",
  "run_id": "run_9Fh1Ab3Cd7",
  "recipe": "tiktok/profile/videos",
  "status": "ok",
  "scheduled_for": "2026-09-23T09:00:00.000Z",
  "alerts_fired": [],
  "rows": [
    {
      "t": "2026-09-23T09:00:00.000Z",
      "row_id": "7412233",
      "metric": "items[].post.engagement.views",
      "value": 184000,
      "prev": 171500,
      "abs_change": 12500,
      "pct_change": 7.29
    }
  ]
}

To append them to a Google Sheet, open Extensions, then Apps Script, paste this, and deploy it as a web app that anyone can access. Apps Script cannot read request headers, so it cannot check x-socialcrawl-signature. Put a long random token in the web app URL you register (.../exec?token=...) and compare it instead.

JavaScript
const TOKEN = "a-long-random-string";

function doPost(e) {
  if (e.parameter.token !== TOKEN) return ContentService.createTextOutput("no");
  const body = JSON.parse(e.postData.contents);
  const sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName("Monitor");
  const values = body.rows.map((r) => [
    r.t, body.run_id, r.row_id, r.metric, r.value, r.prev, r.abs_change, r.pct_change,
  ]);
  if (values.length > 0) {
    sheet.getRange(sheet.getLastRow() + 1, 1, values.length, values[0].length).setValues(values);
  }
  return ContentService.createTextOutput("ok");
}

Google answers the POST with a redirect once the script has run. After the first run, check webhook_delivery on GET /v1/monitors/{monitor_id}/runs to see the status your deployment returned. A retried delivery repeats its run_id, so filter duplicates on that column.

Only the new rows

Add webhook_rows: "new" and each delivery carries only the rows this monitor has never seen, in either format. The monitor remembers the id of every row it has read, so a post that drops off a reshuffled page and comes back later is not sent again. With full, result keeps every other key of the page and its items holds only the new rows, numbers holds only their numbers, and deltas is { "rows": {}, "rows_new": [...], "rows_gone": [] }. With rows, only their entries are sent. The body also says "webhook_rows": "new". The first run is the baseline: it says "baseline": true, carries no rows, and remembers every row it read. A rows_new alert on such a monitor counts the same new rows. The run still stores every tracked number, and the deltas stored on each run (GET /v1/monitors/{monitor_id}/runs, export) still compare with the previous run. Deleting the monitor forgets the rows it has seen.

This turns any list into a "new items only" feed. To watch an account's own posts for new comments, monitor prism/profile/comments, which reads the comments under the profile's recent posts in one call:

cURL
curl -X POST 'https://www.socialcrawl.dev/v1/monitors'   -H 'x-api-key: YOUR_API_KEY'   -H 'Content-Type: application/json'   -d '{
    "recipe": "prism/profile/comments",
    "params": { "url": "https://www.instagram.com/nasa/", "posts": "10", "per_post": "50", "since": "now-3h" },
    "cadence": "hourly",
    "webhook_url": "https://example.com/hooks/socialcrawl",
    "track": { "metrics": ["items[].comment.engagement.likes", "total"], "max_rows": 200 },
    "webhook_rows": "new",
    "alert_rules": [{ "metric": "rows_new", "op": "gt", "value": 0 }],
    "suppress_webhook_unless_alert": true
  }'
  • since: "now-3h" is resolved at every run to the timestamp three hours earlier (now-<N>d gives a date N days earlier). On Instagram and YouTube each post then stops at its first older comment, so a quiet hour reads about one comment page a post. Three hours rather than one leaves room for a run that fails.
  • rows_new gt 0 with suppress_webhook_unless_alert: true sends nothing when no comment is new.
  • Tracking total as well keeps a run with no comment in the window a real reading, charged what the call costs (the post list page), so it is not retried.
  • Rows past max_rows are not stored. The list is newest first, so keep each run's comments under 200 with since.
  • The same works on one post: monitor instagram/post/comments with the post's url and sort: "recent".

Export

GET /v1/monitors/{monitor_id}/export downloads what the Monitor stored, one line per run, row, and metric, oldest first. It is free.

cURL
curl 'https://www.socialcrawl.dev/v1/monitors/mon_7Qk2R9xLpV/export?format=csv&from=2026-09-01T00:00:00Z' \
  -H 'x-api-key: YOUR_API_KEY'
Response
t,row_id,metric,value,abs_change,pct_change
2026-09-23T09:00:00.000Z,7412233,items[].post.engagement.views,184000,12500,7.29

format=ndjson returns one JSON object per line with the same keys.

For a spreadsheet, add shape=rows: one line per run and row, with a column per metric. run_at_utc is DD/MM/YYYY HH:mm in UTC. shape=long (the layout above) stays the default.

cURL
curl 'https://www.socialcrawl.dev/v1/monitors/mon_7Qk2R9xLpV/export?format=csv&shape=rows' \
  -H 'x-api-key: YOUR_API_KEY'
Response
run_at_utc,row_id,items[].post.engagement.views,items[].post.engagement.likes
23/09/2026 09:00,7412233,184000,9120

In a CSV, a text cell that starts with =, +, - or @ (a post title or a handle can) is prefixed with an apostrophe, so a spreadsheet shows it as text and does not run it as a formula. Numbers are written as they are. With format=ndjson&shape=rows each line is one run and row: t in ISO 8601, row_id, then its metrics.

A Monitor without track exports the numeric fields /timeseries finds in its stored results, under row id _. Both /export and /timeseries read the newest 1,000 runs in the window; on a numbers-only Monitor /timeseries also returns series, one { row_id, metric, points } per row and metric, and accepts row= to narrow it.

List jobs: one schedule over many accounts

A list job re-reads a list of accounts on one schedule and delivers only the posts it has not delivered before. Send subjects in place of a single subject in params:

cURL
curl -X POST 'https://www.socialcrawl.dev/v1/monitors' \
  -H 'x-api-key: YOUR_API_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "recipe": "tiktok/profile/videos",
    "subjects": ["nasa", "esa", "spacex"],
    "cadence": "daily",
    "webhook_url": "https://example.com/hooks/socialcrawl"
  }'
  • The recipe must be a newest-first feed that takes stop_at_id: tiktok/profile/videos, youtube/channel/videos, youtube/channel/shorts, twitter/user/tweets, threads/user/posts, instagram/profile/posts, instagram/profile/reels and facebook/profile/posts. Any other recipe is refused for free.
  • Each subject is an account's handle, or a page URL on facebook/profile/posts. To list ids instead (user_id, channelId, pageId), set subject_param to that name. The other params in params apply to every subject.
  • A list holds up to 1,000 subjects. Your plan caps the subjects across all your active lists: 100 on Free, 1,000 on Starter, 10,000 on Growth and 100,000 on Pro. A list does not use one of your Monitor slots.

What a run costs. Each run pays a list premium of 1 credit per 100 subjects, at least 1, plus the pages it reads at the recipe's own price. Each subject is read with stop_at_id set to the newest post already delivered, so a subject with nothing new costs one page and a post is never paid for twice. A subject that posted more than a page since the last run walks up to 5 pages. Its first read is one page. A read that fails is not charged, and a run where every read failed is refunded in full. POST /v1/monitors?dry_run=1 returns the quote: estimated_cost_per_run is one page a subject plus the premium, and max_cost_per_run is every subject walking 5 pages.

What a delivery holds. The webhook body has "event": "monitor.list.run", a list object with the counts (subjects, subjects_ok, subjects_failed, rows_new), and rows, each { "subject": "nasa", "row": { ... } } with the post in the recipe's own shape. The first part also carries subjects, one entry per subject with its status (ok, not_found or error), rows_new, first_read and more_available. A run with more than 500 new rows is sent in parts: list.part and list.parts number them, and every part repeats the run_id. A pinned post is delivered once and not again. Leave webhook_url out to collect the rows yourself: GET /v1/monitors/{monitor_id}/runs?include=result returns each run's new rows (up to 5,000 a run).

Editing the list. PATCH /v1/monitors/{monitor_id} with subjects replaces the list, or with add_subjects and remove_subjects changes it. A subject that stays keeps its place, so it is never delivered twice. GET /v1/monitors/{monitor_id}/subjects lists each subject with last_seen_id, last_status and last_read_at. Editing is free.

Verifying signatures

Every delivery carries an x-socialcrawl-signature header, Stripe-style:

Header
x-socialcrawl-signature: t=1700000000,v1=<hex>

The signature is HMAC-SHA256(secret, "<t>.<rawBody>"), where <t> is the Unix timestamp in seconds and <rawBody> is the exact bytes of the request body. Fold t into your check to enforce a replay window.

Your signing secret is the whsec_... value returned once in the create response. Always verify against the raw request body: parsing and re-serializing the JSON changes the bytes and breaks the signature.

JavaScript
import crypto from "node:crypto";

const TOLERANCE_SECONDS = 300;

function verify(rawBody, header, secret) {
  const parts = Object.fromEntries(
    header.split(",").map((kv) => {
      const i = kv.indexOf("=");
      return [kv.slice(0, i), kv.slice(i + 1)];
    }),
  );
  const t = Number(parts.t);
  if (!Number.isFinite(t)) return false;
  // Replay window: reject anything older than the tolerance.
  if (Math.abs(Math.floor(Date.now() / 1000) - t) > TOLERANCE_SECONDS) {
    return false;
  }

  const expected = crypto
    .createHmac("sha256", secret)
    .update(`${t}.${rawBody}`)
    .digest("hex");
  const a = Buffer.from(parts.v1 ?? "", "hex");
  const b = Buffer.from(expected, "hex");
  // timingSafeEqual throws a RangeError on a length mismatch, and a truncated
  // or non-hex v1 produces a short buffer, so compare lengths first.
  if (a.length !== b.length) return false;
  return crypto.timingSafeEqual(a, b);
}
Python
import hashlib, hmac, time

TOLERANCE_SECONDS = 300

def verify(raw_body: bytes, header: str, secret: str) -> bool:
    parts = dict(kv.split("=", 1) for kv in header.split(","))
    try:
        t = int(parts["t"])
    except (KeyError, ValueError):
        return False
    # Replay window: reject anything older than the tolerance.
    if abs(int(time.time()) - t) > TOLERANCE_SECONDS:
        return False

    expected = hmac.new(
        secret.encode(),
        f"{t}.".encode() + raw_body,
        hashlib.sha256,
    ).hexdigest()
    return hmac.compare_digest(parts.get("v1", ""), expected)

Delivery, retries, and auto-pause

Deliveries go out through a retrying queue: up to 6 attempts (the initial delivery plus 5 retries) with exponential backoff. Respond 2xx promptly to acknowledge; a non-2xx response or a timeout counts as a failed attempt.

A successful delivery resets the failure counter. After 10 consecutive failed deliveries the webhook is automatically paused and stops receiving events, and GET /v1/monitors/{monitor_id} shows webhook.status as paused. What happens to the Monitor depends on whether it has anywhere else to deliver:

  • The webhook was its only destination (a Monitor without track and with email_digest off). The Monitor pauses too, so it stops running and billing, and its paused_because reads webhook_failing. Fix your endpoint, then resume it with PATCH /v1/monitors/{monitor_id} and {"status": "active"}. Resuming turns the webhook back on with a failure count of 0. This holds for any resume of such a Monitor, whatever paused it.
  • It has another destination (a Monitor with track, which stores numbers, or with email_digest set to daily). It keeps running, and you read its numbers with GET /v1/monitors/{monitor_id}/export or in the digest. Nothing turns that webhook back on while the Monitor runs. To get webhooks again, fix your endpoint, export what you need, then delete the Monitor and create it again. The new Monitor has a new id, an active webhook and a new signing secret.

Either way you are told in your notifications, as a job.paused item. When the Monitor itself pauses you also get an email that names the Monitor, the reason and how to resume it.

When does a Monitor pause itself?

A Monitor that cannot do its job pauses, so it stops billing, and paused_because on GET /v1/monitors/{monitor_id} says why:

paused_becauseWhat happenedHow it resumes
consecutive_skipsThree runs in a row were skipped because the balance could not pay for them.Add credits, then PATCH it to {"status": "active"}.
monthly_capThe next run would pass the max_credits_per_month you set. That run is skipped and not charged.By itself on the 1st of the next month (UTC). To run sooner, PATCH a higher max_credits_per_month together with {"status": "active"}.
webhook_failingIts webhook stopped after 10 failed deliveries in a row and it had no other destination.Fix your endpoint, then PATCH it to {"status": "active"}.

paused_because is null while a Monitor is active and after you pause it yourself. Each of these pauses puts a job.paused item in your notifications and sends one email.

The monthly cap

max_credits_per_month is optional, in whole credits: set it when you create the Monitor, or later with PATCH /v1/monitors/{monitor_id}, and send null to remove it. It counts what the Monitor's own runs are charged in the calendar month (UTC), a run in progress included, so two runs that overlap cannot both pass a cap that fits one.

  • It must cover at least one run. A cap below the price of one run is refused with 400, at create and on PATCH, and nothing changes.
  • It is not available on search/multi, tiktokshop/sales and the tag searches. Each source of those recipes is billed as its own call on your account, which the cap would not count, so the request is refused with 400. GET /v1/monitors/{monitor_id} and the create quote show legs_billed_separately: true for them. To limit that spend, set a credit limit on the API key the Monitor runs as.
  • A Monitor paused at its cap resumes by itself on the 1st, at its own next slot (a daily Monitor at its usual time of day).

The daily digest

Set email_digest to daily, at create or with PATCH /v1/monitors/{monitor_id}, and the Monitor's runs are added to one email a day (08:00 UTC) with the runs, new rows, alerts and credits of the last 24 hours for every Monitor that has it on. off is the default. A Monitor with the digest on has a second destination, so it keeps running if its webhook stops.

URL requirements, enforced when you register a webhook:

  • Must be HTTPS.
  • Must be a public host. Loopback, private, and link-local addresses are rejected.
  • Must be 2048 characters or fewer.

Schedule a request you keep making

GET /v1/monitors/suggestions lists the requests your account made on 3 or more days in the last 30 days that no Monitor of yours runs yet: the same successful paid GET, with the same parameters. Each item carries what it cost you (days, calls, credits), a suggested_cadence from how often you made it, a monitor body you can POST /v1/monitors as it is once you add a delivery, and suggested_track, the numbers a daily email or a CSV export keeps for that endpoint. It is free and changes nothing. The answer's window_days is the window it looked at: 30 days, or a shorter one (14, 7 or 3) for an account with a very large request log, so the list comes back in time.

curl https://www.socialcrawl.dev/v1/monitors/suggestions \
  -H "x-api-key: $SOCIALCRAWL_API_KEY"
{
  "items": [
    {
      "recipe": "instagram/profile/posts",
      "params": { "handle": "nasa" },
      "subject": "account",
      "days": 5,
      "calls": 6,
      "credits": 6,
      "suggested_cadence": "weekly",
      "monitor": { "recipe": "instagram/profile/posts", "params": { "handle": "nasa" }, "cadence": "weekly" },
      "suggested_track": { "metrics": ["items[].post.engagement.views", "items[].post.engagement.likes"] }
    }
  ],
  "window_days": 30,
  "min_days": 3
}

To schedule one, send its monitor with webhook_url, or with track set to its suggested_track and email_digest: "daily" for the email and the CSV, and add ?dry_run=1 first for the price. A search whose text the log keeps only as a keyed hash is never offered, because it cannot be replayed. A 503 SUGGESTIONS_UNAVAILABLE means the log could not be read in time: try again in a minute.

The same list opens the Jobs page in the dashboard, where every scheduled job of your account (Monitors, Discover and creator search schedules, search watches and Listening streams) is listed with its price per run and what it spent this month. A link of the form /dashboard/jobs?schedule=tiktok/profile/videos&params={"handle":"nasa"}&cadence=daily opens the New job form filled in, with its price.

Idempotency

Handlers must be idempotent. Retries and occasional duplicate deliveries are normal, and the same run can arrive more than once if your endpoint acknowledged late.

De-duplicate on run_id: it is unique per run, stable across retries of that run, and already in the payload. Record it before you act, and treat a second delivery carrying a run_id you have seen as a no-op.

Testing locally

There is no test-fire endpoint, and a loopback URL is rejected at registration, so the loop is:

  1. Expose your handler on a public HTTPS hostname (a tunnel, or a hosted request inspector).
  2. Create a throwaway Monitor pointed at it with cadence: "hourly" and a cheap recipe such as tiktok/profile, and pass your own webhook_secret so a test fixture can hard-code a known value.
  3. Capture one real delivery, then replay its raw body and header against your verify() function offline. That is the fastest way to debug a signature mismatch, because it removes the network from the loop.
  4. Check GET /v1/monitors/{monitor_id}/runs. Each run row carries webhook_delivery with attempts, lastStatus, deliveredAt, and failedAt, which tells you what your endpoint actually returned. Add ?include=result to see the stored result too.
  5. PATCH the Monitor to {"status": "paused"}, or DELETE it, when you are done, so it stops billing.

Troubleshooting

Next steps