SocialCrawl

Verify a webhook signature

Check that a delivery came from SocialCrawl, with a verifier for each of the two signing schemes.

Every webhook SocialCrawl sends is signed with a secret only you and SocialCrawl hold. Check the signature over the raw request body before you parse it, and reject any delivery that fails.

There are two signing schemes. Event webhooks use Standard Webhooks. Monitor, billing, batch and status webhooks use the x-socialcrawl-signature header.

Your signing secret

Each secret starts with whsec_ and is shown once, when you create the thing that sends to you:

  • Event webhooks: data.secret in the answer to POST /v1/webhook-endpoints. Rotate it with POST /v1/webhook-endpoints/{id}/rotate-secret; the old secret keeps signing for a day, so deploy the new one before then.
  • Monitors: the create answer. Pass your own webhook_secret when you create a test Monitor, so a fixture can hold a known value.
  • Billing: the answer to saving your URL under Billing → Payments. Saving the URL again issues a new secret.
  • Batch jobs: webhook.secret in the 202 answer.
  • Status subscriptions: subscription.secret in the first delivery, subscription.verification.

SocialCrawl keeps only an encrypted copy, so a lost secret cannot be shown again. Create a new endpoint or Monitor instead.

Verify an event webhook

Every attempt carries three headers:

Header
webhook-id: msg_V1StGXR8Z5jdHi6BmyT
webhook-timestamp: 1792231200
webhook-signature: v1,n4OBPwOJOeyYgw9sOfwuvt4WnxtW6CX30IHU5WLqf5Y=

The signature is the base64 HMAC-SHA256 of {webhook-id}.{webhook-timestamp}.{raw body}. The key is your secret after the whsec_ prefix, decoded from base64. Reject a timestamp more than 300 seconds from your clock.

JavaScript
import { createHmac, timingSafeEqual } from "node:crypto";

// True if the Standard Webhooks signature over the raw body is valid and fresh.
export function verify(secret, headers, rawBody, nowSeconds = Date.now() / 1000) {
  const id = headers["webhook-id"], ts = headers["webhook-timestamp"];
  if (Math.abs(nowSeconds - Number(ts)) > 300) return false;
  const key = Buffer.from(secret.replace(/^whsec_/, ""), "base64");
  const expected = createHmac("sha256", key).update(`${id}.${ts}.`).update(rawBody).digest();
  return headers["webhook-signature"].split(" ").some((entry) => {
    const [version, sig] = entry.split(",");
    const got = Buffer.from(sig ?? "", "base64");
    return version === "v1" && got.length === expected.length && timingSafeEqual(got, expected);
  });
}
Python
import base64, hashlib, hmac, time

def verify(secret: str, headers: dict, body: bytes, now: float | None = None) -> bool:
    """True if the Standard Webhooks signature over the raw body is valid and fresh."""
    msg_id, ts, sigs = headers["webhook-id"], headers["webhook-timestamp"], headers["webhook-signature"]
    if abs((now or time.time()) - int(ts)) > 300:
        return False
    key = base64.b64decode(secret.removeprefix("whsec_"))
    signed = f"{msg_id}.{ts}.".encode() + body
    expected = base64.b64encode(hmac.new(key, signed, hashlib.sha256).digest()).decode()
    for entry in sigs.split(" "):
        version, _, sig = entry.partition(",")
        if version == "v1" and hmac.compare_digest(sig, expected):
            return True
    return False

During a secret rotation, webhook-signature holds one v1, entry per secret, separated by a space. A match on any entry is valid.

Check your verifier against these two vectors. Both use the made-up secret whsec_bGlzdGVuaW5nLXRlc3QtdmVjdG9yLWtleS0wMDAwMDE= and webhook-timestamp: 1792231200, so pass that time as now. The body is the exact bytes shown, with no trailing newline.

Vectorwebhook-idBodyExpected webhook-signature
1msg_test_0001{"id":"bat_test_0001","type":"event.batch","data":{"events":[]}}v1,n4OBPwOJOeyYgw9sOfwuvt4WnxtW6CX30IHU5WLqf5Y=
2msg_test_0002the body belowv1,7p7cIiickEFY7ppxKoCdaHAVxoWojbHPo4+AzKLweSY=
Vector 2 body
{"id":"bat_test_0002","type":"event.batch","data":{"events":[{"id":"evt_test_0001","type":"mention.created","test":true,"api_version":"2026-10-01","created_at":"2026-10-17T10:00:00Z","cursor":"c_test_0001","subject":{"kind":"listening_stream","id":"lst_test","name":"Client B: Globex Mobile"},"data":{"platform":"twitter","methodology_version":"1","mention":{"mention_id":"men_test_0001","evidence":{"id":"tw_000000000000","platform":"twitter","url":"https://x.com/example/status/1","published_at":"2026-10-17","text":"Globex Mobile is down again","author":"example","group":"outage","group_tags":{"client":"client-b"}},"first_seen_at":"2026-10-17T09:59:30Z"}}}]}}

Both verifiers accept both vectors. They reject vector 2 with one byte added, and vector 1 checked 301 seconds late.

Verify a Monitor, billing, batch or status webhook

These deliveries carry one header:

Header
x-socialcrawl-signature: t=1792231200,v1=<hex>

v1 is the hex HMAC-SHA256 of <t>.<raw body>, keyed with your whole secret, whsec_ prefix included. t is the Unix time in seconds. On a billing delivery it equals the body's created.

JavaScript
import crypto from "node:crypto";

const TOLERANCE_SECONDS = 300;

export function verify(rawBody, header, secret, nowSeconds = Date.now() / 1000) {
  const parts = Object.fromEntries(
    header.split(",").map((kv) => {
      const i = kv.indexOf("=");
      return [kv.slice(0, i), kv.slice(i + 1)];
    }),
  );
  const t = Number(parts.t);
  if (!Number.isFinite(t)) return false;
  // Replay window: reject anything older than the tolerance.
  if (Math.abs(Math.floor(nowSeconds) - t) > TOLERANCE_SECONDS) {
    return false;
  }

  const expected = crypto
    .createHmac("sha256", secret)
    .update(`${t}.${rawBody}`)
    .digest("hex");
  const a = Buffer.from(parts.v1 ?? "", "hex");
  const b = Buffer.from(expected, "hex");
  // timingSafeEqual throws a RangeError on a length mismatch, and a truncated
  // or non-hex v1 produces a short buffer, so compare lengths first.
  if (a.length !== b.length) return false;
  return crypto.timingSafeEqual(a, b);
}
Python
import hashlib, hmac, time

TOLERANCE_SECONDS = 300

def verify(raw_body: bytes, header: str, secret: str, now: float | None = None) -> bool:
    parts = dict(kv.split("=", 1) for kv in header.split(","))
    try:
        t = int(parts["t"])
    except (KeyError, ValueError):
        return False
    # Replay window: reject anything older than the tolerance.
    if abs(int(now or time.time()) - t) > TOLERANCE_SECONDS:
        return False

    expected = hmac.new(
        secret.encode(),
        f"{t}.".encode() + raw_body,
        hashlib.sha256,
    ).hexdigest()
    return hmac.compare_digest(parts.get("v1", ""), expected)

Check this verifier with the made-up secret whsec_6d6f6e69746f722d746573742d766563746f722d6b65792d3031, now set to 1792231200, and this body with no trailing newline:

Vector 3 body
{"monitor_id":"mon_test_0001","run_id":"run_test_0001","recipe":"tiktok/profile","status":"ok","scheduled_for":"2026-10-17T10:00:00.000Z","alerts_fired":[],"result":{},"deltas":{}}
Expected header
x-socialcrawl-signature: t=1792231200,v1=e44212cc832286a825dd3e0cfb90b25a1db196b00272c24fcdb34d981d6ca30f

De-duplicate on the id

Every push channel delivers at least once, so the same delivery can arrive twice. Record the id before you act, and treat a second delivery with a known id as done.

ChannelDe-duplicate on
Event webhookEach inner event id
Monitor webhookrun_id
Billing webhookevent and created together
Batch webhookjob_id
Status webhookincident.id with the body's event

For calls you send to SocialCrawl, the same idea is the idempotency key.

Best practices

  • Verify over the raw bytes before any JSON parser touches the body. A parsed and re-serialized body has different bytes, so every check fails.
  • Answer 2xx first and do the slow work afterwards, so a delivery is not retried while you work.
  • Never log the secret or the full signature header.

Next steps

On this page