100 free credits. No credit card required.Start building
Logo
Utility logo
100 free credits. No credit card required

Utility Task Recipes API

Returns the task recipes: each job's chain of calls with its inputs, binds, paging and setup, pitfalls, alternatives, and the whole job's credit cost for its stated size.

GET /v1/utility/recipes

Last updated November 2026Maintained by the SocialCrawl team

When to use the Utility Task Recipes API

Use it to copy a proven multi-step job and see what it costs before you build it. It runs nothing. Free to call.

From the API reference

The catalog of task recipes: common jobs (export a video's comments, vet a creator, compare a product's reviews across retailers, watch a competitor's uploads, crawl a site to markdown, and more) as the cheapest chain of calls, with the inputs each needs, which step's rows feed the next (bind), which steps page or run once per row (repeat), any setup call (a monitor, a cohort), pitfalls, cheaper and deeper alternatives, and cost: the whole job's credits for the stated size, computed from current prices, with exact false when a page count had to be guessed. A recipe monitor also shows per_run, one scheduled run. Every recipe carries last_run: what it really spent the last time we ran it end to end against the API (date, calls, credits, the ITEMS it ran with, and measured, the same in one sentence), or null before its first run. Pass id for one recipe and it also carries code: the whole job as a TypeScript program, a Python program that needs nothing installed, and an n8n workflow, each reading your key from SOCIALCRAWL_API_KEY and the row cap from ITEMS, following async runs and jobs to their end, and printing what it spent. A utility/plan job id (creator_profile) returns the recipe that documents it. Free to call.

75 platforms behind one API

·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
Utility API

What can you do with the Task Recipes API?

The Task Recipes endpoint gives you structured Utility data with computed fields in a single request. No scraping infrastructure to build or maintain.

Example Request

# GET /v1/utility/recipes: Free: 0 credits per call.
# --max-time 60: the API's own time limit for this endpoint.
# --retry 3: retries timeouts, 429 and 5xx, honoring Retry-After.
# -o response.json keeps only the final attempt's body.
# One Idempotency-Key per request, reused on every retry, so a retried call is never charged twice.
IDEMPOTENCY_KEY=$(od -An -N16 -tx1 /dev/urandom | tr -d ' \n')
curl -sS --max-time 60 --retry 3 -o response.json \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Idempotency-Key: $IDEMPOTENCY_KEY" \
  "https://www.socialcrawl.dev/v1/utility/recipes?id=export-tiktok-comments" && cat response.json
# GET /v1/utility/recipes: Free: 0 credits per call.
import os
import random
import time
import uuid

import requests

API_KEY = os.environ.get("SOCIALCRAWL_API_KEY", "YOUR_API_KEY")
TIMEOUT_S = 60  # the API's own time limit for this endpoint.
MAX_RETRIES = 3
URL = "https://www.socialcrawl.dev/v1/utility/recipes"


def backoff(attempt):
    # Exponential backoff with jitter: about 1 s, 2 s, then 4 s.
    return 2 ** attempt * (0.5 + random.random() / 2)


def socialcrawl(params):
    # One Idempotency-Key per request, reused on every retry, so a retried call is never charged twice.
    idempotency_key = str(uuid.uuid4())
    for attempt in range(MAX_RETRIES + 1):
        try:
            res = requests.get(
                URL,
                params=params,
                headers={"x-api-key": API_KEY, "Idempotency-Key": idempotency_key},
                timeout=TIMEOUT_S,
            )
            body = res.json()
        except (requests.RequestException, ValueError):
            # A timeout, a dropped connection or a non-JSON gateway page: retry with the same key.
            if attempt == MAX_RETRIES:
                raise
            time.sleep(backoff(attempt))
            continue
        if body["success"]:
            return body
        error = body["error"]
        # Branch on error["type"], never on the message text.
        if error["type"] == "RESOURCE_NOT_FOUND":
            return None  # refunded: nothing to read
        retry_after = res.headers.get("Retry-After")
        # SERVICE_UNAVAILABLE without Retry-After is a withdrawn endpoint: permanent.
        permanent = error["type"] == "SERVICE_UNAVAILABLE" and retry_after is None
        if not error.get("retryable") or permanent or attempt == MAX_RETRIES:
            raise RuntimeError(f"{error['type']}: {error['message']} (request_id {body['request_id']})")
        # Retry-After (seconds) wins; otherwise back off.
        time.sleep(float(retry_after) if retry_after else backoff(attempt))


params = {
    "id": "export-tiktok-comments",
}
body = socialcrawl(params)
if body is not None:
    print(body["data"])
// GET /v1/utility/recipes: Free: 0 credits per call.
const API_KEY = process.env.SOCIALCRAWL_API_KEY ?? "YOUR_API_KEY";
const TIMEOUT_MS = 60_000; // the API's own time limit for this endpoint.
const MAX_RETRIES = 3;

type Page = { next_cursor: string | null; has_more: boolean };
type SocialCrawlResponse =
  | { success: true; data: any; pagination?: Page; credits_used: number; request_id: string }
  | { success: false; error: { type: string; message: string; retryable: boolean }; request_id: string };

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));
// Exponential backoff with jitter: about 1 s, 2 s, then 4 s.
const backoffMs = (attempt: number) => 2 ** attempt * 1000 * (0.5 + Math.random() / 2);

async function socialcrawl(url: URL) {
  // One Idempotency-Key per request, reused on every retry, so a retried call is never charged twice.
  const idempotencyKey = crypto.randomUUID();
  for (let attempt = 0; ; attempt++) {
    let res: Response;
    let body: SocialCrawlResponse;
    try {
      res = await fetch(url, {
        headers: {
          "x-api-key": API_KEY,
          "Idempotency-Key": idempotencyKey,
        },
        signal: AbortSignal.timeout(TIMEOUT_MS),
      });
      body = (await res.json()) as SocialCrawlResponse;
    } catch (error) {
      // A timeout, a dropped connection or a non-JSON gateway page: retry with the same key.
      if (attempt >= MAX_RETRIES) throw error;
      await sleep(backoffMs(attempt));
      continue;
    }
    if (body.success) return body;
    // Branch on error.type, never on the message text.
    if (body.error.type === "RESOURCE_NOT_FOUND") return null; // refunded: nothing to read
    const retryAfter = res.headers.get("Retry-After");
    // SERVICE_UNAVAILABLE without Retry-After is a withdrawn endpoint: permanent.
    const permanent = body.error.type === "SERVICE_UNAVAILABLE" && retryAfter === null;
    if (!body.error.retryable || permanent || attempt >= MAX_RETRIES) {
      throw new Error(`${body.error.type}: ${body.error.message} (request_id ${body.request_id})`);
    }
    // Retry-After (seconds) wins; otherwise back off.
    await sleep(retryAfter ? Number(retryAfter) * 1000 : backoffMs(attempt));
  }
}

const url = new URL("https://www.socialcrawl.dev/v1/utility/recipes");
url.searchParams.set("id", "export-tiktok-comments");

const body = await socialcrawl(url);
if (body) console.log(body.data);

Parameters

ParameterRequiredDescription
idNoOne recipe id (export-tiktok-comments), which adds its runnable code. Omit for the whole catalog
Example Response

What does the Utility Task Recipes API return?

This synthetic fixture shows the documented response shape and fields. Values are illustrative; this is not a live API response.

Example response
{
  "success": true,
  "platform": "utility",
  "endpoint": "/v1/utility/recipes",
  "data": {
    "kind": "recipe_catalog",
    "version": 1,
    "count": 2,
    "recipes": [
      {
        "id": "export-tiktok-comments",
        "title": "Export the latest N comments on a TikTok video",
        "when": [
          "get/export/download comments on this TikTok",
          "last 200 comments",
          "TikTok comments to CSV"
        ],
        "inputs": [
          {
            "name": "url",
            "description": "The TikTok video URL",
            "example": "https://www.tiktok.com/@stoolpresidente/video/7623818255903329566"
          }
        ],
        "steps": [
          {
            "id": "comments",
            "endpoint": "tiktok/post/comments",
            "params": {
              "url": "{url}",
              "sort": "recent",
              "scan_pages": "3"
            },
            "repeat": "until_n_items",
            "note": "Reads up to three pages per call and returns them newest first.",
            "method": "GET"
          }
        ],
        "size": {
          "description": "two hundred comments",
          "items": 200
        },
        "cost": {
          "calls": 5,
          "hold": 15,
          "expected_min": 5,
          "expected_max": 15,
          "exact": false,
          "floor": true,
          "for_items": 200
        },
        "cheaper": [
          {
            "endpoint": "prism/comments",
            "why": "One call with max=N that pages to N for you, metered per comment page scanned."
          }
        ],
        "pitfalls": [
          "sort=recent sorts only what was read; TikTok has no native newest-first order, so raise scan_pages to read more of the thread.",
          "The endpoint is tiktok/post/comments; tiktok/video/comments does not exist.",
          "On prism/comments set max (the default is a thousand) and replies=false unless you want replies, or the hold is far larger than the job."
        ],
        "last_run": {
          "ran_on": "2026-10-25",
          "ok": true,
          "items": 25,
          "calls": 1,
          "credits_used": 3,
          "hold": 3,
          "formats": {
            "typescript": true,
            "python": true
          },
          "measured": "Measured on 25/10/2026: 3 credits in 1 call for ITEMS=25 (the quote held 3). The TypeScript and Python programs both ran."
        },
        "code": {
          "typescript": "// Export the latest N comments on a TikTok video\n//\n// Price for two hundred comments: ≥ 15 credits held up front (settles 5-15; the unused hold is refunded); a count or fan-out is unknown, so this is a floor: quote the whole job with GET /v1/utility/estimate or scripts/estimate.py.\n// Measured on 25/10/2026: 3 credits in 1 call for ITEMS=25 (the quote held 3). The TypeScript and Python programs both ran.\n//\n// ITEMS caps the rows a step collects or loops over (default 200, two hundred comments); the price scales with it.\n// FRESH=1 skips the cache and bills every call; a cache hit is free.\n// Inputs, set to a working example (change them to yours):\n//   url: The TikTok video URL\n//\n// Run it with Node 22 or newer: SOCIALCRAWL_API_KEY=sc_... npx tsx recipe.ts\n\nconst API_KEY = process.env.SOCIALCRAWL_API_KEY;\nif (!API_KEY) throw new Error(\"Set SOCIALCRAWL_API_KEY to your SocialCrawl API key\");\nconst BASE_URL = (process.env.SOCIALCRAWL_BASE_URL ?? \"https://www.socialcrawl.dev\").replace(/\\/+$/, \"\");\nconst TIMEOUT_MS = 300_000; // the longest route time limit, per attempt\nconst MAX_RETRIES = 3;\n// FRESH=1 skips the cache: every call is fetched again and billed normally (a cache hit is free).\nconst FRESH = process.env.FRESH === \"1\";\nconst WAIT_LIMIT_MS = 15 * 60_000; // how long to follow one run or job\nconst RUNNING = new Set([\"queued\",\"running\",\"pending\"]);\n// A job read answers the job's status; read it again until it is terminal.\nconst JOB_READ = /^(web\\/jobs|cohort-queries)\\/[^/]+$/;\n\nlet callCount = 0;\nlet creditsSpent = 0;\nconst runCredits = new Map<string, number>();\nconst createdPaths: string[] = [];\n\nconst sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));\n// Exponential backoff with jitter: about 1 s, 2 s, then 4 s.\nconst backoff = (attempt: number) => 2 ** attempt * (500 + Math.random() * 500);\n\ninterface CallOptions {\n  method?: string;\n  body?: unknown;\n  idempotencyKey?: string;\n}\n\n// One request. Retries a network error and an error the API marks retryable\n// (Retry-After wins), but a POST only when it carries an Idempotency-Key, so\n// a retry never runs a paid write twice. Returns the parsed body, or null for\n// RESOURCE_NOT_FOUND (refunded: nothing to read). Throws on any other error.\nasync function send(url: string, opts: CallOptions = {}): Promise<any> {\n  const method = opts.method ?? \"GET\";\n  const retries = method === \"GET\" || opts.idempotencyKey ? MAX_RETRIES : 0;\n  for (let attempt = 0; ; attempt++) {\n    let res: Response;\n    let body: any;\n    try {\n      res = await fetch(url, {\n        method,\n        headers: {\n          \"x-api-key\": API_KEY!,\n          accept: \"application/json\",\n          ...(FRESH ? { \"cache-control\": \"no-cache\" } : {}),\n          ...(opts.body === undefined ? {} : { \"content-type\": \"application/json\" }),\n          ...(opts.idempotencyKey ? { \"idempotency-key\": opts.idempotencyKey } : {}),\n        },\n        body: opts.body === undefined ? undefined : JSON.stringify(opts.body),\n        signal: AbortSignal.timeout(TIMEOUT_MS),\n      });\n      body = await res.json();\n    } catch (err) {\n      if (attempt >= retries) throw err;\n      await sleep(backoff(attempt));\n      continue;\n    }\n    callCount++;\n    if (res.ok && body?.success !== false) {\n      // A run (prism/discover) reports its whole spend on data.run.credits.spent.\n      const run = body?.data?.run;\n      if (run?.run_id && typeof run.credits?.spent === \"number\") runCredits.set(run.run_id, run.credits.spent);\n      else creditsSpent += body?.credits_used ?? 0;\n      return body;\n    }\n    const error = body?.error ?? { type: `HTTP_${res.status}`, message: res.statusText };\n    // Branch on error.type, never on the message text.\n    if (error.type === \"RESOURCE_NOT_FOUND\") {\n      console.warn(`Skipped, not found (refunded): ${url}`);\n      return null;\n    }\n    const retryAfter = res.headers.get(\"retry-after\");\n    if (!error.retryable || attempt >= retries)\n      throw new Error(`${error.type}: ${error.message} (request_id ${body?.request_id ?? \"none\"})`);\n    await sleep(retryAfter ? Number(retryAfter) * 1000 : backoff(attempt));\n  }\n}\n\n// What a run created and keeps (a monitor, a cohort), printed with how to delete it.\nfunction noteCreated(path: string, method: string | undefined, body: any) {\n  if (method !== \"POST\" || !body) return;\n  const id =\n    path === \"monitors\" ? body.monitor?.id\n    : path === \"cohorts\" ? body.data?.id\n    : path === \"web/monitors\" ? body.data?.monitor_id\n    : undefined;\n  if (!id) return;\n  createdPaths.push(`${path}/${id}`);\n  console.log(`Created ${path}/${id}. Delete it with DELETE /v1/${path}/${id} when you no longer need it.`);\n}\n\n// One API call by its path under /v1. Follows an async run (202) and a job to its end.\nasync function sc(path: string, params: Record<string, unknown> = {}, opts: CallOptions = {}): Promise<any> {\n  const url = new URL(`${BASE_URL}/v1/${path}`);\n  for (const [key, value] of Object.entries(params))\n    if (value !== undefined && value !== null) url.searchParams.set(key, String(value));\n  let body = await send(url.toString(), opts);\n  const deadline = Date.now() + WAIT_LIMIT_MS;\n  // An async run answers 202 and keeps going: follow data.next.status_url until it ends.\n  let statusUrl = body?.data?.next?.status_url;\n  while (statusUrl && RUNNING.has(body?.data?.run?.status)) {\n    if (Date.now() > deadline) throw new Error(`${path} is still running: follow ${statusUrl}`);\n    await sleep(1000);\n    body = await send(statusUrl);\n    statusUrl = body?.data?.next?.status_url ?? statusUrl;\n  }\n  while (JOB_READ.test(path) && RUNNING.has(body?.data?.status)) {\n    if (Date.now() > deadline) throw new Error(`${path} is still ${body.data.status}: read it again later`);\n    await sleep((body.data.poll_after_s ?? 5) * 1000);\n    body = await send(url.toString());\n  }\n  noteCreated(path, opts.method, body);\n  return body;\n}\n\n// Rows in a step's result: a page's items, or the answers of a step run once per row.\nfunction rowsIn(value: any): number {\n  if (Array.isArray(value)) return value.filter((v) => v != null).length;\n  if (!value) return 0;\n  return Array.isArray(value.data?.items) ? value.data.items.length : 1;\n}\n\nconst ITEMS = Number(process.env.ITEMS ?? 200);\n\nconst comments: unknown[] = [];\nlet cursor: string | undefined;\ndo {\n  const r = await sc(\"tiktok/post/comments\", { url: \"https://www.tiktok.com/@stoolpresidente/video/7623818255903329566\", sort: \"recent\", scan_pages: \"3\", cursor });\n  comments.push(...r.data.items);\n  cursor = r.pagination?.has_more ? r.pagination.next_cursor : undefined;\n} while (cursor && comments.length < ITEMS);\n\nconst totalCredits = creditsSpent + [...runCredits.values()].reduce((a, b) => a + b, 0);\nconsole.log(`Spent ${totalCredits} credits in ${callCount} calls.`);\nconst rowCounts = { comments: rowsIn(comments) };\nconsole.log(\n  \"RECIPE_RESULT \" +\n    JSON.stringify({\n      recipe: \"export-tiktok-comments\",\n      calls: callCount,\n      credits_used: totalCredits,\n      rows: rowCounts,\n      created: createdPaths,\n    }),\n);\n",
          "python": "# Export the latest N comments on a TikTok video\n#\n# Price for two hundred comments: ≥ 15 credits held up front (settles 5-15; the unused hold is refunded); a count or fan-out is unknown, so this is a floor: quote the whole job with GET /v1/utility/estimate or scripts/estimate.py.\n# Measured on 25/10/2026: 3 credits in 1 call for ITEMS=25 (the quote held 3). The TypeScript and Python programs both ran.\n#\n# ITEMS caps the rows a step collects or loops over (default 200, two hundred comments); the price scales with it.\n# FRESH=1 skips the cache and bills every call; a cache hit is free.\n# Inputs, set to a working example (change them to yours):\n#   url: The TikTok video URL\n#\n# Run it with Python 3.9 or newer, nothing to install: SOCIALCRAWL_API_KEY=sc_... python3 recipe.py\n\nimport json\nimport os\nimport random\nimport re\nimport sys\nimport time\nimport http.client\nimport urllib.error\nimport urllib.parse\nimport urllib.request\nimport uuid\n\nAPI_KEY = os.environ.get(\"SOCIALCRAWL_API_KEY\") or sys.exit(\"Set SOCIALCRAWL_API_KEY to your SocialCrawl API key\")\nBASE_URL = os.environ.get(\"SOCIALCRAWL_BASE_URL\", \"https://www.socialcrawl.dev\").rstrip(\"/\")\nTIMEOUT_S = 300  # the longest route time limit, per attempt\nMAX_RETRIES = 3\n# FRESH=1 skips the cache: every call is fetched again and billed normally (a cache hit is free).\nFRESH = os.environ.get(\"FRESH\") == \"1\"\nWAIT_LIMIT_S = 15 * 60  # how long to follow one run or job\nRUNNING = {\"queued\", \"running\", \"pending\"}\n# A job read answers the job's status; read it again until it is terminal.\nJOB_READ = re.compile(r\"^(web/jobs|cohort-queries)/[^/]+$\")\n\ncall_count = 0\ncredits_spent = 0\nrun_credits = {}\ncreated_paths = []\n\n\ndef dig(value, *keys):\n    for key in keys:\n        if not isinstance(value, dict):\n            return None\n        value = value.get(key)\n    return value\n\n\ndef backoff(attempt):\n    # Exponential backoff with jitter: about 1 s, 2 s, then 4 s.\n    return 2 ** attempt * (0.5 + random.random() / 2)\n\n\ndef send(url, method=\"GET\", body=None, idempotency_key=None):\n    # One request. Retries a network error and an error the API marks retryable\n    # (Retry-After wins), but a POST only when it carries an Idempotency-Key, so\n    # a retry never runs a paid write twice. Returns the parsed body, or None for\n    # RESOURCE_NOT_FOUND (refunded: nothing to read). Exits on any other error.\n    global call_count, credits_spent\n    headers = {\"x-api-key\": API_KEY, \"accept\": \"application/json\"}\n    if FRESH:\n        headers[\"Cache-Control\"] = \"no-cache\"\n    data = None\n    if body is not None:\n        data = json.dumps(body).encode()\n        headers[\"content-type\"] = \"application/json\"\n    if idempotency_key:\n        headers[\"Idempotency-Key\"] = idempotency_key\n    retries = MAX_RETRIES if method == \"GET\" or idempotency_key else 0\n    for attempt in range(retries + 1):\n        request = urllib.request.Request(url, data=data, method=method, headers=headers)\n        try:\n            with urllib.request.urlopen(request, timeout=TIMEOUT_S) as res:\n                status, retry_after, payload = res.status, res.headers.get(\"Retry-After\"), json.loads(res.read() or b\"null\")\n        except urllib.error.HTTPError as err:\n            status, retry_after = err.code, err.headers.get(\"Retry-After\")\n            try:\n                payload = json.loads(err.read() or b\"null\")\n            except (ValueError, OSError, http.client.HTTPException):\n                payload = None\n        # A network error, a timeout, a connection cut mid-body, or a body that is not JSON.\n        except (OSError, http.client.HTTPException, ValueError):\n            if attempt == retries:\n                raise\n            time.sleep(backoff(attempt))\n            continue\n        call_count += 1\n        if 200 <= status < 300 and dig(payload, \"success\") is not False:\n            # A run (prism/discover) reports its whole spend on data.run.credits.spent.\n            run = dig(payload, \"data\", \"run\") or {}\n            spent = dig(run, \"credits\", \"spent\")\n            if run.get(\"run_id\") and isinstance(spent, (int, float)):\n                run_credits[run[\"run_id\"]] = spent\n            elif isinstance(payload, dict):\n                credits_spent += payload.get(\"credits_used\") or 0\n            return payload\n        # A gateway page that is not our JSON (a 502 or 503 in front of the API) is retryable.\n        error = dig(payload, \"error\") or {\"type\": f\"HTTP_{status}\", \"message\": \"no error body\", \"retryable\": status >= 500}\n        # Branch on error type, never on the message text.\n        if error.get(\"type\") == \"RESOURCE_NOT_FOUND\":\n            print(f\"Skipped, not found (refunded): {url}\", file=sys.stderr)\n            return None\n        if not error.get(\"retryable\") or attempt == retries:\n            sys.exit(f\"{error.get('type')}: {error.get('message')} (request_id {dig(payload, 'request_id')})\")\n        time.sleep(float(retry_after) if retry_after else backoff(attempt))\n\n\ndef note_created(path, method, body):\n    # What a run created and keeps (a monitor, a cohort), printed with how to delete it.\n    if method != \"POST\" or not body:\n        return\n    ids = {\"monitors\": dig(body, \"monitor\", \"id\"), \"cohorts\": dig(body, \"data\", \"id\"), \"web/monitors\": dig(body, \"data\", \"monitor_id\")}\n    created = ids.get(path)\n    if created:\n        created_paths.append(f\"{path}/{created}\")\n        print(f\"Created {path}/{created}. Delete it with DELETE /v1/{path}/{created} when you no longer need it.\")\n\n\ndef sc(path, params=None, method=\"GET\", body=None, idempotency_key=None):\n    # One API call by its path under /v1. Follows an async run (202) and a job to its end.\n    url = f\"{BASE_URL}/v1/{path}\"\n    query = {k: (str(v).lower() if isinstance(v, bool) else str(v)) for k, v in (params or {}).items() if v is not None}\n    if query:\n        url += \"?\" + urllib.parse.urlencode(query)\n    result = send(url, method, body, idempotency_key)\n    deadline = time.time() + WAIT_LIMIT_S\n    # An async run answers 202 and keeps going: follow data.next.status_url until it ends.\n    status_url = dig(result, \"data\", \"next\", \"status_url\")\n    while status_url and dig(result, \"data\", \"run\", \"status\") in RUNNING:\n        if time.time() > deadline:\n            sys.exit(f\"{path} is still running: follow {status_url}\")\n        time.sleep(1)\n        result = send(status_url)\n        status_url = dig(result, \"data\", \"next\", \"status_url\") or status_url\n    while JOB_READ.match(path) and dig(result, \"data\", \"status\") in RUNNING:\n        if time.time() > deadline:\n            sys.exit(f\"{path} is still running: read it again later\")\n        wait_s = dig(result, \"data\", \"poll_after_s\")\n        time.sleep(5 if wait_s is None else wait_s)\n        result = send(url)\n    note_created(path, method, result)\n    return result\n\n\ndef rows_in(value):\n    # Rows in a step's result: a page's items, or the answers of a step run once per row.\n    if isinstance(value, list):\n        return len([v for v in value if v is not None])\n    if not value:\n        return 0\n    items = dig(value, \"data\", \"items\")\n    return len(items) if isinstance(items, list) else 1\n\n\nITEMS = int(os.environ.get(\"ITEMS\", \"200\"))\n\ncomments = []\nparams = {\"url\": \"https://www.tiktok.com/@stoolpresidente/video/7623818255903329566\", \"sort\": \"recent\", \"scan_pages\": \"3\"}\nwhile True:\n    r = sc(\"tiktok/post/comments\", params)\n    comments.extend(r[\"data\"][\"items\"])\n    page = r.get(\"pagination\") or {}\n    if not page.get(\"has_more\") or not page.get(\"next_cursor\") or len(comments) >= ITEMS:\n        break\n    params = {**params, \"cursor\": page[\"next_cursor\"]}\n\ntotal_credits = credits_spent + sum(run_credits.values())\nprint(f\"Spent {total_credits} credits in {call_count} calls.\")\nrow_counts = {\"comments\": rows_in(comments)}\nprint(\n    \"RECIPE_RESULT \"\n    + json.dumps(\n        {\n            \"recipe\": \"export-tiktok-comments\",\n            \"calls\": call_count,\n            \"credits_used\": total_credits,\n            \"rows\": row_counts,\n            \"created\": created_paths,\n        }\n    )\n)\n",
          "n8n": "{\n  \"name\": \"SocialCrawl: Export the latest N comments on a TikTok video\",\n  \"nodes\": [\n    {\n      \"id\": \"node-1\",\n      \"position\": [\n        260,\n        0\n      ],\n      \"name\": \"Run recipe\",\n      \"type\": \"n8n-nodes-base.manualTrigger\",\n      \"typeVersion\": 1,\n      \"parameters\": {}\n    },\n    {\n      \"id\": \"node-2\",\n      \"position\": [\n        520,\n        0\n      ],\n      \"name\": \"comments\",\n      \"type\": \"n8n-nodes-base.httpRequest\",\n      \"typeVersion\": 4.2,\n      \"parameters\": {\n        \"method\": \"GET\",\n        \"url\": \"https://www.socialcrawl.dev/v1/tiktok/post/comments?url=https%3A%2F%2Fwww.tiktok.com%2F%40stoolpresidente%2Fvideo%2F7623818255903329566&sort=recent&scan_pages=3\",\n        \"authentication\": \"genericCredentialType\",\n        \"genericAuthType\": \"httpHeaderAuth\",\n        \"options\": {\n          \"timeout\": 300000,\n          \"pagination\": {\n            \"pagination\": {\n              \"paginationMode\": \"updateAParameterInEachRequest\",\n              \"parameters\": {\n                \"parameters\": [\n                  {\n                    \"type\": \"qs\",\n                    \"name\": \"cursor\",\n                    \"value\": \"={{ $response.body.pagination.next_cursor }}\"\n                  }\n                ]\n              },\n              \"paginationCompleteWhen\": \"other\",\n              \"completeExpression\": \"={{ !$response.body.pagination || !$response.body.pagination.has_more }}\",\n              \"limitPagesFetched\": true,\n              \"maxRequests\": 10\n            }\n          }\n        }\n      },\n      \"credentials\": {\n        \"httpHeaderAuth\": {\n          \"name\": \"SocialCrawl API key\"\n        }\n      },\n      \"executeOnce\": true,\n      \"retryOnFail\": true,\n      \"maxTries\": 4,\n      \"waitBetweenTries\": 2000\n    },\n    {\n      \"id\": \"note\",\n      \"name\": \"How this recipe works\",\n      \"type\": \"n8n-nodes-base.stickyNote\",\n      \"typeVersion\": 1,\n      \"position\": [\n        0,\n        -460\n      ],\n      \"parameters\": {\n        \"content\": \"## Export the latest N comments on a TikTok video\\n\\nPrice for two hundred comments: ≥ 15 credits held up front (settles 5-15; the unused hold is refunded); a count or fan-out is unknown, so this is a floor: quote the whole job with GET /v1/utility/estimate or scripts/estimate.py.\\n\\nMeasured on 25/10/2026: 3 credits in 1 call for ITEMS=25 (the quote held 3). The TypeScript and Python programs both ran.\\n\\nITEMS caps the rows a step collects or loops over (default 200, two hundred comments); the price scales with it.\\n\\nFRESH=1 skips the cache and bills every call; a cache hit is free.\\n\\nInputs, set to a working example (change them to yours):\\n\\n  url: The TikTok video URL\\n\\n**Key:** create a Header Auth credential named SocialCrawl API key, with Name x-api-key and your API key as Value, and pick it on every request node.\\n\\n**Rows:** a step that runs once per row takes the first 200 rows (change the number in its rows node). A paging step reads up to 10 pages: raise Max Pages under Options, Pagination to fetch more.\\n\\n**Errors:** a request node stops the workflow on an API error. Read error.type in its output; retry only when error.retryable is true.\",\n        \"width\": 520,\n        \"height\": 420\n      }\n    }\n  ],\n  \"connections\": {\n    \"Run recipe\": {\n      \"main\": [\n        [\n          {\n            \"node\": \"comments\",\n            \"type\": \"main\",\n            \"index\": 0\n          }\n        ]\n      ]\n    }\n  },\n  \"settings\": {}\n}\n"
        }
      }
    ],
    "notes": [
      "cost is the whole job for size.items rows (paging and per-row fan-out included), from the same pricing the API bills with. hold is what is held up front across every call; unused hold is refunded.",
      "exact is false when a page count, a price unit or a fan-out had to be guessed; floor is true when the cost is a lower bound.",
      "Setup calls (monitors, cohorts) are free to create and are not priced here; per_run is one scheduled run of a recipe monitor.",
      "GET /v1/utility/plan?query= turns a job in plain words into one of these recipes with your own values filled in.",
      "last_run is what the recipe spent the last time we ran it end to end against the API (measured is the same in one sentence); null means it has not been measured yet.",
      "Pass id for one recipe to get code: the whole job as a TypeScript program, a Python program (standard library only) and an n8n workflow, each reading your key from SOCIALCRAWL_API_KEY and the row cap from ITEMS."
    ]
  },
  "credits_used": 0,
  "request_id": "req_example000000",
  "cached": false
}

Synthetic fixture for the documented response shape. Values are illustrative, not a production capture.

API Details

How does the Utility Task Recipes API work?

Send a GET request with your API key and get back clean, structured JSON in our unified schema. Supported computed fields are populated when the source provides the required inputs.

Method

GET

Response

JSON

Why SocialCrawl

Why use SocialCrawl for Utility Task Recipes data?

We handle the complexity of Utility data extraction so you can focus on building. Unified schema, AI enrichment, and zero platform logic in your code.

Developer First

How do you scrape social media data in seconds?

The fastest social media scraping API for developers. Scrape profiles, posts, comments, and analytics from 75 platforms covering 10B+ monthly active users.

One schema, every platform

Query 75 platforms with identical response structures. Write your integration once.

Computed fields, not just scraped

When an endpoint supports these metrics and the source provides the required inputs, the normalized record includes engagement_rate, estimated_reach, content_category, and language. Ready to use.

See your data before you code

Visual Data Explorer. Paste any URL, get rich result cards, sortable tables, CSV export.

import requests

response = requests.get(
    'https://www.socialcrawl.dev/v1/tiktok/profile',
    params={'handle': 'charlidamelio'},
    headers={'x-api-key': 'sc_YOUR_API_KEY'}
)
data = response.json()
[ .JSON ]
{
  "success": true,
  "platform": "tiktok",
  "data": {
    "author": {
      "username": "charlidamelio",
      "followers": 152400000
    },
    "engagement": {
      "likes": 12400000000,
      "engagement_rate": 0.087
    },
    "metadata": {
      "language": "en",
      "content_category": "lifestyle"
    }
  }
}
+ 75 platforms
FAQ

Have a question? We got answers

Find answers to frequently asked questions about SocialCrawl's API, pricing, and capabilities.

Contact us
What does the Task Recipes API return?
Returns the task recipes: each job's chain of calls with its inputs, binds, paging and setup, pitfalls, alternatives, and the whole job's credit cost for its stated size. The response uses the same JSON envelope as every SocialCrawl endpoint, with the data, credits_used and request_id fields.
When should I use the Task Recipes endpoint?
Use it to copy a proven multi-step job and see what it costs before you build it. It runs nothing. Free to call.
How much does the Task Recipes API cost?
This endpoint is free to call and uses no credits. Every new account still gets 100 free credits for the paid endpoints. Credits for failed or empty calls are refunded automatically, and there is no subscription.
What parameters does the Task Recipes API need?
No parameter is required, so the bare URL works. It also accepts 1 optional parameter, listed on this page.
How do I call the Task Recipes API?
Send a GET request to https://www.socialcrawl.dev/v1/utility/recipes with your API key in the x-api-key header. An Authorization: Bearer header works too.

Ask AI about SocialCrawl

Ready to scrape Utility Task Recipes data?

Get your API key and start pulling Utility data in under 60 seconds.

Start for free

🤖 AI agent or LLM? Read this page as markdown