SocialCrawl

Social listening pulse-check

Stream a "what is the internet saying right now about X" feed across 12 platforms with one API call — ranked, clustered, and enriched with real comments.

Social listening pulse-check

Build a streaming "what is the internet saying right now about X" feed for a brand-monitoring dashboard, an editorial newsroom, or an investor screening a thesis.

How do you search all social media platforms at once?

Call GET /v1/search/everywhere with a query. SocialCrawl fans the query out across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, Threads, Pinterest, Perplexity, and Tavily in parallel, then returns ranked, clustered results enriched with each post's top real-people comments — one request, one flat 20-credit charge.

The problem

Listening to one platform misses the conversation. A product launch trends on X, gets dissected on Reddit, becomes a meme on TikTok, and shows up as a prediction market on Polymarket — all in the same afternoon. Stitching together a dozen platform searches means a dozen integrations, a dozen response shapes, and a dozen ranking problems.

The solution

One endpoint does the fan-out, fusion, and ranking for you:

  • GET /v1/search/everywhere — fans out across Reddit, X, YouTube, TikTok, Instagram, Hacker News, Polymarket, GitHub, Threads, Pinterest, Perplexity, and Tavily; returns ranked + clustered results with the top comments from each post

The endpoint supports two response modes selected by the Accept header. Streaming is the default below because chunks arrive as enrichment lands — you can render results before the request finishes.

// recipe-pulse-check.ts
// Streams universal search results for a single topic.
// Run with: SOCIALCRAWL_KEY=sc_... npx tsx recipe-pulse-check.ts

const KEY = process.env.SOCIALCRAWL_KEY;
if (!KEY) throw new Error("Set SOCIALCRAWL_KEY");

const url = new URL("https://www.socialcrawl.dev/v1/search/everywhere");
url.searchParams.set("query", "anthropic claude 4");
url.searchParams.set("lookback_days", "7");

const res = await fetch(url, {
  headers: {
    "x-api-key": KEY,
    accept: "text/event-stream",
  },
});

if (!res.ok || !res.body) {
  throw new Error(`Search failed: ${res.status} ${await res.text()}`);
}

const decoder = new TextDecoder();
let buffer = "";

for await (const chunk of res.body) {
  buffer += decoder.decode(chunk as Uint8Array, { stream: true });
  const events = buffer.split("\n\n");
  buffer = events.pop() ?? "";

  for (const event of events) {
    const dataLine = event
      .split("\n")
      .find((line) => line.startsWith("data: "));
    if (!dataLine) continue;
    const payload = JSON.parse(dataLine.slice(6));

    switch (payload.type) {
      case "plan_refined":
        console.log("plan ready, sub-queries:", payload.plan.subqueries.length);
        break;
      case "clusters":
        console.log(`clustered into ${payload.clusters.length} themes`);
        break;
      case "comments_enriched":
        for (const item of payload.items) {
          const top = item.source_items[0]?.metadata?.top_comments?.[0];
          console.log(`[${item.source}] ${item.title}`);
          if (top) console.log(`  → "${top.text.slice(0, 120)}…"`);
        }
        break;
      case "done":
        console.log(
          `done — ${payload.summary.total_items} results from ${payload.summary.sources_called.length} sources, charged ${payload.summary.credits_used}cr`,
        );
        break;
      case "error":
        console.error("search error:", payload.message);
        break;
    }
  }
}

If you don't need streaming, switch the Accept header to application/json and the same endpoint returns the full ranked set in one response body — same data, no chunk handling.

What you get back

// One `comments_enriched` SSE chunk — many of these arrive over the stream
{
  "type": "comments_enriched", // <-- discriminator
  "items": [
    {
      "candidate_id": "rd_18gha22",
      "source": "reddit", // <-- one of the 12 platforms
      "title": "Claude 4 is shockingly good at refactors",
      "url": "https://reddit.com/r/ClaudeAI/comments/...",
      "snippet": "I gave it a 4k-line Rust file and...",
      "final_score": 0.91, // <-- RRF + rerank fused
      "cluster_id": "cluster_3", // <-- groups related posts
      "source_items": [
        {
          "metadata": {
            "top_comments": [
              // <-- real-people sentiment
              {
                "text": "Same. Just shipped a migration in 20 min.",
                "score": 142,
              },
            ],
          },
        },
      ],
    },
  ],
}

Credits cost

Cost per run: 20 credits flat, regardless of how many sources respond or how much comment enrichment lands. Running this hourly for a day costs 480 credits; daily for a month costs 600.

Take it further

  • See Universal social search for the full chunk schema, narrowing with sources= / exclude=, and date-bounded queries.
  • Swap query to a brand, a competitor, a person, a meme, a vertical, an asset — the endpoint doesn't care.
  • For an alerting loop with deduplication, see Brand mention monitoring.
  • Next: Hybrid search-then-enrich feeds these results into per-platform transcripts.
  • New to the API? Start at the Quickstart.

On this page

Social listening pulse-checkHow do you search all social media platforms at once?The problemThe solutionWhat you get backCredits costTake it further
How to Pulse-Check Any Topic Across Social Media with One API Call | Socialcrawl