Universal Search Forums API
Scrape Universal Search Forums data with one API call. Fans your topic across Reddit, Hacker News, and Korean forums, RRF-fuses + clusters the threads, and enriches hero threads with their top comments by default (East + West). Naver threads are labeled raw (no comment endpoint). Returns a fused `items[]`, raw per-source buckets, thread clusters, and a deterministic computed block (question_share, top_communities). Each row of `items[].top_comments[]` is `{ score, excerpt, truncated, author, url, date }`. The body is `excerpt`, not `text`: this is the search-lane enrichment shape, not the canonical Comment archetype, and it is clipped to 300 characters on a word boundary, with an ellipsis and `truncated: true` when it was cut. `date` is the comment ISO timestamp. Flat 10cr with coverage-based partial refund.
Last updated October 2026Maintained by the SocialCrawl team
Returns one merged, clustered list of discussion threads from Reddit, Hacker News, and Korean forums, with top comments on the main threads clipped to 300 characters.
Use it when you want what real people are asking and answering; search/everywhere also covers video, code, and social posts.
Searching 68 platforms in parallel
What can you do with the Forums API?
The Forums endpoint gives you structured Universal Search data with computed fields in a single request. No scraping infrastructure to build or maintain.
Example Request
curl -H "x-api-key: YOUR_API_KEY" \
"https://www.socialcrawl.dev/v1/search/forums?query=airpods+pro+3+battery&sources=reddit%2Chackernews%2Cnaver_kin&comments=on"import requests
response = requests.get(
"https://www.socialcrawl.dev/v1/search/forums",
params={
'query': 'airpods pro 3 battery',
'sources': 'reddit,hackernews,naver_kin',
'comments': 'on',
},
headers={"x-api-key": "YOUR_API_KEY"},
)
data = response.json()const response = await fetch(
"https://www.socialcrawl.dev/v1/search/forums?query=airpods+pro+3+battery&sources=reddit%2Chackernews%2Cnaver_kin&comments=on",
{
headers: { "x-api-key": "YOUR_API_KEY" },
},
);
const data = await response.json();Parameters
| Parameter | Required | Description |
|---|---|---|
| query | Yes | Search query (2-256 chars), forwarded to every forum search. |
| sources | No | Optional CSV allowlist of forum sources (reddit, hackernews, naver_kin, naver_cafe). Mutually exclusive with exclude. |
| exclude | No | Optional CSV blocklist of forum sources. Mutually exclusive with sources. |
| comments | No | Comment enrichment toggle (on|off, default on). 'off' returns thread-only. |
| timeframe | No | Recency window passed to Reddit; HN filtered client-side (all|day|week|month|year, default all). |
| lookback_days | No | Alt recency window in days (1-365); HN filtered client-side. |
| relevance | No | Optional. Without it every thread already carries items[].computed.relevance ({ p, sense, on_topic }: is the thread about what your query means, or a different thing that shares its words?), free, in the fused order (see judgments). score also reorders threads by relevance blended with their fused rank. filter also drops the threads that are not on topic, lists them in data.relevance.dropped_ids, and counts stance_split, question_share and top_communities over the threads it kept. A thread that could not be judged, or is too short to judge, is never dropped and carries relevance: null. Omit it to keep every thread in the fused order. Included in the flat 10 credits. (score | filter) |
| relevance_threshold | No | Optional, only with relevance. The probability (0 to 1) a thread must reach to count as on topic. Default 0.5. Lower keeps more threads, higher keeps fewer. |
| judgments | No | Optional. on (default) or off. On, every thread also carries items[].computed.relevance ({ p, sense, on_topic }) from the same stance judgment, with data.relevance (mode default) counting it; the fused order is kept and nothing is dropped (relevance=score or filter reorders and drops). off returns the response without it. Included in the flat 10 credits. (on | off) |
What does the Universal Search Forums API return?
Every response follows one unified schema. Here is a real, unmodified response body, so you can see the exact fields you get back before spending a credit.
Example response
{
"success": true,
"platform": "search",
"endpoint": "/v1/search/forums",
"data": {
"items": [
{
"title": "Do Airpods Pro 3 really have terrible battery life?",
"url": "https://www.reddit.com/r/AirpodsPro/comments/1w63726/do_airpods_pro_3_really_have_terrible_battery_life/",
"source": "reddit",
"container": "r/AirpodsPro",
"enrichable": true,
"post": {
"item_id": "1w63726",
"source": "reddit",
"title": "Do Airpods Pro 3 really have terrible battery life?",
"body": "The charging case continues to lose battery even when the AirPods are not in use for several days. Charging the AirPods from 84% to 100% also consumes around 20% of the case’s battery. It's a brand new Airpods Pro 3.",
"url": "https://www.reddit.com/r/AirpodsPro/comments/1w63726/do_airpods_pro_3_really_have_terrible_battery_life/",
"author": "user_79c0e5",
"published_at": "2026-09-03T10:09:52.000Z",
"date_confidence": "low",
"engagement": {
"score": 25,
"num_comments": 54
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "",
"metadata": {
"enrichable": true,
"top_comments": [
{
"score": 42,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_5beea1",
"url": "https://example.com/redacted",
"date": "2026-09-03T10:27:53.966Z"
},
{
"score": 16,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_51150e",
"url": "https://example.com/redacted",
"date": "2026-09-03T12:36:21.327Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"container": "r/AirpodsPro",
"local_relevance": 0.7964285714285714,
"freshness": 51,
"engagement_score": 64,
"source_quality": 0.6,
"local_rank_score": 0.7091785714285714
},
"top_comments": [
{
"score": 42,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_5beea1",
"url": "https://example.com/redacted",
"date": "2026-09-03T10:27:53.966Z"
},
{
"score": 16,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_51150e",
"url": "https://example.com/redacted",
"date": "2026-09-03T12:36:21.327Z"
}
],
"rrf_score": 0.01639344262295082,
"cluster_id": "cluster-1",
"computed": {
"stance": {
"label": "negative",
"confidence": 1
},
"relevance": {
"p": 0.96,
"sense": "target",
"on_topic": true
}
}
},
{
"title": "Ask HN: How can I make wired Airpods Pro?",
"url": null,
"source": "hackernews",
"container": "Hacker News",
"enrichable": true,
"post": {
"item_id": "29185994",
"source": "hackernews",
"title": "Ask HN: How can I make wired Airpods Pro?",
"body": "I mostly use my Airpods Pro for their Active Noise Cancelling whilst working at my desk for ~8 hours a day.<p>I'm finding their relatively short battery life and gradually deteriorating capacity highly frustrating (and ultimately wasteful).<p>Is there a way to power these in wired mode? Surely something like a USB-C cable can do this?<p>For comparison, I still have and use my original wired EarPods with their 3.5mm jack from back in the day - which work perfectly after many years of heavy use.<p>I wish Apple would release a USB-C wired version of these with Active Noise Cancelling like in their AirPods Pro.<p>It would be helluva practical for those long work sessions and also better for the environment.<p>My next best bet is the Bose QC20 - but even these have a quasi-battery solution which ultimately depletes.",
"url": null,
"author": "user_02ee93",
"published_at": "2021-11-11T09:05:52.000Z",
"date_confidence": "low",
"engagement": {
"points": 15,
"comments": 20
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "",
"metadata": {
"enrichable": true,
"top_comments": [
{
"score": null,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": true,
"author": "user_68d2e3",
"url": "https://example.com/redacted",
"date": "2021-11-11T09:52:04.000Z"
},
{
"score": null,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_765036",
"url": "https://example.com/redacted",
"date": "2021-11-11T11:12:28.000Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"container": "Hacker News",
"local_relevance": 0.6393333333333333,
"freshness": 0,
"engagement_score": 100,
"source_quality": 0.8,
"local_rank_score": 0.5155666666666666
},
"top_comments": [
{
"score": null,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": true,
"author": "user_68d2e3",
"url": "https://example.com/redacted",
"date": "2021-11-11T09:52:04.000Z"
},
{
"score": null,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_765036",
"url": "https://example.com/redacted",
"date": "2021-11-11T11:12:28.000Z"
}
],
"rrf_score": 0.01639344262295082,
"cluster_id": "cluster-1",
"computed": {
"stance": {
"label": "negative",
"confidence": 0.89
},
"relevance": {
"p": 0.63,
"sense": "target",
"on_topic": true
}
}
}
],
"items_by_source": {
"hackernews": {
"items": [
{
"item_id": "29185994",
"source": "hackernews",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"url": "",
"author": "user_02ee93",
"published_at": "2021-11-11T09:05:52.000Z",
"date_confidence": "low",
"engagement": {
"points": 15,
"comments": 20
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"enrichable": true,
"top_comments": [
{
"score": null,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": true,
"author": "user_68d2e3",
"url": "https://example.com/redacted",
"date": "2021-11-11T09:52:04.000Z"
},
{
"score": null,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_765036",
"url": "https://example.com/redacted",
"date": "2021-11-11T11:12:28.000Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"container": "Hacker News",
"local_relevance": 0.6393333333333333,
"freshness": 0,
"engagement_score": 100,
"source_quality": 0.8,
"local_rank_score": 0.5155666666666666
},
{
"item_id": "42610644",
"source": "hackernews",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"url": "https://github.com/user_837162/aln",
"author": "user_aff807",
"published_at": "2025-01-06T14:07:21.000Z",
"date_confidence": "low",
"engagement": {
"points": 3,
"comments": 2
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"enrichable": true,
"top_comments": [
{
"score": null,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_818c06",
"url": "https://example.com/redacted",
"date": "2025-01-06T19:20:01.000Z"
},
{
"score": null,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_e5d0d6",
"url": "https://example.com/redacted",
"date": "2025-01-06T19:39:30.000Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"container": "Hacker News",
"local_relevance": 0.4851478494623655,
"freshness": 0,
"engagement_score": 28,
"source_quality": 0.8,
"local_rank_score": 0.3433461021505376
}
],
"enrichable": true
},
"naver_kin": {
"items": [],
"enrichable": false,
"raw": true
},
"reddit": {
"items": [
{
"item_id": "1w63726",
"source": "reddit",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"url": "https://www.reddit.com/r/AirpodsPro/comments/1w63726/do_airpods_pro_3_really_have_terrible_battery_life/",
"author": "user_79c0e5",
"published_at": "2026-09-03T10:09:52.000Z",
"date_confidence": "low",
"engagement": {
"score": 25,
"num_comments": 54
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"enrichable": true,
"top_comments": [
{
"score": 42,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_5beea1",
"url": "https://example.com/redacted",
"date": "2026-09-03T10:27:53.966Z"
},
{
"score": 16,
"excerpt": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"truncated": false,
"author": "user_51150e",
"url": "https://example.com/redacted",
"date": "2026-09-03T12:36:21.327Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"container": "r/AirpodsPro",
"local_relevance": 0.7964285714285714,
"freshness": 51,
"engagement_score": 64,
"source_quality": 0.6,
"local_rank_score": 0.7091785714285714
},
{
"item_id": "1ws7b1y",
"source": "reddit",
"title": "Sample comment text (redacted).",
"body": "Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted). Sample comment text (redacted).",
"url": "https://www.reddit.com/r/airpods/comments/1ws7b1y/worst_experience_in_my_life_with_airpods_pro_3/",
"author": "user_517fb2",
"published_at": "2026-09-28T06:43:40.000Z",
"date_confidence": "low",
"engagement": {
"score": 0,
"num_comments": 18
},
"relevance_hint": 0.5,
"why_relevant": "",
"snippet": "Sample comment text (redacted).",
"metadata": {
"enrichable": true,
"top_comments": [
{
"score": 11,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_2f89ca",
"url": "https://example.com/redacted",
"date": "2026-09-28T07:00:02.855Z"
},
{
"score": 5,
"excerpt": "Sample comment text (redacted).",
"truncated": false,
"author": "user_000cfd",
"url": "https://example.com/redacted",
"date": "2026-09-28T07:48:17.877Z"
}
]
},
"media": {
"thumbnail_url": null,
"video_url": null,
"duration_s": null,
"image_urls": [],
"aspect_ratio": null,
"author_avatar_url": null,
"favicon_url": null
},
"container": "r/airpods",
"local_relevance": 0.5208333333333333,
"freshness": 84,
"engagement_score": 0,
"source_quality": 0.6,
"local_rank_score": 0.5485416666666666
}
],
"enrichable": true
}
},
"clusters": [
{
"cluster_id": "cluster-1",
"title": "Do Airpods Pro 3 really have terrible battery life?",
"candidate_ids": [
"https://reddit.com/r/airpodspro/comments/1w63726/do_airpods_pro_3_really_have_terrible_battery_life",
"hackernews:29185994"
],
"representative_ids": [
"https://reddit.com/r/airpodspro/comments/1w63726/do_airpods_pro_3_really_have_terrible_battery_life",
"hackernews:29185994"
],
"sources": [
"hackernews",
"reddit"
],
"score": 16.393442622950822,
"uncertainty": "thin-evidence"
},
{
"cluster_id": "cluster-3",
"title": "Worst experience in my life with AirPods Pro 3",
"candidate_ids": [
"https://reddit.com/r/airpods/comments/1ws7b1y/worst_experience_in_my_life_with_airpods_pro_3"
],
"representative_ids": [
"https://reddit.com/r/airpods/comments/1ws7b1y/worst_experience_in_my_life_with_airpods_pro_3"
],
"sources": [
"reddit"
],
"score": 16.129032258064516,
"uncertainty": "single-source"
}
],
"computed": {
"question_share": 0.5,
"question_share_method": "Sample comment text (redacted).",
"stance_split": {
"positive": 0,
"negative": 1,
"neutral": 0
},
"stance_coverage": {
"threads": 10,
"counted": 4,
"no_opinion": 3,
"uncertain": 3,
"unjudged": 0
},
"top_communities": [
{
"container": "r/airpods",
"thread_count": 5
},
{
"container": "Hacker News",
"thread_count": 3
}
]
},
"legs": [
{
"endpoint": "/v1/hackernews/search",
"status": 200,
"credits_used": 1,
"latency_ms": 576,
"error": null
},
{
"endpoint": "/v1/naver/kin/search",
"status": 200,
"credits_used": 0,
"latency_ms": 1713,
"error": null
}
],
"sources_called": [
"hackernews",
"naver_kin"
],
"sources_succeeded": [
"hackernews",
"naver_kin"
],
"coverage": 1,
"partial_failure": false,
"methodology_version": "search-forums/1.1",
"relevance": {
"mode": "default",
"threshold": 0.5,
"threads": 10,
"scored": 10,
"kept": 10,
"dropped": 0,
"dropped_ids": [],
"unjudged": 0,
"too_short": 0,
"reordered": false,
"methodology_version": "search-forums-relevance/1"
},
"dropped": 0
},
"credits_used": 10,
"request_id": "req_example000000",
"cached": false,
"pagination": {
"next_cursor": null,
"has_more": false,
"page_size": 10
},
"credits_remaining": 9999,
"source": "captured",
"captured_at": "2026-10-02T12:48:25.920Z",
"redacted": true
}Live sample illustrating the unified response shape. Field values reflect the record you query.
How does the Universal Search Forums API work?
Send a GET request with your API key and get back clean, structured JSON in our unified schema. Supported computed fields are populated when the source provides the required inputs.
Method
GET
Response
JSON
How do you scrape social media data in seconds?
The fastest social media scraping API for developers. Scrape profiles, posts, comments, and analytics from 68 platforms covering 10B+ monthly active users.
One schema, every platform
Query 68 platforms with identical response structures. Write your integration once.
Computed fields, not just scraped
When an endpoint supports these metrics and the source provides the required inputs, the normalized record includes engagement_rate, estimated_reach, content_category, and language. Ready to use.
See your data before you code
Visual Data Explorer. Paste any URL, get rich result cards, sortable tables, CSV export.
import requests
response = requests.get(
'https://www.socialcrawl.dev/v1/tiktok/profile',
params={'handle': 'charlidamelio'},
headers={'x-api-key': 'sc_YOUR_API_KEY'}
)
data = response.json(){
"success": true,
"platform": "tiktok",
"data": {
"author": {
"username": "charlidamelio",
"followers": 152400000
},
"engagement": {
"likes": 12400000000,
"engagement_rate": 0.087
},
"metadata": {
"language": "en",
"content_category": "lifestyle"
}
}
}Have a question? We got answers
Find answers to frequently asked questions about SocialCrawl's API, pricing, and capabilities.
Contact usWhat is the SocialCrawl forum search API?
Which forums does it search?
How much does a forum search cost?
Can I control the time window and comments?
How is this different from calling each forum separately?
Ask AI about SocialCrawl
Ready to scrape Universal Search Forums data?
Get your API key and start pulling Universal Search data in under 60 seconds.
