# Labels (/docs/labels) SocialCrawl labels on comments, posts, reviews and search results. On by default, free, and switched off with one parameter. Comment, post, review and search lists come back with SocialCrawl labels attached: how a comment feels, whether a post is a paid promotion, which problem a review reports, whether a search result is about what you searched for. They are on by default, they add no credits, and every field you already read stays exactly as it was. ```bash title="cURL" curl "https://www.socialcrawl.dev/v1/tiktok/post/comments?url=https://www.tiktok.com/@stoolpresidente/video/7623818255903329566" \ -H "x-api-key: YOUR_API_KEY" ``` ```json title="Response" { "success": true, "platform": "tiktok", "endpoint": "/v1/tiktok/post/comments", "data": { "items": [ { "comment": { "text": "Ordering it for my 13 year old ❤️" }, "computed": { "language": "en", "labels": { "sentiment": { "level": 4, "score_0_1": 0.91, "confidence": 0.71 }, "question": { "p": 0.03 }, "purchase_intent": { "p": 0.98 }, "complaint": { "p": 0.02 } } } } ], "labels": { "presets": ["sentiment", "question", "purchase_intent", "complaint"], "mode": "default", "status": "complete", "rows": 50, "labelled": 50, "cached": 0, "skipped": 0, "unjudged": 0, "pending": 0, "pending_ids": [], "dropped": 0, "dropped_ids": [], "extra_credits": 0, "methodology_version": "labels/1" } }, "credits_used": 1, "cached": false } ``` The response is trimmed to one comment, and the labels shown are the ones that comment received on a recorded run. The page cost 1 credit, the same as it did before labels existed. Each comment gained `computed.labels`, and `data.labels` reports how many rows were judged, how many are still waiting, and what the labels cost you (nothing: `extra_credits: 0`). A judgment is a model's answer to one narrow question about one row ("is this comment asking a question?"), published as a probability from 0 to 1 or as a level, so you choose the cut-off. Code does everything else: which rows are worth sending, the counting, the drops you ask for, and the credits. Labels are signals to read, never a verdict on a post, a review or a person. ## What each list gets by default | List | Default labels, free | Where they land | | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | Comment lists | `sentiment`, `question`, `purchase_intent`, `complaint` | `items[].computed.labels`, reported in `data.labels` | | Post lists | `sponsored`, `intent` (without `fits_offer`), `niche` | `items[].computed.labels`, reported in `data.labels` | | Post search lists | relevance against your `query`, as a score: nothing is dropped or reordered | `items[].computed.relevance`, reported in `data.relevance` | | Review lists | `sentiment` (with `rating_mismatch`), `issue` | `items[].computed.labels`, reported in `data.labels` | | [`/v1/search/everywhere`](/docs/search/everywhere.md) | stance toward your query, and relevance from the ranking step | `items[].computed.stance`, `items[].computed.relevance`, `data.stance_split`, `data.relevance` | | [`/v1/search/forums`](/docs/search.md#forums-and-creators) | relevance of each thread to your query | `items[].computed.relevance`, `data.relevance` | | [`/v1/search/news`](/docs/search/news.md) | articles about the same event grouped into stories | `data.stories`, `data.story_grouping` | | [`/v1/search/creators`](/docs/search.md#forums-and-creators) | what kind of account each creator is, for the first 40 | `data.creators[].account`, `data.account_kinds` | Prism composites already judged their rows before this change; see [Prism](/docs/prism.md). **Comment lists:** `/v1/tiktok/post/comments`, `/v1/tiktok/video/comment/replies`, `/v1/instagram/post/comments`, `/v1/instagram/post/comment/replies`, `/v1/youtube/video/comments`, `/v1/youtube/video/comment/replies`, `/v1/reddit/post/comments`, `/v1/facebook/post/comments`, `/v1/facebook/post/comment/replies`, `/v1/twitter/tweet/replies`. **Post lists:** `/v1/instagram/profile/posts`, `/v1/instagram/profile/reels`, `/v1/instagram/search/reels`, `/v1/tiktok/profile/videos`, `/v1/tiktok/search/hashtag`, `/v1/twitter/user/tweets`, `/v1/twitter/search/tweets`, `/v1/reddit/search`, `/v1/reddit/subreddit`, `/v1/facebook/search/posts`, `/v1/hackernews/search`, and the four search lists below that walk several pages (`/v1/tiktok/search`, `/v1/youtube/search`, `/v1/threads/search`, `/v1/linkedin/search/posts`). **Post search lists (relevance):** `/v1/instagram/search/reels`, `/v1/twitter/search/tweets`, `/v1/hackernews/search`, `/v1/facebook/search/posts`, `/v1/reddit/search`, `/v1/tiktok/search`, `/v1/youtube/search`, `/v1/threads/search`, `/v1/linkedin/search/posts`. **Review lists:** `/v1/amazon/reviews`, `/v1/walmart/reviews`, `/v1/target/reviews`, `/v1/wayfair/reviews`, `/v1/home_depot/reviews`, `/v1/sephora/reviews`, `/v1/aliexpress/reviews`, `/v1/kohls/reviews`, `/v1/klarna/reviews`, `/v1/klarna/reviews/pro`, `/v1/google_shopping/reviews`, `/v1/tiktokshop/product/reviews`, `/v1/trustpilot/reviews`, `/v1/g2/reviews`, `/v1/yelp/business/reviews`, `/v1/google/business/extended-reviews`, `/v1/tripadvisor/reviews`, `/v1/tripadvisor/restaurant/reviews`, `/v1/tripadvisor/attraction/reviews`, `/v1/tripadvisor/cruise/reviews`, `/v1/google_play/app-reviews`, `/v1/app_store/app-reviews`. Every other endpoint answers exactly as before and has no `judgments` parameter. ## Field shapes Every label is a key under `computed.labels`, named after its preset. A preset key that is missing means that preset has not judged the row yet. ### Comments | Label | Value | Meaning | | ----------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | | `sentiment` | `{ level, score_0_1, confidence }` | How the comment feels, on five levels from 0 (very negative) to 4 (very positive); `score_0_1` is the same on a 0 to 1 scale | | `question` | `{ p }` | A real question. A rhetorical one that complains or mocks is not | | `purchase_intent` | `{ p }` | Bought it, ordered it, is switching to it, or asks for a link or a price. "Is it worth it?" is not | | `complaint` | `{ p }` | Something that happened to the author. Criticism of a price or a spec is not | ### Posts | Label | Value | Meaning | | ----------- | --------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `sponsored` | `{ p, disclosed, undisclosed, brand }` | Is it a paid or gifted promotion (`p`), did it carry a marker such as #ad or 광고 (`disclosed`), is it likely paid with no marker (`undisclosed`), and which mentioned account it promotes (`brand`, or `null`) | | `intent` | `{ label, confidence, buyer, seller, urgency, fits_offer }` | What the author is mainly doing (`asking_for_recommendation`, `comparing_options`, `switching_away`, `complaining`, `promoting`, `news_or_discussion`, `other`, or `null` when unsure), whether they read as a buyer, how pressing the need is (0 to 3). `fits_offer` is `null` unless you pass `offer=` | | `niche` | `{ label, confidence, taxonomy }` | Which niche of the published `sc-niche-v1` taxonomy the caption belongs to, or `personal_no_niche` or `other`. `label` is `null` when the caption is too thin to tell. It reads the caption only, not the video | ### Post search relevance `items[].computed.relevance` is `{ p, sense, depth, spam }`. `p` is the probability that the row is about what your query means. `sense` says which thing the query's key word refers to in the row: `target` (what you meant), `other_named_thing` (a different brand, person or place with the same name), `generic_word`, or `not_present`. `depth` runs from 0 (not about it) to 3 (mainly about it), and `spam` is the probability that the row is a mass-posted template. A row with too little text to judge carries `relevance: null`. ### Reviews | Label | Value | Meaning | | ----------- | ---------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `sentiment` | `{ level, score_0_1, confidence, rating_mismatch }` | How the reviewer feels, read from the title and text only, never the stars. `rating_mismatch` is `true` when the words clearly contradict the star rating, and `null` when the review has no rating | | `issue` | `{ label, confidence }` | The main problem the review reports: `product_defect`, `sizing_or_fit`, `shipping_or_delivery`, `customer_service`, `price_or_value`, `missing_feature`, `other`, or `none`. `label` is `null` when unsure | ### Search endpoints | Endpoint | Per row | Page report | | ------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | | `search/everywhere` | `computed.stance`: `{ label, probabilities, confidence }`, with `label` one of `positive`, `negative`, `neutral`, `no_opinion`, or `null` under the confidence floor. `computed.relevance`: `{ score, on_topic, entity }` | `data.stance_split` counts stances overall and per platform with the ids behind every count; `data.relevance` counts off-topic results (`off_topic`, `off_topic_ids`) | | `search/forums` | `computed.relevance`: `{ p, sense, on_topic }` | `data.relevance` with `mode: "default"` and `reordered: false` | | `search/news` | none: `items[]` is unchanged | `data.stories[]`: `{ id, head_id, member_ids, size, outlets, outlet_count, countries, languages, first_seen, last_seen, judged }`, and `data.story_grouping` | | `search/creators` | `account`: `{ account_kind, account_kind_confidence, named_person }`, with `account_kind` one of `individual_creator`, `brand_or_business`, `media_or_publisher`, `fan_or_repost_page`, `cannot_tell`, or `null` when unsure | `data.account_kinds`: `{ status, judged, unjudged, methodology_version }` | On `search/everywhere`, `on_topic: false` marks exactly the results `relevance=filter` would drop. On `search/creators`, `individual_creator` is published only when the name, handle or bio shows one particular person; otherwise the creator reads `cannot_tell`. ## `null` and `pending` A row whose labels are `null` was not judged in this response. That is different from a low probability: `{ "p": 0.02 }` means "judged, and no", `null` means "not judged yet". A label that could not be judged never drops a row and is never billed. Default labels are judged under a short time budget so they never slow your page down. On list endpoints, rows not finished in time ship with `labels: null` (or `relevance: null`), are counted in `data.labels.pending` and listed in `pending_ids`, and keep being judged after the response is sent. The result goes into a shared per-row store, so your next call for the same page, including a free cache hit, carries them. In practice a cold comment page usually arrives fully labelled; a cold post or review page often arrives mostly pending and is complete from the next call. At most 60 cold rows are started per request, so a longer page can take more than one call to fill; rows past that count are `unjudged`, not `pending`. `data.labels.status` sums it up: | `status` | Meaning | | ---------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `complete` | Every row that can be judged carries its labels. | | `partial` | Some rows are `null`: still being judged (`pending`) or not judged this time (`unjudged`). | | `skipped` | Judging was switched off for this request because capacity was short. No row is touched, not even with labels judged earlier, so the page is exactly the `judgments=off` page plus the report. | Need every row labelled in this response? Ask for the labels by name: `label=sentiment,question,purchase_intent,complaint` is still free, and a label you ask for is judged in the foreground under a longer time budget. `relevance=score` does the same for search relevance. The four search endpoints judge inside the request and keep no per-row store. A result not judged in time stays `null` for the life of that cached response (2 minutes, 5 minutes on `search/news`). There, a late or failed judgment is also named in `data._warnings`: `stance_partial`, `stance_unavailable` or `stance_skipped` on `search/everywhere`, `stories_pending` or `stories_unavailable` on `search/news`. `search/forums` and `search/creators` report it in `status` only. ## The report `data.labels` describes the labels on this page: | Field | Meaning | | ------------------------------ | ----------------------------------------------------------------------------------------------------------------------- | | `presets` | The labels this report covers. | | `mode` | `default` when you passed no `label=`, `requested` when you did. | | `status` | `complete`, `partial` or `skipped`, as above. | | `rows` | Rows on the page before any drop. | | `labelled`, `cached` | Rows that carry every label, and how many of those came from the shared per-row store. | | `pending`, `pending_ids` | Rows still being judged when the page was sent. | | `unjudged`, `skipped` | Rows not judged this time, and deleted or empty rows that are never sent or billed. | | `dropped`, `dropped_ids` | Rows removed by an `exclude=` you asked for. Default labels never drop a row. | | `extra_credits` | What labels added to this call. Always `0` for default labels. | | `methodology_version` | The version of the label questions, so you can tell when they change. | | `default` | Present when you asked for some labels by name: the report of the default labels you did not name, with its own `mode`, `status`, `pending` and `pending_ids`. Absent when you named every default. | `data.relevance` on the post search lists has the same `status`, `pending` and `pending_ids`, plus `origin` (`default` or `requested`), because its `mode` field already says `score` or `filter`. ## Turn them off ```bash title="cURL" curl "https://www.socialcrawl.dev/v1/tiktok/post/comments?url=https://www.tiktok.com/@stoolpresidente/video/7623818255903329566&judgments=off" \ -H "x-api-key: YOUR_API_KEY" ``` `judgments=off` returns the page exactly as it was before default labels: no `computed.labels`, no `computed.relevance` or `computed.stance`, no `data.labels`, `data.relevance`, `data.stance_split`, `data.stories` or `account` blocks. `label=none` does the same on list endpoints, also inside a list such as `label=mention,none`. It is part of the cache key, so an `off` request never receives a labelled cached page. It does not cancel a `label=` or `relevance=` you pass in the same request: those still run. ## Ask for more: `label=` adds to the defaults Naming labels adds them to the defaults rather than replacing them: ```bash title="cURL" curl "https://www.socialcrawl.dev/v1/instagram/search/reels?query=buldak+noodles&label=mention&brand=Buldak" \ -H "x-api-key: YOUR_API_KEY" ``` Each reel keeps `sponsored`, `intent`, `niche` and its relevance score, and gains `mention`: is the post about Buldak rather than something that shares the name, how it feels about the brand, which aspect it talks about, and whether the author bought or used it. `data.labels` reports the labels you named (`mode: "requested"`), and `data.labels.default` reports the default ones. A row can carry the labels you named now and the default ones on the next call. ## What stays opt-in and metered Labels that need your own input, and labels without a blind measurement yet, run only when you ask for them. | Ask with | On | Adds | | ------------------------------------------- | ------------------------- | -------------------------------------------------------------------------------------------------------- | | `label=mention&brand=` | post lists | `mention`: about the brand or not, sentiment toward it, sarcasm, aspect, first-hand use | | `label=intent&offer=` | post lists | `intent.fits_offer`: would your offer plausibly help this author | | `label=spam`, `label=toxic`, `label=low_quality` | comment lists | `{ p }` each (`low_quality` also `generic` and `bait`); `exclude=spam` or `exclude=low_quality` drops matches at 0.8 or higher | | `label=reports&reports=` | review lists | `reports.p`: does the review say the thing you describe, in any wording or language | | `label=incentivized` | review lists | `{ p, disclosed }`: does the reviewer say they got the product free, discounted or rewarded | | `relevance=score` or `filter` with `relevant_to=` | post search lists | relevance against a topic you write, instead of the query | | `brief=` | `search/creators` | `brief_match` per creator and a re-ranked list: see [Universal Search](/docs/search.md#forums-and-creators) | On the lists, a metered label holds a few extra credits up front (4 on most lists, 5 on `tiktok/search`, 8 on `linkedin/search/posts`) and refunds down to **1 credit per started 25 rows newly judged** on this request. Rows already judged for anyone are free, a page where nothing could be judged refunds the whole hold, and a cached page is free. `brief=` on `search/creators` adds 2 credits. Asking by name for a label that is also a default one, such as `label=sponsored`, stays free. ## Filtering is free `relevance=filter` drops the rows that are not about your query and lists their ids in `data.relevance.dropped_ids`. It is free with your query as the topic. ```bash title="cURL" curl "https://www.socialcrawl.dev/v1/instagram/search/reels?query=mustang+horse&relevance=filter" \ -H "x-api-key: YOUR_API_KEY" ``` - **Post search lists:** a row is kept at `p` of 0.5 or more, or the `relevance_threshold` you set. Pagination is unchanged, so a filtered page can hold fewer rows. - **`search/everywhere`:** drops the ranked results the ranking step already judged off-topic, inside the flat 20 credits. `items_by_source` is not filtered. - **`search/forums`:** `relevance=score` reorders threads by relevance blended with their rank, and `filter` also drops the off-topic ones, inside the flat 10 credits. - **`search/creators`:** `relevance=filter` works only with `brief=` and drops creators that do not fit it. A row that could not be judged is never dropped. ## Credits and caching Default labels never add credits, holds or refunds: a call costs exactly what the same call costs with `judgments=off`. A response cache hit is still 0 credits and still labelled, because labels are read from a shared per-row store that lasts 30 days and is keyed by the row's text, so an edited comment is judged again. See [Credits](/docs/credits.md#labels-add-no-credits) and [Caching](/docs/caching.md#labels-on-a-cache-hit). ## How we measure [#how-we-measure] A label is on by default only when it has a blind measurement: rows labelled by hand before any judgment ran on them, on queries and sources never used to write or tune the questions. Every number below comes from one labeller, not a panel, so read it as a strong signal rather than a guarantee, and keep a person in the loop for decisions that matter. | Judgment | Blind set | Result | | ------------------------------------------ | ------------------------------------------------ | ------------------------------------------------------------------------------------------------ | | Post `sponsored` | 39 posts, 15 Korean | precision 1.000, recall 0.870 | | Post `intent`, the `buyer` flag | two sets, 87 and 108 posts | precision 0.903 and 0.852, recall 0.903 and 0.885. Korean on the second set 0.667: treat Korean `buyer` as a hint | | Post `niche` | 80 posts in English, Korean and German | 75 of 75 published labels right, 6% left `null` | | Review `sentiment` | 101 reviews from 8 sources, 38 Korean | 0.960 agreement, against 0.930 for the star rating | | Review `issue` | the same 101 reviews | 0.965 on the 85 reviews it labels | | Post search relevance, `relevance=filter` | 233 rows from 15 queries, 42 Korean | precision 0.968 (0.824 unfiltered), recall 0.943 | | `search/everywhere` stance | 64 posts, English and Korean | 0.826 agreement on the posts it labels; 72% of posts get a label | | `search/forums` relevance, `relevance=filter` | 168 threads, 140 Korean | Korean precision 0.955 (0.921 unfiltered) at recall 0.992; English 0.923 (0.630 unfiltered) at recall 0.706 | | `search/news` stories | 100 articles, 1,047 article pairs | pairwise precision 0.603, recall 0.915: the weakest default, which tends to join an event's preview with its outcome | | `search/creators` account kind | 240 creators | 0.843 accuracy; published `individual_creator` precision 0.917 | Two Prism composites that judge rows on request were measured the same way: | Judgment | Blind set | Result | | ------------------------------------------ | ------------------------------------------------ | ------------------------------------------------------------------------------------------------ | | `prism/adverse-screen`, any flag | 166 posts, 55 Korean, 69 of them near-misses such as news and scam warnings | precision 0.852, recall 0.927; 6 of 69 near-misses flagged (Korean 4 of 31). `sanctions_or_terror_links` and `sexual_misconduct` had no positives and are unmeasured | | `prism/comment-leads`, the lead rule | 236 flagged comments | 0.962 show buyer intent; 0.792 show buyer intent for the offer's category | The default comment labels were measured on held-out rows of their tuning set rather than on a separate blind set, so their numbers are not quoted here. `spam`, `toxic` and `incentivized` have too few real positives in any set to measure, which is why they stay opt-in. ## Next steps - [Computed fields](/docs/computed-fields.md): The code-derived fields that sit next to the labels. - [Response schema](/docs/response-schema.md#data-level-reports): Where data.labels, data.relevance and the other reports sit. - [Sentiment analysis recipe](/docs/recipes/sentiment-analysis.md): Comment sentiment across platforms, with and without your own model. - [Universal Search](/docs/search.md): Stance, forum relevance, news stories and creator account kinds.