# Cohorts (/docs/cohorts) Cohorts [#cohorts] Most social listening answers "who is talking about this brand?" Cohorts answers a narrower and usually more valuable question: **"which of *these specific people* are talking about it?"** You bring the list. Upload up to 10,000 platform-qualified public identities, submit a keyword query bounded to a recent window, and get back the matching posts per member — plus an honest coverage record for every member, including the ones that matched nothing. This is the endpoint to use when you have already decided who matters (a purchaser panel, a customer list, a creator roster, a set of tracked competitors' employees) and you want their public social activity, not the open firehose. Cohorts is **not** audience discovery. It does not find people, infer demographics, or score interests. It takes a list you already have and tells you what those public accounts posted. What SocialCrawl receives [#what-socialcrawl-receives] Only two things per member: * `platform` + `handle` — the public identity (handle, profile URL, or channel ID) * `external_id` — your own opaque key, echoed back on every match so you can join results to your own records Whatever you used to build the list — purchase receipts, CRM segments, panel attributes, demographics — stays on your side. SocialCrawl never receives it and has no field to put it in. Both the identity and your `external_id` are encrypted at rest with a key derived per account. `GET /v1/cohorts/{cohortId}` returns counts and metadata; it will not read the identities back to you, and they never appear in logs, job payloads, or error messages. Supported platforms [#supported-platforms] `bluesky` · `instagram` · `kwai` · `linkedin` · `threads` · `tiktok` · `truth-social` · `twitch` · `twitter` · `youtube` Anything else is rejected at upload with `400 COHORT_IDENTITY_PLATFORM_UNSUPPORTED`, so an unsupported identity can never silently cost you a query that returns nothing. The lifecycle [#the-lifecycle] Four calls. Everything except the query itself costs 0 credits. | Step | Call | Cost | | ---- | ------------------------------------------------------- | ------- | | 1 | `POST /v1/cohorts` | 0 | | 2 | `PUT /v1/cohorts/{cohortId}/members` (repeat per 1,000) | 0 | | 3 | `POST /v1/cohorts/{cohortId}/queries` → `202` | metered | | 4 | `GET /v1/cohort-queries/{queryId}` then `.../results` | 0 | Every mutating call requires an `Idempotency-Key` UUID header. Replaying the same key with the same body returns the original resource; replaying it with a different body returns `409`. 1\. Create the cohort [#1-create-the-cohort] ```bash curl -X POST "https://www.socialcrawl.dev/v1/cohorts" \ -H "x-api-key: sc_your_api_key_here" \ -H "Idempotency-Key: 11111111-1111-4111-8111-111111111111" \ -H "content-type: application/json" \ -d '{"name":"August purchaser panel","retention_days":30}' ``` `retention_days` is 7–90 and defaults to 30. When it elapses, the cohort and everything under it is purged automatically. Uploading members or submitting a query renews the clock. 2\. Upload members [#2-upload-members] Up to 1,000 per call, 10,000 per cohort. Send as many chunks as you need. ```bash curl -X PUT "https://www.socialcrawl.dev/v1/cohorts/{cohortId}/members" \ -H "x-api-key: sc_your_api_key_here" \ -H "Idempotency-Key: 22222222-2222-4222-8222-222222222222" \ -H "content-type: application/json" \ -d '{ "members": [ { "external_id": "buyer_01983", "platform": "instagram", "handle": "natgeo" }, { "external_id": "buyer_04711", "platform": "youtube", "handle": "UCxxxxxxxxxxxxxxxxxxxxxx" }, { "external_id": "buyer_04712", "platform": "linkedin", "handle": "https://www.linkedin.com/in/someone" } ] }' ``` The response reports `inserted`, `updated`, `unchanged`, and the new `member_count`. Re-sending an `external_id` you have already uploaded **updates** its identity rather than adding a row, so a nightly full re-push is safe and does not inflate the count. Two different `external_id`s cannot claim the same normalized identity inside one cohort — that returns `409 COHORT_IDENTITY_CONFLICT` rather than quietly double-counting one person. 3\. Submit a query [#3-submit-a-query] ```bash curl -X POST "https://www.socialcrawl.dev/v1/cohorts/{cohortId}/queries" \ -H "x-api-key: sc_your_api_key_here" \ -H "Idempotency-Key: 33333333-3333-4333-8333-333333333333" \ -H "content-type: application/json" \ -d '{ "keywords": ["acme", "acme pro"], "date_from": "2026-08-01T00:00:00.000Z", "max_pages_per_identity": 3, "max_items_per_identity": 100, "max_credits": 12000 }' ``` `date_from` is a **full RFC3339 timestamp**, not a bare calendar date. `platforms` is optional — omit it to query every platform present in the cohort, or pass a subset to run one platform at a time. The call returns `202` immediately: ```json { "query_id": "cqy_3nT7hV2Wq9Ls5Mb8Kd1Rx", "status": "queued", "member_count": 10000, "shard_count": 200, "reserved_credits": 8000, "estimated_credits": 8000, "status_url": "/v1/cohort-queries/cqy_3nT7hV2Wq9Ls5Mb8Kd1Rx", "result_url": "/v1/cohort-queries/cqy_3nT7hV2Wq9Ls5Mb8Kd1Rx/results" } ``` 4\. Poll, then read results [#4-poll-then-read-results] ```bash curl "https://www.socialcrawl.dev/v1/cohort-queries/{queryId}" \ -H "x-api-key: sc_your_api_key_here" ``` `status` moves through `queued` → `running` → `succeeded` (or `failed`, `cancelled`, `expired`). `progress` carries durable counters — `members_completed`, `shards_completed`, `pages_succeeded`, `pages_failed` — so a long run is observable rather than opaque. Once terminal, read the matches: ```bash curl "https://www.socialcrawl.dev/v1/cohort-queries/{queryId}/results?limit=100" \ -H "x-api-key: sc_your_api_key_here" ``` ```json { "items": [ { "external_id": "buyer_01983", "platform": "instagram", "content_id": "3401...", "canonical_url": "https://www.instagram.com/p/...", "published_at": "2026-08-14T09:12:44.000Z", "text_excerpt": "finally switched to acme pro and ...", "matched_keywords": ["acme", "acme pro"], "route": "profile/posts", "retrieved_at": "2026-08-20T11:02:03.000Z" } ], "coverage": [ { "external_id": "buyer_04711", "platform": "youtube", "status": "complete", "pages": 2, "oldest_seen": "2026-07-29T00:00:00.000Z", "window_complete": true, "route_errors": [] } ], "next_cursor": "eyJ..." } ``` Page with `next_cursor` until it is `null`. Pages are additionally bounded so the serialized body never exceeds 1 MB — a page may come back smaller than your `limit` for that reason, which is not the end of the result set. Coverage is the important field [#coverage-is-the-important-field] `coverage` has one record per member, whether or not it matched. This is what stops a partial crawl from reading as "nobody talked about you". * `window_complete: true` — the route reached your `date_from` boundary or the account's end of feed. What you got back is everything in the window. * `window_complete: false` — the page budget ran out, the account was unreachable, or the route errored. There may be more posts you did not see. * `oldest_seen` — the oldest post observed for that member, even when nothing matched. It tells you how far back the crawl actually got. * `route_errors` — sanitized per-route failure codes for that member. A query can be `succeeded` with `pages_failed > 0`. Read `coverage` before treating a result set as exhaustive; raise `max_pages_per_identity` and re-run if you need to reach the boundary. Matching is deterministic [#matching-is-deterministic] Keyword matching is code, not a model. Both the keyword and the candidate text are normalized with Unicode NFKC, case-folded, and whitespace-collapsed, then matched as a literal substring. `matched_keywords` echoes the keywords you submitted. There is no stemming, fuzzy matching, semantic expansion, sentiment scoring, or brand-alias inference. If you want "Acme" to also catch "AcmeCo", pass both. Only content the source attributes to the identity you supplied is eligible. Profile lookups are used to resolve the account and its feed cursor; they never become results. Credits [#credits] Every lifecycle call — create, upload, status, cancel, delete, results — costs **0 credits**. A query reserves its worst-case ceiling at submission: ```text ceiling = Σ(member × route page cap × credits per page) ``` Most platforms cost **1 credit per successful upstream page**; LinkedIn costs **5**. Instagram and YouTube each run two routes per member, so they cost 2 per page-round. You are then charged **only for pages that actually succeeded**. Failed, timed-out, cancelled, and skipped pages cost nothing. When the query reaches a terminal state the unspent reservation is refunded exactly once, so `actual_credits + refunded_credits` always equals `reserved_credits`. `max_credits` is your own safety limit. If the computed ceiling exceeds it, submission fails with `400` before any work is created or any credit is held. It is never permission to spend beyond the ceiling. Cancelling and deleting [#cancelling-and-deleting] `DELETE /v1/cohort-queries/{queryId}` stops a query. Queued work never starts; running work stops at the next page boundary. Pages already fetched stay billable, and the unspent reservation is refunded once. `DELETE /v1/cohorts/{cohortId}` removes the cohort and cascades its members, queries, and results, cancelling anything still in flight. Your credit-ledger receipts are never deleted — billing history survives the data. Errors [#errors] | Code | Status | Meaning | | -------------------------------------- | ------ | ----------------------------------------------------------------- | | `COHORT_MEMBER_LIMIT_EXCEEDED` | 400 | The upload would push the cohort past 10,000 members | | `COHORT_IDENTITY_PLATFORM_UNSUPPORTED` | 400 | That platform is not supported for cohort queries | | `COHORT_IDENTITY_CONFLICT` | 409 | Another `external_id` already claims that identity in this cohort | | `COHORT_QUERY_NOT_READY` | 409 | The query has not reached a terminal state yet | | `COHORT_QUERY_NOT_CANCELLABLE` | 409 | The query is already terminal | | `COHORT_RESULT_TOO_LARGE` | 413 | A single stored result cannot fit under the 1 MB page ceiling | | `INSUFFICIENT_CREDITS` | 402 | The reservation ceiling exceeds your available balance | Any cohort or query that is not yours returns `404` — never `403`. A cross-tenant probe is indistinguishable from a resource that does not exist. Retention and privacy [#retention-and-privacy] * Cohort membership is purged after `retention_days` (7–90, default 30). * Query results expire no later than their parent cohort, and 30 days after the query goes terminal if that comes sooner. * Identities and external IDs are encrypted at rest and never appear in logs, job payloads, traces, or error responses. * Only matched content and coverage are stored — not the unmatched feeds the workers paged through. * Your GDPR data export includes cohort metadata, query accounting, and your own `external_id`s. It deliberately excludes the stored platform identities and matched post content. You remain the controller for how the panel was selected and for the lawful basis of that selection. SocialCrawl processes the public identifiers and public content you submit, for the job you submitted.