# Universal news search (/docs/search/news)



`GET /v1/search/news` [#get-v1searchnews]

Send one query and get back a planned, localized, multi-country news search in a single call. An LLM plans your query into search angles and localizes each angle into the language of every requested country edition (one combined call, fail-open), the lane fans out up to 12 parallel legs against Google News (50 supported country editions), and the articles merge into one deduplicated list. Every leg reports truthful provenance: `query_source` tells you whether its keyword was `translated`, confirmed `original`, or a `fallback_original` after a localization failure. Returns one JSON envelope by default, or a live SSE stream when you ask for one.

```bash
curl 'https://www.socialcrawl.dev/v1/search/news?query=samsung+galaxy+launch&countries=KR,US,JP' \
  -H 'x-api-key: sc_...'
```

**Credit cost:** metered, **2-14 credits** per call: a 2-credit base covers planning and localization, then 1 credit per leg that returned at least one article. The upfront hold is `2 + min(5 x countries, max_legs, 12)` and the unused remainder is refunded automatically, so the `credits_used` you see in the response is what you actually paid. See [Billing](#billing) and [Endpoint pricing](/docs/endpoint-pricing.md).

**Auth:** `x-api-key` header, same as every other endpoint.

**Method:** `GET`, all inputs as query params.

***

Request parameters [#request-parameters]

| Param        | Required | Type                                 | Default | Description                                                                                                                                                            |
| ------------ | -------- | ------------------------------------ | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `query`      | yes      | string (1 to 500 chars)              | none    | News topic or question. Boolean AND/OR/NOT and quoted phrases are supported; Google advanced operators (`site:`, `intitle:`, `before:`) are rejected with a 400.       |
| `countries`  | no       | CSV of ISO alpha-2 codes             | `US`    | 1 to 12 of the 50 supported country editions (e.g. `KR,US,JP`). Each country searches its Google News edition in that edition's default language; duplicates collapse. |
| `time_range` | no       | `day` \| `week` \| `month` \| `year` | `day`   | Recency window. Ignored when `from`/`to` is present (explicit dates win).                                                                                              |
| `from`       | no       | `YYYY-MM-DD` or Unix                 | none    | Lower publish-date bound. Pins every leg to the primary news source (see below).                                                                                       |
| `to`         | no       | `YYYY-MM-DD` or Unix                 | none    | Upper publish-date bound. Pins every leg to the primary news source.                                                                                                   |
| `publisher`  | no       | bare domain                          | none    | Publisher filter, e.g. `bbc.com` (no scheme, no path). Pins every leg to the primary news source.                                                                      |
| `depth`      | no       | integer 10-100, steps of 10          | `10`    | Articles requested per leg.                                                                                                                                            |
| `max_legs`   | no       | integer 1-12                         | `12`    | Hard cap on billable legs. The upfront hold shrinks with it, so it is your cost-control knob.                                                                          |

`publisher`, `from`, and `to` pin every leg to the primary Google News source: the fallback source cannot express those filters, and silently dropping a filter would return unfiltered data. Pinned requests trade away the fallback's empty-result rescue for filter correctness.

***

How a request becomes legs [#how-a-request-becomes-legs]

1. **Plan.** One combined LLM call turns your query into up to 5 weighted search angles and localizes each angle into the default language of every requested country. The call is fail-open: on timeout or error the lane degrades to a single identity angle (your query, clipped to a keyword) and every leg is stamped `fallback_original`, never silently.
2. **Expand.** Angles cross with countries into legs, capped at `min(5 x countries, max_legs, 12)`. Identical (country, language, keyword) legs collapse before dispatch, so two angles that localize to the same keyword never bill twice.
3. **Fan out.** Every leg runs in parallel as an in-process `google_news/search` call (primary source with automatic fallback and empty-result rescue, unless the request is pinned).
4. **Assemble.** Articles dedupe across legs on `canonical_url ?? url`; the first leg's copy wins the body and every later leg that returned the same article is appended to that item's `legs_returned` provenance list.

The `query_source` vocabulary on every leg is part of the public contract:

* `translated`: the localizer produced a different keyword for that language.
* `original`: the localizer confirmed the planned keyword is already correct for that language. This value is never stamped on a failure path.
* `fallback_original`: localization was unavailable for that language (call failed, timed out, or the language was missing from the result), so the leg searched the planned keyword as-is. A total localizer outage is therefore always distinguishable from "keyword already correct".

***

What you get back (JSON) [#what-you-get-back-json]

By default (`Accept: application/json`) you get one envelope. `data` carries:

| Field                 | Type   | Description                                                                                                                       |
| --------------------- | ------ | --------------------------------------------------------------------------------------------------------------------------------- |
| `query`               | string | Your query as parsed.                                                                                                             |
| `search_status`       | string | `succeeded` (every leg returned), `partial` (some legs failed), or `failed` (every leg failed).                                   |
| `plan`                | object | The resolved plan: `topic`, `source` (`llm` or `identity`), `degraded`, `subqueries[]`, `notes[]`.                                |
| `legs`                | array  | One truthful outcome row per dispatched leg (below).                                                                              |
| `items`               | array  | Deduplicated articles across all legs, each with `legs_returned` provenance.                                                      |
| `total_articles`      | number | `items.length`.                                                                                                                   |
| `countries_searched`  | array  | The resolved country codes, first-seen order.                                                                                     |
| `credits_charged`     | number | The net charge for this call (base + productive legs). Matches the envelope's own `credits_used` and the `X-Credits-Used` header. |
| `methodology_version` | string | `search-news/1.0`.                                                                                                                |

Each `legs[]` row: `leg_id` (`angleId|COUNTRY`), `country`, `language`, `angle_id`, `angle_label`, `effective_query` (the keyword actually searched), `query_source`, `status` (`succeeded` / `failed` / `skipped`), `article_count`, `quarantined_count` (rows dropped as schema-invalid or for having no usable http(s) URL), `credits_used` (1 only when the leg succeeded with at least one article, else 0), `latency_ms`, `error`.

Each `items[]` article: `id` (upstream article id, `null` when the source omits one), `title`, `url` (always http(s)), `canonical_url` (lane-derived: fragment dropped, tracking params stripped, host lowercased), `source_name`, `domain`, `snippet`, `image_url`, `published_at` (UTC ISO or `null`, never fabricated), `rank`, `placement`, plus the provenance stamps from the leg that found it (`country_code`, `language_code`, `angle_id`, `effective_query`, `query_source`) and `legs_returned` (every leg that returned it).

* **Zero articles with succeeded legs is a 200 success** with an empty `items` and base-only billing, not an error.
* **All legs failed is a `502 ALL_LEGS_FAILED`** with `ok: false`; the full hold is refunded automatically. The 502 body is the standard error envelope on its own: per-leg detail (`legs`, `plan`) is not returned on this path, so ask for the SSE stream when you need to see which legs failed and why.

***

Streaming (SSE) [#streaming-sse]

Add `Accept: text/event-stream` and you get a live stream instead of one envelope. Chunk types in order:

```
{ "type": "meta",           "query": "...", "plan": {...}, "sources_planned": ["KR", "US", ...] }
{ "type": "plan_refined",   "plan": {...}, "legs": [{ "leg_id": "sq0|KR", "country": "KR", "language": "ko", "keyword": "...", "query_source": "translated", "angle_id": "sq0", "angle_label": "..." }] }
{ "type": "source_started", "source": "sq0|KR" }
{ "type": "items",          "source": "sq0|KR", "items": [...], "duration_ms": 1234 }
{ "type": "source_failed",  "source": "sq1|US", "error": { "code": "...", "message": "..." } }
{ "type": "done",           "summary": { "credits_used": 4, "refunded": true, "coverage": 1, ... } }
```

Order and guarantees:

* **`meta` is first**, carrying the deterministic identity plan before the LLM call resolves.
* **`plan_refined` is always emitted on this lane** (unlike `/v1/search/everywhere`, where it is conditional on the LLM changing the plan). It carries the final plan plus the expanded `legs[]` with per-leg `query_source`, so a consumer can render localization honesty (translated vs fallback chips) at plan time, before any leg settles. A degraded run still emits it, with every leg stamped `fallback_original`.
* **Legs are keyed as `source` strings** in the form `angleId|COUNTRY` (e.g. `sq0|KR`; angle ids are positional with an `sq` prefix). These are per-request keys, not the fixed source ids of `/v1/search/everywhere`; the everywhere client reducer is not suitable for news streams. Skipped legs (aborted before dispatch) surface as `source_failed` with `error.code: "aborted"`.
* **`items` yield in completion order**, so fast editions land first. The deduplicated cross-leg view is only available in sync mode or by deduplicating client-side on `canonical_url ?? url`.
* **`done` is always last** with the run summary: `credits_used` (net), `refunded`, `coverage`, `sources_called` / `sources_succeeded` / `sources_failed`, `total_items`, `partial_failure`. On a stream this is the **only** place the net charge appears: response headers are flushed before the run settles, so a streaming response carries no `X-Credits-Used` / `X-Credits-Remaining` header (the sync mode does).

Billing on a stream is charge-for-delivered: if you disconnect mid-stream you pay the base plus the productive legs whose `items` chunks you received; the rest of the hold is refunded. On an internal error the whole hold is refunded and the terminal `done` reports `credits_used: 0`, `refunded: true`.

Streaming responses are never cached and never touch the sync cache. They are also not replayable with an `Idempotency-Key`: replay applies to sync JSON responses only, and an SSE-accept request that reuses a spent key will receive the stored JSON body back instead of a stream.

***

Billing [#billing]

Metered, **2-14 credits**: `2` base plus `1` per productive leg, held upfront at `2 + min(5 x countries, max_legs, 12)` and settled down to actual after the run.

| Scenario                               | Charge                                            |
| -------------------------------------- | ------------------------------------------------- |
| 3 countries, all legs return articles  | 2 + productive legs (up to 12 with `max_legs=12`) |
| Zero articles anywhere, legs succeeded | 2 (base only), 200 response                       |
| Every leg failed                       | 0 (full automatic refund), `502 ALL_LEGS_FAILED`  |
| Cache hit (sync, within 5 minutes)     | 0, `X-Cache: HIT`                                 |
| Idempotency-Key replay (sync)          | 0, `X-Idempotent-Replay: true`                    |

`X-Credits-Used` on the response (and `done.summary.credits_used` on a stream) is the source of truth for what you paid. Use `max_legs` to cap the worst case: `max_legs=3` caps the hold at 5 credits regardless of how many countries you request.

***

Latency and failure modes [#latency-and-failure-modes]

The lane plans before it fans out, so the combined plan+localize call is on the critical path. Its budget scales with the number of distinct languages (8s base + 0.7s per extra language, capped at 18s); on timeout the run degrades to the identity plan with every leg stamped `fallback_original` and a note in `plan.notes`, it does not fail. Legs then run fully in parallel, each bounded by the remaining route budget. Typical end-to-end: a few seconds for one country, 10 to 30s for a wide multi-language fan-out.

* **One leg times out or errors:** its row reports `status: "failed"` with an error class; sibling legs continue; the leg bills 0.
* **The localizer drops a language** (operator-laden, overlong, or script-mismatched keyword): that leg searches the planned keyword stamped `fallback_original`.
* **Client disconnect (SSE):** work stops, undelivered legs are refunded, the charge is settled exactly once.

Supported countries: the 50 editions in the lane's catalog (up to date list in the error message of an unknown `countries` code). Notable exclusions: Russia and Lebanon are not servable by the primary news source and are not in the catalog.

***

Related [#related]

* [`GET /v1/search/everywhere`](/docs/search/everywhere.md): the universal social search sibling.
* [`GET /v1/google_news/search`](/docs/google_news.md): the single-edition news search each leg rides on.
* [Credits](/docs/credits.md) and [Endpoint pricing](/docs/endpoint-pricing.md).