Tavily
Web grounding via the Tavily API — search, extract, map, and crawl
Tavily
Web grounding via the Tavily API. Search the live web with optional LLM-generated answers, extract clean content from URLs, and crawl or map sitegraphs. 4 endpoints available.
Base URL: /v1/tavily/...
Tavily uses the same authentication and credit model as every other SocialCrawl endpoint — pass your x-api-key header, pay 1 credit per call, get a unified response envelope. The fact that the upstream is POST + Bearer auth is invisible to you.
Getting Started
1. Run a web search
curl "https://api.socialcrawl.dev/v1/tavily/search?query=claude+opus+release+notes" \
-H "x-api-key: sc_your_api_key_here"2. Search with an LLM-generated answer
curl "https://api.socialcrawl.dev/v1/tavily/search?query=who+founded+anthropic&include_answer=true" \
-H "x-api-key: sc_your_api_key_here"The response includes both ranked results[] and a synthesised answer string.
3. Extract clean content from a URL
curl "https://api.socialcrawl.dev/v1/tavily/extract?urls=https://en.wikipedia.org/wiki/Lionel_Messi" \
-H "x-api-key: sc_your_api_key_here"You can pass up to 20 comma-separated URLs in a single call.
Endpoints
| Endpoint | Path | Credit Tier |
|---|---|---|
| Tavily web search with optional LLM-generated answer | /v1/tavily/search | standard (1cr) |
| Extract clean content from one or more URLs | /v1/tavily/extract | standard (1cr) |
| Map a website's sitegraph | /v1/tavily/map | standard (1cr) |
| Crawl a website with LLM-driven path selection | /v1/tavily/crawl | standard (1cr) |
Response Shape
Tavily endpoints use the Analytics archetype, which means the upstream response passes through with only the standard envelope-key strip applied. You see Tavily's full payload — including the answer field for search, the failed_results array for extract, and the instructions echo for crawl — exactly as Tavily returns it.
This differs from the social platforms (TikTok, Instagram, etc.) which apply a unified Author / Post field map. Web search results aren't social posts and aren't shoehorned into that schema.
Notes
- All endpoints use
GETmethod with query parameters - Authentication via
x-api-keyheader (your SocialCrawl key, not your Tavily key) - Array-typed Tavily params (e.g.
urls,include_domains,select_paths) are accepted as comma-separated strings on this surface and split server-side - Empty search results (
results: []) are billed normally — you paid for the search, an empty result is a real answer - Caching: search responses cached 2 min, extract 10 min, map / crawl 30 min