Web Scraping Search API
Scrape Web Scraping Search data with one API call. Searches web, news, and image sources and returns a single normalized list of web page results.
Last updated October 2026Maintained by the SocialCrawl team
Returns ranked web, news, and image results for a search query as one normalized list, with a title, URL, and snippet for each.
Use it when you do not know yet which URLs you need; once you have a URL, scrape it.
Try the Web Scraping Search API
See real data before writing a single line
Searching 68 platforms in parallel
TikTok·
Instagram·
YouTube·
Facebook·
X·
LinkedIn·
Reddit·
Threads·
Pinterest·
Twitch·
Truth Social·
Snapchat·
Kick·
Bluesky·
Kwai·
Rumble·
Spotify·
Apple Music·TikTok Shop·
Amazon Shop·
Google Shopping·
Trustpilot·
TripAdvisor·
Yelp·
Linktree·
Komi·
Pillar·
lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·
Finance·
Polymarket·
Tavily·
Hacker News·
GitHub·
Perplexity·
Naver·
Utility·
Universal SearchTikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal SearchTikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal SearchTikTok·Instagram·YouTube·Facebook·X·LinkedIn·Reddit·Threads·Pinterest·Twitch·Truth Social·Snapchat·Kick·Bluesky·Kwai·Rumble·Spotify·Apple Music·TikTok Shop·Amazon Shop·Google Shopping·Trustpilot·TripAdvisor·Yelp·Linktree·Komi·Pillar·lnk.bio·Facebook Ads·Google Ads·LinkedIn Ads·Google Search·Google News·Finance·Polymarket·Tavily·Hacker News·GitHub·Perplexity·Naver·Utility·Universal SearchWhat can you do with the Search API?
The Search endpoint gives you structured Web Scraping data with computed fields in a single request. No scraping infrastructure to build or maintain.
Example Request
curl -H "x-api-key: YOUR_API_KEY" \
"https://www.socialcrawl.dev/v1/web/search?query=social+media+data+api&limit=10"import requests
response = requests.get(
"https://www.socialcrawl.dev/v1/web/search",
params={
'query': 'social media data api',
'limit': '10',
},
headers={"x-api-key": "YOUR_API_KEY"},
)
data = response.json()const response = await fetch(
"https://www.socialcrawl.dev/v1/web/search?query=social+media+data+api&limit=10",
{
headers: { "x-api-key": "YOUR_API_KEY" },
},
);
const data = await response.json();Parameters
| Parameter | Required | Description |
|---|---|---|
| query | Yes | Search query. |
| sources | No | Comma-separated sources: web,news,images. |
| categories | No | CSV of source categories to restrict the search to, e.g. 'github,research'. Omit for an unrestricted web search. |
| limit | No | Results per source, from 1 to 100. |
| country | No | ISO 3166-1 alpha-2 country code the search is run from, e.g. 'US' (default) or 'GB'. Changes which results rank. |
| location | No | Free-text locality to bias results towards, e.g. 'London, United Kingdom'. Finer-grained than country. |
| time_range | No | Recency window as a Google `tbs` value: 'qdr:d' past day, 'qdr:w' past week, 'qdr:m' past month, 'qdr:y' past year. |
| sort_by_date | No | Sort newest-first instead of by relevance. Combines with time_range rather than replacing it. |
| include_domains | No | CSV of domains to restrict results to, e.g. 'nytimes.com,ft.com'. Cannot be combined with exclude_domains - sending both is a 400 before billing. |
| exclude_domains | No | CSV of domains to drop from the results. Cannot be combined with include_domains. |
| include_content | No | Also scrape each result and return its page content as markdown, not just the SERP row. This is what makes the call expensive: it scrapes one page per result, which is why the range runs to 120 credits. |
How does the Web Scraping Search API work?
Send a GET request with your API key and get back clean, structured JSON in our unified schema. Supported computed fields are populated when the source provides the required inputs.
Method
GET
Response
JSON
How do you scrape social media data in seconds?
The fastest social media scraping API for developers. Scrape profiles, posts, comments, and analytics from 68 platforms covering 10B+ monthly active users.
One schema, every platform
Query 68 platforms with identical response structures. Write your integration once.
Computed fields, not just scraped
When an endpoint supports these metrics and the source provides the required inputs, the normalized record includes engagement_rate, estimated_reach, content_category, and language. Ready to use.
See your data before you code
Visual Data Explorer. Paste any URL, get rich result cards, sortable tables, CSV export.
import requests
response = requests.get(
'https://www.socialcrawl.dev/v1/tiktok/profile',
params={'handle': 'charlidamelio'},
headers={'x-api-key': 'sc_YOUR_API_KEY'}
)
data = response.json(){
"success": true,
"platform": "tiktok",
"data": {
"author": {
"username": "charlidamelio",
"followers": 152400000
},
"engagement": {
"likes": 12400000000,
"engagement_rate": 0.087
},
"metadata": {
"language": "en",
"content_category": "lifestyle"
}
}
}Ready to scrape Web Scraping Search data?
Get your API key and start pulling Web Scraping data in under 60 seconds.