Web Scraping
Web Scraping
Extract data from Web Scraping
Web Scraping
Extract data from Web Scraping.
Base URL: /v1/web/...
Getting Started
Make your first request to any endpoint below.
3. Read computed fields
Every response includes computed fields: engagement_rate, language, content_category, and estimated_reach. See Computed fields for formulas, clamping rules, and null semantics.
Endpoints
24 endpoints available.
| Endpoint | Path | Credit Tier |
|---|---|---|
| Start an async batch scrape | /v1/web/batch-scrape | standard (1cr) |
| Get an async batch scrape job | /v1/web/batch-scrape/{job_id} | standard (0cr) |
| Cancel an async batch scrape job | /v1/web/batch-scrape/{job_id} | standard (0cr) |
| Start an async web crawl | /v1/web/crawl | standard (1cr) |
| Get an async crawl job | /v1/web/crawl/{job_id} | standard (0cr) |
| Cancel an async crawl job | /v1/web/crawl/{job_id} | standard (0cr) |
| List async web jobs | /v1/web/jobs | standard (0cr) |
| Map URLs on a site | /v1/web/map | standard (1cr) |
| Create a web monitor | /v1/web/monitors | standard (0cr) |
| List web monitors | /v1/web/monitors | standard (0cr) |
| Get a web monitor | /v1/web/monitors/{monitor_id} | standard (0cr) |
| Update a web monitor | /v1/web/monitors/{monitor_id} | standard (0cr) |
| Delete a web monitor | /v1/web/monitors/{monitor_id} | standard (0cr) |
| List web monitor checks | /v1/web/monitors/{monitor_id}/checks | standard (0cr) |
| Parse an uploaded document | /v1/web/parse | standard (1cr) |
| Scrape a web page | /v1/web/scrape | standard (1cr) |
| Search the web | /v1/web/search | standard (2cr) |
| List interactive web sessions | /v1/web/sessions | standard (0cr) |
| Get an interactive web session | /v1/web/sessions/{session_id} | standard (0cr) |
| Close an interactive web session | /v1/web/sessions/{session_id} | standard (0cr) |
| Execute an interaction in a web session | /v1/web/sessions/{session_id}/execute | standard (0cr) |
| Extract structured data from a web page | /v1/web/extract | advanced (5cr) |
| Create an interactive web session | /v1/web/sessions | advanced (5cr) |
| Start an async web agent job | /v1/web/agent | premium (25cr) |
Notes
- All endpoints use
GETmethod with query parameters - Authentication via
x-api-keyheader - Responses follow the unified SocialCrawl schema