100 free credits — no credit card required

Web Batch Scrape API

Name: Web Batch Scrape API
Brand: SocialCrawl
Availability: InStock

Scrape Web Batch Scrape data with one API call. Starts an async batch scrape job for multiple URLs and returns a SocialCrawl job id.

Last updated July 2026

Start for free View Documentation

Try the Web Batch Scrape API

See real data before writing a single line

GET/v1/web/batch-scrape

Searching 44 platforms in parallel

TikTok·

Instagram·

YouTube·

Facebook·

X·

LinkedIn·

Reddit·

Threads·

Pinterest·

Twitch·

Truth Social·

Snapchat·

Kick·

Bluesky·

Kwai·

Rumble·

Spotify·

TikTok Shop·

Amazon Shop·

Google Shopping·

Trustpilot·

TripAdvisor·

Linktree·

Komi·

Pillar·

lnk.bio·

Facebook Ads·

Google Ads·

LinkedIn Ads·

Google Search·

Google News·

Google Finance·

Polymarket·

Tavily·

Hacker News·

GitHub·

Perplexity·

Naver·Utility·

Universal Search

TikTok·

Instagram·

YouTube·

Facebook·

X·

LinkedIn·

Reddit·

Threads·

Pinterest·

Twitch·

Truth Social·

Snapchat·

Kick·

Bluesky·

Kwai·

Rumble·

Spotify·

TikTok Shop·

Amazon Shop·

Google Shopping·

Trustpilot·

TripAdvisor·

Linktree·

Komi·

Pillar·

lnk.bio·

Facebook Ads·

Google Ads·

LinkedIn Ads·

Google Search·

Google News·

Google Finance·

Polymarket·

Tavily·

Hacker News·

GitHub·

Perplexity·

Naver·Utility·

Universal Search

TikTok·

Instagram·

YouTube·

Facebook·

X·

LinkedIn·

Reddit·

Threads·

Pinterest·

Twitch·

Truth Social·

Snapchat·

Kick·

Bluesky·

Kwai·

Rumble·

Spotify·

TikTok Shop·

Amazon Shop·

Google Shopping·

Trustpilot·

TripAdvisor·

Linktree·

Komi·

Pillar·

lnk.bio·

Facebook Ads·

Google Ads·

LinkedIn Ads·

Google Search·

Google News·

Google Finance·

Polymarket·

Tavily·

Hacker News·

GitHub·

Perplexity·

Naver·Utility·

Universal Search

TikTok·

Instagram·

YouTube·

Facebook·

X·

LinkedIn·

Reddit·

Threads·

Pinterest·

Twitch·

Truth Social·

Snapchat·

Kick·

Bluesky·

Kwai·

Rumble·

Spotify·

TikTok Shop·

Amazon Shop·

Google Shopping·

Trustpilot·

TripAdvisor·

Linktree·

Komi·

Pillar·

lnk.bio·

Facebook Ads·

Google Ads·

LinkedIn Ads·

Google Search·

Google News·

Google Finance·

Polymarket·

Tavily·

Hacker News·

GitHub·

Perplexity·

Naver·Utility·

Universal Search

Web API

What can you do with the Batch Scrape API?

The Batch Scrape endpoint gives you structured Web data with computed fields in a single request. No scraping infrastructure to build or maintain.

Example Request

curl -H "x-api-key: YOUR_API_KEY" \
  "https://www.socialcrawl.dev/v1/web/batch-scrape?urls=https%3A%2F%2Fexample.com%2Fa%2Chttps%3A%2F%2Fexample.com%2Fb"

import requests

response = requests.get(
    "https://www.socialcrawl.dev/v1/web/batch-scrape",
    params={
    'urls': 'https://example.com/a,https://example.com/b',
    },
    headers={"x-api-key": "YOUR_API_KEY"},
)

data = response.json()

const response = await fetch(
  "https://www.socialcrawl.dev/v1/web/batch-scrape?urls=https%3A%2F%2Fexample.com%2Fa%2Chttps%3A%2F%2Fexample.com%2Fb",
  {
    headers: { "x-api-key": "YOUR_API_KEY" },
  },
);

const data = await response.json();

Parameters

Parameter	Required	Description
urls	Yes	Array or comma-separated list of public URLs to scrape.
ignore_invalid_urls	No	Skip invalid URLs instead of failing the job.
formats	No
only_main_content	No
proxy	No	(basic \| auto \| enhanced)
webhook_url	No

Example Response

What does the Web Batch Scrape API return?

Every response follows one unified schema. Here is a real, unmodified response body, so you can see the exact fields you get back before spending a credit.

Example response

{
  "success": true,
  "platform": "instagram",
  "endpoint": "/v1/instagram/engagement",
  "data": {
    "engagement_rate_percentages": 38.33,
    "recent_posts": 12,
    "followers": 87608035,
    "comments": 528912,
    "likes": 33049046,
    "recent_posts_explanation": "Statistics based on the last 12 posts",
    "id_user": "2278169415",
    "username": "mrbeast",
    "is_private": false,
    "posts_details": [
      {
        "likes": 5636982,
        "comments": 69484,
        "taken_at": 1781457954,
        "datetime": "2026-06-14 20:25:54",
        "hours_since_post": 461,
        "time_ago": "19 days ago",
        "likes_per_hour": 12228,
        "comments_per_hour": 151
      },
      {
        "likes": 20000768,
        "comments": 223510,
        "taken_at": 1732824650,
        "datetime": "2024-11-28 23:10:50",
        "hours_since_post": 13971,
        "time_ago": "2 years ago",
        "likes_per_hour": 1432,
        "comments_per_hour": 16
      },
      {
        "likes": 929226,
        "comments": 30633,
        "taken_at": 1782232475,
        "datetime": "2026-06-23 19:34:35",
        "hours_since_post": 246,
        "time_ago": "10 days ago",
        "likes_per_hour": 3777,
        "comments_per_hour": 125
      },
      {
        "likes": 487761,
        "comments": 22482,
        "taken_at": 1781799425,
        "datetime": "2026-06-18 19:17:05",
        "hours_since_post": 366,
        "time_ago": "15 days ago",
        "likes_per_hour": 1333,
        "comments_per_hour": 61
      },
      {
        "likes": 712265,
        "comments": 15716,
        "taken_at": 1781366405,
        "datetime": "2026-06-13 19:00:05",
        "hours_since_post": 487,
        "time_ago": "20 days ago",
        "likes_per_hour": 1463,
        "comments_per_hour": 32
      },
      {
        "likes": 1475116,
        "comments": 35386,
        "taken_at": 1781277094,
        "datetime": "2026-06-12 18:11:34",
        "hours_since_post": 512,
        "time_ago": "21 days ago",
        "likes_per_hour": 2881,
        "comments_per_hour": 69
      },
      {
        "likes": 1108220,
        "comments": 26632,
        "taken_at": 1780160249,
        "datetime": "2026-05-30 19:57:29",
        "hours_since_post": 822,
        "time_ago": "1 months ago",
        "likes_per_hour": 1348,
        "comments_per_hour": 32
      },
      {
        "likes": 542948,
        "comments": 28476,
        "taken_at": 1779375582,
        "datetime": "2026-05-21 17:59:42",
        "hours_since_post": 1040,
        "time_ago": "1 months ago",
        "likes_per_hour": 522,
        "comments_per_hour": 27
      },
      {
        "likes": 698514,
        "comments": 24401,
        "taken_at": 1779120014,
        "datetime": "2026-05-18 19:00:14",
        "hours_since_post": 1111,
        "time_ago": "2 months ago",
        "likes_per_hour": 629,
        "comments_per_hour": 22
      },
      {
        "likes": 468000,
        "comments": 13548,
        "taken_at": 1778947209,
        "datetime": "2026-05-16 19:00:09",
        "hours_since_post": 1159,
        "time_ago": "2 months ago",
        "likes_per_hour": 404,
        "comments_per_hour": 12
      },
      {
        "likes": 526594,
        "comments": 24411,
        "taken_at": 1777737719,
        "datetime": "2026-05-02 19:01:59",
        "hours_since_post": 1495,
        "time_ago": "2 months ago",
        "likes_per_hour": 352,
        "comments_per_hour": 16
      },
      {
        "likes": 462652,
        "comments": 14233,
        "taken_at": 1777580305,
        "datetime": "2026-04-30 23:18:25",
        "hours_since_post": 1538,
        "time_ago": "2 months ago",
        "likes_per_hour": 301,
        "comments_per_hour": 9
      }
    ]
  },
  "credits_used": 5,
  "credits_remaining": 9999,
  "request_id": "req_example000000",
  "cached": false
}

Live sample illustrating the unified response shape. Field values reflect the record you query.

API Details

How does the Web Batch Scrape API work?

Send a GET request with your API key and get back clean, structured JSON. Every response follows our unified schema with computed fields.

Method

GET

Response

JSON

Why SocialCrawl

Why use SocialCrawl for Web Batch Scrape data?

We handle the complexity of Web data extraction so you can focus on building. Unified schema, AI enrichment, and zero platform logic in your code.

One schema for every platform — write your integration once
Computed fields on every response: engagement rate, reach, category, language
No scraping infrastructure to build, no proxies to manage, no rate limits to fight
Real-time data from Web — no caching, no delays

Developer First

How do you scrape social media data in seconds?

The fastest social media scraping API for developers. Scrape profiles, posts, comments, and analytics from 44 platforms covering 10B+ monthly active users.

One schema, every platform

Query 44 platforms with identical response structures. Write your integration once.

Computed fields, not just scraped

Every response includes engagement_rate, estimated_reach, content_category, and language — ready to use.

See your data before you code

Visual Data Explorer — paste any URL, get rich result cards, sortable tables, CSV export.

import requests

response = requests.get(
    'https://www.socialcrawl.dev/v1/tiktok/profile',
    params={'handle': 'charlidamelio'},
    headers={'x-api-key': 'sc_YOUR_API_KEY'}
)
data = response.json()

[ .JSON ]

{
  "success": true,
  "platform": "tiktok",
  "data": {
    "author": {
      "username": "charlidamelio",
      "followers": 152400000
    },
    "engagement": {
      "likes": 12400000000,
      "engagement_rate": 0.087
    },
    "metadata": {
      "language": "en",
      "content_category": "lifestyle"
    }
  }
}

+ 44 platforms

Ready to scrape Web Batch Scrape data?

Get your API key and start pulling Web data in under 60 seconds.

Start for free