100 크레딧 무료, 카드 등록 없이지금 시작하기
Logo
무료로 시작하기
Web Scraping logo
100 크레딧 무료, 카드 등록 없이

Web Scraping Scrape API

API 한 번의 호출로 Web Scraping Scrape 데이터를 받아 가세요. Fetches a public web page and returns clean content, metadata, and optional media in the unified WebPage schema.

2026년 10월 업데이트SocialCrawl 팀이 직접 관리해요

무료로 시작하기문서 보기

웹 페이지 하나의 내용을 깔끔한 마크다운이나 HTML로 반환합니다. 최종 URL과 상태 코드, 수집 메타데이터가 함께 오고 원하면 스크린샷까지 붙습니다.

사이트 전체가 대상이라면 crawl 작업을 걸어야 합니다. URL이 이미 정해진 한 페이지를 가져올 때 사용하세요.

Web Scraping Scrape API, 직접 써 보세요

코드 한 줄 짜기 전에, 실제 데이터부터

GET/v1/web/scrape

Public URL to fetch.

선택 파라미터 14개

Comma-separated output formats such as markdown,screenshot.

Strip nav, headers, footers and sidebars and keep the article body. Default true; send false when you need the whole page, including the chrome. A markdown-only scrape of a site the primary scraper cannot fetch is served by a second extractor, which returns the whole page.

Milliseconds to wait after load before capturing, for pages that render their content in JavaScript. Leave unset unless the page comes back empty or half-built.

Render in a mobile viewport with a mobile user agent. Use it when the site serves a different layout to phones.

Hard cap on the page load in milliseconds, 1 to 30000. A page that exceeds it fails and is refunded rather than returning a partial capture.

Accept an upstream-cached copy up to this many milliseconds old, which is faster and still one credit. Default 172800000 (48 hours); send 0 to force a live fetch.

ISO 3166-1 alpha-2 country code to fetch from, e.g. 'us' or 'de'. Use it for geo-varying pages such as pricing or availability.

When a screenshot format is requested, capture the entire scrollable page instead of the visible viewport.

CSV of CSS selectors or HTML tags to keep, e.g. 'article,main'. Everything outside them is dropped. Use it to pin extraction to a known container.

CSV of CSS selectors or HTML tags to drop, e.g. 'nav,footer,.cookie-banner'. Applied after include_tags.

Proxy tier: basic, auto, or enhanced.

Parse the target as a PDF and return its extracted text. Send it when the URL points at a PDF rather than an HTML page.

Block ad and tracker requests, which is faster and quieter. Default true.

Drop inline base64-encoded images from the returned content instead of carrying them in the payload. Default true.

68개 플랫폼을 병렬로 검색합니다

·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
·TikTok logoTikTok·Instagram logoInstagram·YouTube logoYouTube·Facebook logoFacebook·X logoX·LinkedIn logoLinkedIn·Reddit logoReddit·Threads logoThreads·Pinterest logoPinterest·Twitch logoTwitch·Truth Social logoTruth Social·Snapchat logoSnapchat·Kick logoKick·Bluesky logoBluesky·Kwai logoKwai·Rumble logoRumble·Spotify logoSpotify·Apple Music logoApple Music·TikTok Shop logoTikTok Shop·Amazon Shop logoAmazon Shop·Google Shopping logoGoogle Shopping·Trustpilot logoTrustpilot·TripAdvisor logoTripAdvisor·Yelp logoYelp·Linktree logoLinktree·Komi logoKomi·Pillar logoPillar·lnk.bio logolnk.bio·Facebook Ads logoFacebook Ads·Google Ads logoGoogle Ads·LinkedIn Ads logoLinkedIn Ads·Google Search logoGoogle Search·Google News logoGoogle News·Finance logoFinance·Polymarket logoPolymarket·Tavily logoTavily·Hacker News logoHacker News·GitHub logoGitHub·Perplexity logoPerplexity·Naver logoNaver·Utility logoUtility·Universal Search logoUniversal Search
Web Scraping API

Scrape API로 무엇을 할 수 있을까요

Scrape 엔드포인트가 통합 스키마와 계산 필드를 담은 Web Scraping 데이터를 한 번의 요청으로 보내드려요. 스크래핑 인프라를 직접 만들거나 유지할 필요가 없어요.

요청 예시

curl -H "x-api-key: YOUR_API_KEY" \
  "https://www.socialcrawl.dev/v1/web/scrape?url=https%3A%2F%2Fexample.com&formats=markdown"
import requests

response = requests.get(
    "https://www.socialcrawl.dev/v1/web/scrape",
    params={
    'url': 'https://example.com',
    'formats': 'markdown',
    },
    headers={"x-api-key": "YOUR_API_KEY"},
)

data = response.json()
const response = await fetch(
  "https://www.socialcrawl.dev/v1/web/scrape?url=https%3A%2F%2Fexample.com&formats=markdown",
  {
    headers: { "x-api-key": "YOUR_API_KEY" },
  },
);

const data = await response.json();

파라미터

파라미터필수설명
url예Public URL to fetch.
formats아니오Comma-separated output formats such as markdown,screenshot.
only_main_content아니오Strip nav, headers, footers and sidebars and keep the article body. Default true; send false when you need the whole page, including the chrome. A markdown-only scrape of a site the primary scraper cannot fetch is served by a second extractor, which returns the whole page.
wait_for아니오Milliseconds to wait after load before capturing, for pages that render their content in JavaScript. Leave unset unless the page comes back empty or half-built.
mobile아니오Render in a mobile viewport with a mobile user agent. Use it when the site serves a different layout to phones.
timeout아니오Hard cap on the page load in milliseconds, 1 to 30000. A page that exceeds it fails and is refunded rather than returning a partial capture.
max_age아니오Accept an upstream-cached copy up to this many milliseconds old, which is faster and still one credit. Default 172800000 (48 hours); send 0 to force a live fetch.
location_country아니오ISO 3166-1 alpha-2 country code to fetch from, e.g. 'us' or 'de'. Use it for geo-varying pages such as pricing or availability.
screenshot_full_page아니오When a screenshot format is requested, capture the entire scrollable page instead of the visible viewport.
include_tags아니오CSV of CSS selectors or HTML tags to keep, e.g. 'article,main'. Everything outside them is dropped. Use it to pin extraction to a known container.
exclude_tags아니오CSV of CSS selectors or HTML tags to drop, e.g. 'nav,footer,.cookie-banner'. Applied after include_tags.
proxy아니오Proxy tier: basic, auto, or enhanced. (basic | auto | enhanced)
pdf_parse아니오Parse the target as a PDF and return its extracted text. Send it when the URL points at a PDF rather than an HTML page.
block_ads아니오Block ad and tracker requests, which is faster and quieter. Default true.
remove_base64_images아니오Drop inline base64-encoded images from the returned content instead of carrying them in the payload. Default true.
응답 예시

Web Scraping Scrape API는 무엇을 돌려주나요

모든 응답은 하나의 통합 스키마를 따라요. 크레딧을 쓰기 전에 어떤 필드가 돌아오는지, 실제 응답 본문 그대로 확인해 보세요.

응답 예시 보기
{
  "success": true,
  "platform": "web",
  "endpoint": "/v1/web/scrape",
  "data": {
    "page": {
      "url": "https://example.com",
      "final_url": "https://example.com/",
      "status_code": 200,
      "scrape_id": "01a0fca4-f04b-7369-9cd6-64e0f4d82214",
      "fetched_at": null,
      "content": {
        "markdown": "Thisdomainisforuseindocumentationexampleswithoutneedingpermission.Thisisnotaservice,avoidrelyingonitfortestingandmonitoringpurposes.\n\nهذا النطاق مُخصص للاستخدام في أمثلة التوثيق دون الحاجة إلى إذن. هذه ليست خدمة، يُرجى تجنب الاعتماد عليها لأغراض الاختبار والمراقبة.\n\n该域名仅用于文档示例，无需获得许可。这并非一项服务，请勿将其用于测试和监控目的。\n\nL’usagedecedomaineestréservéàdesexemplesdedocumentation,sansautorisationpréalable.Ilnes’agitpasd’unservice;sonutilisationàdesfinsdetestoudesurveillanceestàéviter.\n\nДанныйдоменпредназначендляиспользованиявпримерахдокументациибезнеобходимостиполученияпредварительногоразрешения.Этонесервис;нерекомендуетсяегоиспользованиедлятестированияимониторинга.\n\nEstedominioestádestinadoalusoenejemplosdedocumentaciónsinnecesidaddepermiso.Estonoesunservicio,evitarutilizarlopararealizarpruebasomonitoreos.\n\n [Learn more](https://iana.org/help/example-domains)",
        "html": null,
        "raw_html": null,
        "summary": null
      },
      "media": {
        "screenshot_url": null,
        "audio_url": null,
        "video_url": null
      },
      "extraction": null,
      "answer": null,
      "highlights": null,
      "change_tracking": null,
      "page_count": null,
      "total_page_count": null,
      "fetch": {
        "cache_state": "hit",
        "cached_at": "2026-10-02T12:40:55.009Z",
        "proxy_tier": null
      },
      "page_state": {
        "value": "content",
        "confidence": 0.85,
        "skipped_judge": false
      }
    }
  },
  "credits_used": 1,
  "request_id": "req_example000000",
  "cached": false,
  "credits_remaining": 9999,
  "source": "captured",
  "captured_at": "2026-10-02T12:44:23.265Z",
  "redacted": true
}

통합 응답 구조를 보여 주는 실제 샘플이에요. 필드 값은 조회한 대상에 따라 달라져요.

API 상세

Web Scraping Scrape API는 어떻게 동작하나요

API 키와 함께 GET 요청을 보내면, 통합 스키마와 계산 필드를 담은 깔끔한 JSON이 돌아와요.

메서드

GET

응답 형식

JSON

왜 SocialCrawl인가요

Web Scraping Scrape 데이터는 왜 SocialCrawl로 가져올까요

Web Scraping 데이터 추출에서 까다로운 부분은 SocialCrawl이 가려 드려요. 통합 스키마, AI 인리치먼트, 그리고 코드에 남지 않는 플랫폼 로직까지.

개발자 우선

소셜 미디어 데이터를 몇 초 만에 수집하는 방법

개발자를 위한 가장 빠른 소셜 미디어 스크래핑 API. 월간 활성 사용자 100억 명 이상을 포괄하는 68개 플랫폼에서 프로필, 게시물, 댓글, 분석 데이터를 수집하세요.

모든 플랫폼을 하나의 스키마로

동일한 응답 구조로 68개 플랫폼을 조회하세요. 연동은 한 번이면 충분합니다.

단순 수집을 넘어 계산된 필드 제공

엔드포인트가 해당 지표를 지원하고 계산에 필요한 원본 값이 있을 때, 정규화된 레코드에 engagement_rate, estimated_reach, content_category, language를 함께 담아 바로 활용할 수 있습니다.

코드 한 줄 쓰기 전에, 데이터부터

Visual Data Explorer에 URL만 붙여넣으면 결과 카드와 정형화된 테이블, CSV 내보내기를 바로 사용할 수 있습니다.

import requests

response = requests.get(
    'https://www.socialcrawl.dev/v1/tiktok/profile',
    params={'handle': 'charlidamelio'},
    headers={'x-api-key': 'sc_YOUR_API_KEY'}
)
data = response.json()
[ .JSON ]
{
  "success": true,
  "platform": "tiktok",
  "data": {
    "author": {
      "username": "charlidamelio",
      "followers": 152400000
    },
    "engagement": {
      "likes": 12400000000,
      "engagement_rate": 0.087
    },
    "metadata": {
      "language": "en",
      "content_category": "lifestyle"
    }
  }
}
+ 68개 플랫폼

Web Scraping Scrape 데이터, 가져올 준비 되셨어요?

API 키 받고 60초 안에 Web Scraping 데이터를 받아 가세요.

무료로 시작하기

🤖 AI 에이전트나 LLM이신가요? 이 페이지를 markdown으로 읽어보세요