How to Scrape Twitter (X) in 2026: A Developer's Guide

The quickest way to scrape Twitter, now X, in 2026 is a single GET request to a data API, no logged-in session and no paying for an expensive API tier just to read public posts. Here is the whole thing:

curl "https://www.socialcrawl.dev/v1/twitter/profile?handle=nasa" \
  -H "x-api-key: sc_your_api_key_here"

That returns a normalized profile: followers, following, tweet count, bio, verification, for 1 credit. The rest of this guide is the honest version: what the official X API costs and why that pricing pushes many developers elsewhere, what a DIY Twitter scraper really involves, the legal picture, and how to pull profiles and tweets through a real endpoint with runnable Python, Node, and curl.

Stack: Python 3.10+ or Node 18+ · requests / fetch · a SocialCrawl API key (free tier: 100 credits, no card)

---

Can you get Twitter data from the official X API?

Yes, but the pricing is the reason this guide exists.

Unlike Instagram or LinkedIn, X does offer an official API that can read public tweets and profiles. The catch is cost and access. Since the 2023 overhaul, X's API has been a paid product with steep tiers: the Free tier is effectively write-only with a tiny monthly cap and no meaningful read access, the Basic tier runs a few hundred dollars a month with modest read limits, the Pro tier jumps into the low thousands per month, and Enterprise is custom-priced. In 2026 X has continued to move toward usage-based pricing on top of these tiers, so real costs depend on volume. Pricing at this level changes frequently, so confirm the current numbers on the X developer portal before you commit.

The practical upshot: if you need to read public tweets and profiles at any real volume, the official API is either too limited (Free, Basic) or too expensive (Pro, Enterprise) for many projects. That economics gap is exactly why third-party data APIs and DIY scrapers remain common for X.

Should you build your own Twitter scraper?

The DIY route is viable for X, more so than for LinkedIn, but it has real costs.

Anti-bot reality. X increasingly gates content behind login and rate-limits unauthenticated access, so scraping it yourself usually means driving a headless browser with logged-in sessions or reverse-engineering the private endpoints its web client uses. Both face fingerprinting, rate limits, and account suspensions, and success rates depend on proxies and rotating accounts.

Maintenance. X's frontend and internal endpoints have changed frequently through its ongoing product churn, so a scraper that works today can break without notice, returning empty results rather than clean errors. You need monitoring on top of the scraper itself.

Legal considerations. Scraping publicly accessible data has survived legal challenge in US courts (the hiQ Labs v. LinkedIn line of cases), but that is not a blanket permission slip. X's Terms of Service prohibit automated collection, so ToS exposure is real, and tweet and profile data is personal data subject to GDPR and CCPA depending on your jurisdiction and use case. No tool can honestly promise it "gets around" X's terms. What a data API removes is the collection, cost, and account-suspension problem; the legal and product questions remain yours. Read the legal and technical guide to social media scraping before shipping anything commercial.

The rest of this guide takes the API route: public X data in a normalized shape at a flat per-call cost, so you skip both the expensive official tiers and the DIY maintenance.

Setting up: one key, one request

Get an API key at socialcrawl.dev, sign up, then open Dashboard → API Keys. Keys look like sc_ plus 32 random bytes, shown in full once, so copy it immediately. New accounts get 100 free credits that never expire, and a Twitter profile call is 1 credit.

Keep the key out of source control:

export SOCIALCRAWL_API_KEY=sc_your_api_key_here

To see the response shape before writing code, paste an X profile URL into the Visual Explorer.

Scraping a Twitter (X) profile

The endpoint is GET /v1/twitter/profile. It takes one required parameter, handle (the username without @), and returns the normalized author object.

import os
import requests

API_KEY = os.environ["SOCIALCRAWL_API_KEY"]
BASE = "https://www.socialcrawl.dev/v1"

resp = requests.get(
    f"{BASE}/twitter/profile",
    params={"handle": "nasa"},
    headers={"x-api-key": API_KEY},
    timeout=30,
)
author = resp.json()["data"]["author"]
print(author["username"], author["followers"], "followers")

The same call in Node:

const res = await fetch(
  "https://www.socialcrawl.dev/v1/twitter/profile?handle=nasa",
  { headers: { "x-api-key": process.env.SOCIALCRAWL_API_KEY } },
);
const { data } = await res.json();
console.log(data.author.username, data.author.followers, "followers");

Here is a real, unmodified response captured with a live dev key in July 2026:

{
  "success": true,
  "platform": "twitter",
  "endpoint": "/v1/twitter/profile",
  "data": {
    "author": {
      "id": "11348282",
      "username": "NASA",
      "display_name": "NASA",
      "avatar_url": "https://pbs.twimg.com/profile_images/1321163587679784960/0ZxKlEKB_normal.jpg",
      "bio": "Making the seemingly impossible, possible. ✨",
      "verified": true,
      "followers": 92203816,
      "following": 119,
      "posts_count": 74231,
      "private": false
    },
    "computed": {
      "language": "en",
      "content_category": "other"
    }
  },
  "credits_used": 1,
  "credits_remaining": 184261,
  "request_id": "req-ZNlOy9hIaO06Mqo9",
  "cached": false
}

The author field names (username, followers, verified, bio) are identical across all 44 platforms, so the parser you write for X works unchanged when you add Instagram or TikTok.

Scraping tweets from a user

To pull a user's tweets, GET /v1/twitter/user/tweets takes the handle and paginates with a cursor. Read data.items and pass data.next_cursor back until it disappears:

def scrape_user_tweets(handle: str, max_pages: int = 5) -> list[dict]:
    tweets, cursor = [], None
    for _ in range(max_pages):
        params = {"handle": handle}
        if cursor:
            params["cursor"] = cursor  # pass next_cursor back verbatim
        payload = requests.get(
            f"{BASE}/twitter/user/tweets",
            params=params,
            headers={"x-api-key": API_KEY},
            timeout=30,
        ).json()
        if not payload.get("success"):
            raise RuntimeError(payload["error"]["message"])
        tweets.extend(payload["data"]["items"])
        cursor = payload["data"].get("next_cursor")
        if not cursor:
            break
    return tweets

For a single tweet's detail, GET /v1/twitter/tweet takes the tweet URL. Both return the canonical post shape, so the same export code works across the tweet, user-tweets, and search reads.

Pagination, rate limits, and credits

Three operational facts, all from the live API:

• Pagination is a cursor contract. The response carries data.next_cursor; pass it back verbatim as the cursor parameter and stop when it is null. The pagination page lists the exact parameter per endpoint.
• There is no requests-per-minute quota. The only ceiling is 50 concurrent requests per API key; exceed it and you get a 429 CONCURRENCY_LIMIT envelope with no credits charged. Compare that to the official X API, where per-tier read caps are the main constraint.
• Failures refund credits. Errors use the same envelope with success: false. Empty-upstream 404s and transient 502/503s refund the credit automatically, so a retry wrapper never double-spends. The full matrix is in the error handling docs.

Twitter profile, user tweets, and tweet detail are Standard-tier at 1 credit per page, so the math stays flat, and there is no monthly platform fee like the official API's tiers. The free 100 credits cover 100 reads; after that, pricing is pay-as-you-go and never expires (£15 for 2,500 credits, £49 for 20,000).

Where to go from here

You now have X profiles and tweets in one schema for 1 credit each, without paying for a Pro tier or maintaining a scraper. Next steps:

• Browse the rest of the surface on the Twitter/X platform page: profiles, user tweets, tweet detail, communities, and AI-grounded search.
• Monitor mentions across platforms with the free brand mention checker, or preview any profile with no code in the Visual Explorer.
• Compare the paid options in the best Twitter/X data APIs roundup, or read the X (Twitter) API 2026 guide for the official-API pricing detail.
• Start on the free tier: 100 credits, no card, a hundred reads to test the fit.

Frequently asked questions

Is it legal to scrape Twitter (X)?

Scraping publicly accessible data has survived legal challenge in US courts (the hiQ v. LinkedIn line of cases), but that is not blanket permission. X's Terms of Service prohibit automated collection, so ToS exposure is real, and tweet and profile data is personal data subject to GDPR and CCPA depending on jurisdiction and use case. No API can honestly claim to get around X's terms. Read the legal and technical guide before anything commercial, and talk to a lawyer if revenue depends on it.

Can you scrape Twitter without the official API?

Yes. A third-party data API runs its own collection infrastructure and returns public X data through your API key, so you skip the official API's paid tiers entirely and never handle an X login. A DIY browser scraper is the other option, but it faces rate limits and account suspensions.

How much does the official X API cost in 2026?

X's API is a paid product with tiered pricing: the Free tier is effectively write-only with a tiny cap, Basic runs a few hundred dollars a month, Pro jumps into the low thousands, and Enterprise is custom, with usage-based pricing layered on top in 2026. Read access at volume is the expensive part. Confirm current numbers on the X developer portal, since this pricing changes often.

Can you scrape tweets without logging in?

With a data API, yes, the API handles collection and returns public tweets through your key, no X session required. Unauthenticated DIY scraping of X has become unreliable as more content is gated behind login and rate limits.

How do you scrape a user's tweets?

Use GET /v1/twitter/user/tweets with the handle. Read data.items and pass data.next_cursor back as the cursor parameter until it is null. Each page is 1 credit and returns the canonical post shape, ready to export to CSV.

What is the best language for scraping Twitter?

Any language that can make an HTTP request. The endpoints are single authenticated GETs, so Python with requests, Node with fetch, or a plain curl command work identically, with the same envelope and cursor pagination.

How much does it cost to scrape Twitter with a data API?

On SocialCrawl, Twitter profile, user tweets, and tweet detail are 1 credit per page, with no monthly platform fee. The free tier is 100 credits (no card, never expires); after that £15 buys 2,500 credits and £49 buys 20,000, pay-as-you-go with no expiry.