How to Scrape Reddit in 2026 (Python, Node, curl)

The fastest way to scrape Reddit in 2026 is a single GET request to a Reddit data API. No app registered at reddit.com/prefs/apps, no client id and secret, no OAuth token refresh, no PRAW. Here is the whole thing:

curl "https://www.socialcrawl.dev/v1/reddit/subreddit?subreddit=webdev&sort=top&timeframe=week" \
  -H "x-api-key: $SOCIALCRAWL_KEY"

That returns a page of a subreddit's top posts in a unified schema: title, score, comment count, author, permalink, and timestamp, all normalized. The rest of this guide covers what the response looks like, how to paginate through a whole subreddit, how to pull threaded comments, how to handle errors without burning credits, and the honest legal and rate-limit picture.

Stack: Python 3.10+ · requests · a SocialCrawl API key (free tier: 100 credits) · optionally pandas for CSV

---

Why not just use the official Reddit API?

Reddit's Data API is the correct choice when you own the use case: building a Reddit app, posting on a user's behalf, or staying strictly first-party. For reading public data at scale, three things get in the way.

First, the rate limit. The official API allows 100 queries per minute per registered OAuth client id, averaged over a 10-minute window so short bursts are fine. Traffic without OAuth is blocked outright. If you are sweeping hundreds of subreddits or backfilling a comment corpus, 100 QPM is a real ceiling you have to engineer around.

Second, the price. Reddit no longer publishes a self-serve commercial rate on any live page. The widely cited $0.24 per 1,000 calls figure comes from Reddit's June 2023 announcement, repeated by blogs ever since, and is not on a current pricing page. The free tier is for eligible non-commercial use. For commercial terms you have to talk to Reddit, which means you cannot budget a build against a public number.

Third, the setup tax. You register an app at reddit.com/prefs/apps, receive a client id and secret, implement OAuth2, and send a descriptive User-Agent. PRAW hides most of that in Python, but it still inherits the app registration and the 100-QPM ceiling.

That leaves two routes for public data: reverse-engineer Reddit's own JSON endpoints yourself (which drift, throttle, and require rotating proxies), or use a Reddit scraper API that maintains the collection layer and hands you a stable contract. This guide takes the second route.

Is it legal to scrape Reddit?

Scraping publicly accessible pages has repeatedly survived legal challenge in US courts (the hiQ v. LinkedIn line of cases), and reading public Reddit posts sits on that precedent. That is not the whole story: Reddit's User Agreement and API Terms govern use of its content, comment text carries copyright, and the personal data inside posts and usernames brings GDPR and CCPA obligations that depend on your jurisdiction and use case. The honest answer is longer than a paragraph, so read our legal and technical guide to social media scraping before you ship anything commercial, and talk to a lawyer if revenue depends on the answer. This section is a technical overview, not legal advice.

Setup: one key, no Reddit app

Install the only required dependency:

pip install requests

Get an API key at socialcrawl.dev, then Dashboard → API Keys. Keys look like sc_ plus 32 random bytes and are shown in full once, so copy it immediately. New accounts get 100 free credits, the subreddit endpoint costs 1 credit per page, and credits never expire. Keep the key out of your source:

export SOCIALCRAWL_API_KEY=sc_your_api_key_here

There is no Reddit developer account, no client id or secret, and no OAuth dance. One header does it.

Scraping a subreddit's posts

The endpoint is GET /v1/reddit/subreddit. It takes one required parameter, subreddit (the name without the r/ prefix), plus optional sort (best, hot, new, top, rising), timeframe (day, week, month, year, all), and after for pagination. When you pass a timeframe, the API auto-selects sort=top, because the upstream only honors a time window with top sorting.

import os
import requests

API_KEY = os.environ["SOCIALCRAWL_API_KEY"]
BASE = "https://www.socialcrawl.dev/v1"

resp = requests.get(
    f"{BASE}/reddit/subreddit",
    params={"subreddit": "webdev", "sort": "top", "timeframe": "week"},
    headers={"x-api-key": API_KEY},
    timeout=30,
)
payload = resp.json()

Here is a real, trimmed response from that exact call (one item shown):

{
  "success": true,
  "platform": "reddit",
  "endpoint": "/v1/reddit/subreddit",
  "data": {
    "items": [
      {
        "post": {
          "id": "1v0ovvg",
          "url": "https://www.reddit.com/r/webdev/comments/1v0ovvg/why_do_urls_use_after_http_and_https_is_there_a/",
          "content": {
            "text": "Why do URLs use // after http: and https:? Is there a historical reason?",
            "media_urls": "https://i.redd.it/zs412pvag6eh1.jpeg"
          },
          "author": { "username": "daddieslittleson" },
          "engagement": { "likes": 675, "comments": 232 },
          "flags": { "nsfw": false, "spoiler": false, "deleted": false },
          "published_at": "2026-07-19T12:07:57.000Z",
          "ext": { "subreddit": "webdev" }
        },
        "computed": { "language": "en", "content_category": "other" }
      }
    ]
  },
  "credits_used": 1,
  "credits_remaining": 99
}

Two things worth noting before you write the loop. The post shape is canonical: content.text, author.username, engagement.likes, engagement.comments, published_at are the same field names you read on the Instagram, YouTube, and TikTok post endpoints, so a parser you write here transfers unchanged. And engagement.likes is Reddit's net score (upvotes minus downvotes), surfaced under the unified name so your cross-platform code does not special-case Reddit.

Paginating through a subreddit

The response carries an after token. You pass it back as the after query parameter on the next request, verbatim, and stop when it is missing.

def scrape_subreddit(name: str, max_pages: int = 20) -> list[dict]:
    """Fetch posts across a subreddit, one page per credit."""
    posts, after = [], None
    for _ in range(max_pages):
        params = {"subreddit": name, "sort": "new"}
        if after:
            params["after"] = after  # pass the token back verbatim
        resp = requests.get(
            f"{BASE}/reddit/subreddit",
            params=params,
            headers={"x-api-key": API_KEY},
            timeout=30,
        )
        payload = resp.json()
        if not payload.get("success"):
            raise RuntimeError(payload["error"]["message"])
        posts.extend(payload["data"]["items"])
        after = payload["data"].get("after")
        if not after:
            break  # no more pages
    return posts


print(len(scrape_subreddit("webdev")), "posts")

The max_pages guard matters: a busy subreddit paginates a long way, and each page is a credit. Decide your budget up front and encode it.

The same call in Node and curl

The endpoint, parameters, and envelope are identical across languages. Node:

const res = await fetch(
  "https://www.socialcrawl.dev/v1/reddit/subreddit?subreddit=webdev&sort=top&timeframe=week",
  { headers: { "x-api-key": process.env.SOCIALCRAWL_API_KEY! } },
);
const { data } = await res.json();
console.log(data.items.length, "posts, next page token:", data.after);

And the one-liner for a quick check:

curl "https://www.socialcrawl.dev/v1/reddit/subreddit?subreddit=webdev&sort=new&after=t3_1v0ovvg" \
  -H "x-api-key: $SOCIALCRAWL_KEY"

Scraping comment threads

Posts are half the story. To pull a thread, pass a post URL to GET /v1/reddit/post/comments. This call expands the comment tree in a single request and costs 5 credits (it does the work of many raw Reddit calls). It paginates with a cursor parameter, and the response carries a pagination object telling you whether more is available.

def scrape_comments(post_url: str, max_pages: int = 5) -> list[dict]:
    comments, cursor = [], None
    for _ in range(max_pages):
        params = {"url": post_url}
        if cursor:
            params["cursor"] = cursor
        payload = requests.get(
            f"{BASE}/reddit/post/comments",
            params=params,
            headers={"x-api-key": API_KEY},
            timeout=30,
        ).json()
        comments.extend(payload["data"]["items"])
        cursor = payload.get("pagination", {}).get("next_cursor")
        if not cursor:
            break
    return comments

If you want to search rather than walk a single subreddit, GET /v1/reddit/search runs a site-wide keyword sweep, and the omni-search endpoint expands the top threads' comments in the same call. Reddit video posts have spoken words too: GET /v1/reddit/post/transcript returns a transcript for 10 credits.

Handling errors and rate limits

Failures use the same envelope with success: false and a typed error object. The types you will actually meet scraping Reddit:

Error type	Status	What it means	Your move
RESOURCE_NOT_FOUND	404	Subreddit banned, private, or typo'd	Skip it. Empty upstream refunds the credit.
UPSTREAM_ERROR	502	Reddit hiccuped	Retry with backoff; credit refunded.
SERVICE_UNAVAILABLE	503	Circuit breaker open	Honor the Retry-After header.
CONCURRENCY_LIMIT	429	More than 50 concurrent requests / key	Cap your worker pool below 50.
INSUFFICIENT_CREDITS	402	Balance below the call cost	Top up.

Note what is not there: a queries-per-minute quota. Unlike the official API's 100-QPM ceiling, the only throughput constraint here is 50 concurrent requests per key. And empty-upstream 404s, 502s, and 503s automatically refund the credit, so a retry wrapper does not double-spend. The full matrix is in the error handling docs.

Scraping Reddit at scale: the credits math

The math is flat enough for a napkin. The subreddit endpoint is 1 credit per page; threaded comments are 5.

Workload	Requests	Credits
One subreddit, first page (~25 posts)	1	1
One subreddit, 20 pages (~500 posts)	20	20
500 posts + comment threads on the top 50	20 + 50	~270
100 subreddits × 10 pages	1,000	1,000

The free tier's 100 credits cover the first rows. Past that, pricing is pay-as-you-go and credits never expire: 2,500 credits is £15, and the Growth plan's 20,000 (£49) funds the 100-subreddit sweep twenty times. Repeated requests for the same subreddit inside the cache window cost 0 credits, so you re-run your parser against cached pages for free while developing. For a marketing-flavored view of the same data, the free Reddit marketing tool surfaces subreddit and thread signals in your browser with no code.

Frequently asked questions

How do I scrape Reddit without the official API?

Use a third-party Reddit API that runs the collection layer for you. GET /v1/reddit/subreddit takes a subreddit name and an x-api-key header and returns posts in a unified schema, with no app registered at reddit.com/prefs/apps, no OAuth, and no PRAW. You paginate with the after token and pull comment threads from GET /v1/reddit/post/comments.

Is scraping Reddit legal?

Reading public Reddit data has meaningful precedent in its favor, but it depends on jurisdiction, data type, and access method, and Reddit's User Agreement and API Terms govern use of its content. Comment text carries copyright and usernames are personal data under GDPR and CCPA. Check Reddit's current terms and consult qualified counsel for commercial use. This is a technical overview, not legal advice.

What are the Reddit API rate limits?

The official Reddit Data API allows 100 queries per minute per registered OAuth client id, averaged over a 10-minute window, and blocks traffic without OAuth. A third-party API removes that ceiling: SocialCrawl charges per credit with no queries-per-minute cap, and the only throughput constraint is 50 concurrent requests per key.

Is SocialCrawl a good PRAW alternative?

Yes, for reading public Reddit data. PRAW is a Python wrapper around the official API, so it still needs a registered app and OAuth and inherits the 100-QPM limit. SocialCrawl returns subreddit feeds, threaded comments, and search from one x-api-key with no OAuth, in a schema that matches every other platform.

How much does it cost to scrape Reddit?

The subreddit endpoint costs 1 credit per page and threaded comments cost 5. New accounts get 100 credits that never expire, then pricing is pay-as-you-go from £15 for 2,500 credits. The official API publishes no live commercial rate, so third-party pricing is the only number you can actually budget against.

Can I scrape Reddit comments in one call?

Yes. GET /v1/reddit/post/comments takes a post URL and expands the comment tree server-side in a single request for 5 credits, paginating with a cursor parameter. You do not walk the tree yourself or make one call per comment.

Does the same code scrape other platforms?

Almost. Reddit posts come back in the same canonical shape as Instagram, YouTube, TikTok, and Facebook posts (content.text, author, engagement.likes, published_at), so your parsing and export code transfers. The differences are the input parameter and the pagination token name per platform, both listed in the pagination docs.

---

Where to go from here

You now have a Reddit scraper that costs 1 credit per page and survives Reddit's frontend and pricing changes because maintaining that compatibility is the API's job, not yours. Next steps: browse the other endpoints on the Reddit platform page, read the Reddit API key and limits guide for the official-key details, or compare providers in best Reddit data APIs (2026). Start with the free tier: 100 credits, no card is about a hundred pages of posts, more than enough to find out whether the data answers your question.