Glossary

What is a User-Agent header?

A User-Agent is an HTTP request header that identifies the client software making the request, such as a browser, bot, or app. APIs use it for analytics, debugging, and access control. Reddit, for instance, rejects requests that do not send a descriptive, unique User-Agent string.

Start building Read the API docs

When you scrape directly, a generic or stale User-Agent is one of the fastest ways to get blocked, so scrapers rotate realistic values alongside their proxies.

SocialCrawl sets and rotates the User-Agent for you upstream, so you never manage headers to stay unblocked; you just call the endpoint.

Do it with SocialCrawl

SocialCrawl sets a valid User-Agent upstream for you

curl "https://www.socialcrawl.dev/v1/reddit/subreddit?subreddit=python" \
  -H "x-api-key: sc_your_api_key_here"

See the endpoint reference

FAQ

Have a question? We got answers

Find answers to frequently asked questions about SocialCrawl's API, pricing, and capabilities.

Ask AI about SocialCrawl

Ask ChatGPT Ask Perplexity Ask Gemini Ask Claude Ask Grok

Related terms

Start building