SocialCrawl

AI Agent Integration

Machine-readable discovery files, Markdown twins, and the OpenAPI spec that let an agent learn the API on its own

SocialCrawl is built to be consumed by AI agents. Every response is the same JSON envelope, every endpoint takes one header, and the whole catalogue is published as machine-readable files an agent can read before it writes a single request.

If you only wire up one thing, point your agent at /llms-full.txt. It is the complete, self-contained reference.

Prerequisites

  • Nothing to install. Every discovery file below is public and needs no key.
  • An API key from socialcrawl.dev (Dashboard → API Keys) for any call that returns data. New accounts start with 100 free credits.
  • Using an MCP client or an agent skill instead? Skills & MCP is a one-line setup that skips the plumbing on this page.

When an endpoint supports a computed field and the required source inputs are present, its response includes that optional field for downstream reasoning. Depending on the endpoint, these fields can include engagement_rate, language, content_category, and estimated_reach. See Computed fields.

How do AI agents discover SocialCrawl?

Four machine-readable files describe the API, in increasing detail. Point your agent at /llms-full.txt for everything it needs: the entire 646-endpoint catalogue is included.

FileWhat it holds
/llms.txtCompact overview of the API
/llms-full.txtComplete self-contained reference, including the GET /v1/credits/balance meta endpoint, idempotency semantics, oneOf constraints, and every error code
/llms.jsonMachine-readable catalog: per-platform endpoint counts, auth, credit tiers, response envelope
/agents.txtAI agent access file: capabilities, quick start, and crawl guidance in one place

How do I read a docs page as Markdown?

Append .md to any docs URL:

You can also send Accept: text/markdown on the HTML URL. Every docs page carries an "AI agent or LLM? Read this page as markdown" link pointing at the same twin. Twins are served as text/plain, marked X-Robots-Tag: noindex, and listed in /sitemaps/llm.xml.

How can agents use the OpenAPI specification?

The full OpenAPI 3.1 specification is available in both formats:

What per-platform documentation is available?

Fetch only the endpoints you need. There is one file per active platform.

Each file contains only that platform's endpoints, with parameters, descriptions, and cURL examples. /llms.txt links the current list; treat that as the source of truth if this table ever trails a new platform launch.

How do I integrate SocialCrawl with my AI agent?

Read your key from the environment

Never inline a key in agent code or a prompt.

TypeScript
const apiKey = process.env.SOCIALCRAWL_API_KEY;
if (!apiKey) throw new Error("Set SOCIALCRAWL_API_KEY");

Call the endpoint with x-api-key

TypeScript
const response = await fetch(
  "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio",
  { headers: { "x-api-key": apiKey } },
);
const json = await response.json();

Branch on success before you hand anything to the model

TypeScript
if (!json.success) {
  // Machine-readable failure: branch on error.type, follow error.doc_url.
  throw new Error(`${json.error.type}: ${json.error.message}`);
}
// json.data is structured and ready for AI reasoning

Never hand a raw { "success": false, ... } envelope back to a model as if it were data. It will reason around the error instead of retrying or reporting it. Use error.type to decide what to do, and honour the Retry-After header on a 429 or 503. The full list is on the Errors page.

Troubleshooting

Official AI framework documentation

Which calls should an agent make before it writes code?

Three free calls replace guessing. Each costs 0 credits, answers in about 150 ms, and is generated from the endpoint registry at request time, so an agent that reads them never invents a path or a parameter.

cURL
# Find the endpoint for a job
curl "https://www.socialcrawl.dev/v1/utility/endpoints?search=comments" \
  -H "x-api-key: YOUR_API_KEY"

# Read its exact parameters, cost, paging rule, and an example call
curl "https://www.socialcrawl.dev/v1/utility/endpoint?id=tiktok/post/comments" \
  -H "x-api-key: YOUR_API_KEY"

# Confirm the key works and see the balance
curl "https://www.socialcrawl.dev/v1/credits/balance" \
  -H "x-api-key: YOUR_API_KEY"

Put those three lines at the top of any agent prompt or system instruction that touches SocialCrawl. The MCP server and the Agent Skill already do this before every request.

Next steps