SocialCrawl

Use SocialCrawl with Pydantic AI

Give a Pydantic AI agent social media data, with the price shown before it spends.

Pydantic AI registers plain typed functions as tools, so two small functions give an agent SocialCrawl data and a Pydantic model types what it returns. SocialCrawl publishes no Pydantic AI package, and the wrapper below needs none.

Setup

Install Pydantic AI and requests, then set your keys. Create the SocialCrawl key in the dashboard under API Keys; it looks like sc_your_key_here.

Terminal
pip install pydantic-ai requests
export SOCIALCRAWL_API_KEY=sc_your_key_here
export ANTHROPIC_API_KEY=your_anthropic_key

Connect

Decorate each function with @agent.tool_plain. The type hints become the argument schema and the docstring becomes the description.

Python
import os

import requests

BASE = "https://www.socialcrawl.dev"
HEADERS = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]}


def estimate_cost(endpoint: str, params: dict[str, str | int]) -> dict:
    """Quote the credits a call will hold. Free. endpoint looks like tiktok/profile."""
    r = requests.get(
        f"{BASE}/v1/utility/estimate",
        headers=HEADERS,
        params={"id": endpoint, **params},
        timeout=30,
    )
    body = r.json()
    return body.get("data", body)


def call_endpoint(platform: str, resource: str, params: dict[str, str | int]) -> dict:
    """Run one SocialCrawl call, for example tiktok and profile with a handle."""
    r = requests.get(
        f"{BASE}/v1/{platform}/{resource}", headers=HEADERS, params=params, timeout=60
    )
    body = r.json()
    if body.get("success"):
        return body["data"]
    # Give the model the error type, not the whole envelope.
    err = body.get("error", {})
    return {"error": err.get("type"), "message": err.get("message"), "status": r.status_code}
from pydantic_ai import Agent, UsageLimits

agent = Agent(
    "anthropic:claude-sonnet-5-5",
    instructions=(
        "Call estimate_cost and report the hold before any call_endpoint. "
        "If a call returns INSUFFICIENT_CREDITS, tell the user and stop."
    ),
)
agent.tool_plain(estimate_cost)
agent.tool_plain(call_endpoint)

Read a profile and its posts

Quote the call yourself, then run the agent. request_limit is the step limit, and the run raises UsageLimitExceeded when it is reached.

Python
quote = estimate_cost("tiktok/profile", {"handle": "nasa"})
print("hold:", quote["hold"], "valid:", quote["valid"])

result = agent.run_sync(
    "Read the TikTok profile for nasa and its latest videos, then summarise them.",
    usage_limits=UsageLimits(request_limit=5),
)
print(result.output)

Keep spend in check

Call estimate_cost before any paid call and show the answer to the person who owns the key. request_limit caps the loop, and max_credits stops a list walk where you set it. A 402 means the account is out of credits. Tell the user it is an offer to top up and stop, because a retry cannot succeed. See Errors and Credits.

Typed results

Give the agent an output_type and the answer arrives as a validated model. The model fills the fields from the tool data, so a missing value comes back as None rather than a guess.

Python
from pydantic import BaseModel


class ProfileReport(BaseModel):
    handle: str
    followers: int | None
    recent_topics: list[str]


typed = Agent("anthropic:claude-sonnet-5-5", output_type=ProfileReport)
typed.tool_plain(estimate_cost)
typed.tool_plain(call_endpoint)

report = typed.run_sync(
    "Quote, then read the TikTok profile and latest videos for nasa.",
    usage_limits=UsageLimits(request_limit=5),
)
print(report.output)

Resources

On this page