Pydantic AI Tools: Validate Real API Data, Not Toy JSON

A Pydantic AI agent's tools are only as trustworthy as the code inside them. Pydantic validates a tool's arguments against your function signature automatically.

It does nothing to check what a tool returns. The moment that tool calls a real API, you're on your own.

This post builds a Pydantic AI agent whose tools call SocialCrawl, a live social data API, and validates every response against a typed model: pydantic-ai>=2.31,<3, tested against 2.31.1 on 19/08/2026.

Five real calls, five real JSON payloads, including one field the API refused to publish because the number it computed wasn't a real engagement rate. Every response below is unedited, and every example is reproducible with your own key. No mock objects, no synthetic dicts, no toy JSON.

What does Pydantic AI validate, and what doesn't it check?

Pydantic AI validates two different things, and conflating them is the fastest way to ship a tool that looks safe and isn't.

Tool arguments are checked automatically: "its arguments are validated against the function's signature using Pydantic. If validation fails... the framework automatically generates a RetryPromptPart... sent back to the LLM" (docs). A bad call from the model becomes a repair instruction, not a stack trace.

Tool return values are not checked against your type annotation. "Tools can return anything that Pydantic can serialize to JSON" (docs), and nothing stops that from being a malformed dict with the wrong keys in it. If you want the response an upstream API actually sent to be validated, the tool body has to call Model.model_validate(payload) itself. That's the argument this post exists to prove, five real calls at a time.

One freshness check while we're here: output_type= is correct on 2.31.x. result_type= and result.data were removed in v0.6.0 (06/08/2025) (changelog). If a tutorial you're reading still shows either, it predates the current API by over a year.

Pydantic AI's distinctive combination is typed deps_type, RunContext[T], and output_type on one agent object, all checkable statically before anything runs. For where that sits next to LangChain and LangGraph, see our framework field guide rather than re-litigating it here.

How do you build a Pydantic AI agent that calls a live API?

pip install "pydantic-ai>=2.31,<3"

Tested against 2.31.1, 19/08/2026. uv add pydantic-ai works the same way if that's your tool of choice.

Every field on the model below exists somewhere in the two profile responses you're about to see. None of it is guessed ahead of time:

from dataclasses import dataclass

import httpx
from pydantic import BaseModel
from pydantic_ai import Agent, RunContext


class Author(BaseModel):
    id: str
    username: str
    display_name: str
    bio: str | None = None
    verified: bool | None = None
    followers: int
    following: int | None = None
    posts_count: int
    likes_count: int | None = None
    url: str | None = None
    private: bool | None = None
    joined_at: str | None = None
    ext: dict | None = None


@dataclass
class SocialCrawlDeps:
    api_key: str


agent = Agent(
    "openai:gpt-5.2",
    deps_type=SocialCrawlDeps,
    output_type=str,
    instructions="You look up creator profiles and describe what you find.",
)


@agent.tool
async def get_tiktok_profile(ctx: RunContext[SocialCrawlDeps], handle: str) -> Author:
    async with httpx.AsyncClient() as client:
        r = await client.get(
            "https://www.socialcrawl.dev/v1/tiktok/profile",
            params={"handle": handle},
            headers={"x-api-key": ctx.deps.api_key},
        )
        r.raise_for_status()
        payload = r.json()
        return Author.model_validate(payload["data"]["author"])

The key comes in through RunContext, not a module-level constant. Set it once as SOCIALCRAWL_API_KEY, then run the agent with agent.run_sync("Look up mrbeast on TikTok", deps=SocialCrawlDeps(api_key=os.environ["SOCIALCRAWL_API_KEY"])).

Here's what the SocialCrawl API actually returned for that call, on 19/08/2026:

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile",
  "data": {
    "author": {
      "id": "6614519312189947909",
      "username": "mrbeast",
      "display_name": "MrBeast",
      "bio": "Checkout My New Book!👇",
      "verified": true,
      "followers": 137654911,
      "following": 354,
      "posts_count": 467,
      "likes_count": 1400000000,
      "url": null,
      "private": false,
      "joined_at": null
    },
    "computed": {
      "engagement_rate": null,
      "language": null,
      "content_category": "other",
      "estimated_reach": null
    },
    "_warnings": [
      "computed.engagement_rate: author ratio exceeded 1.0 (raw: 10.17036); returned null — a lifetime likes/followers ratio is not a real engagement rate"
    ]
  },
  "credits_used": 1,
  "credits_remaining": 169326,
  "request_id": "req-yxwJV4VNJhzpul4L",
  "cached": false
}

computed.engagement_rate came back null, not a bogus number. The raw ratio was 10.17, lifetime likes divided by followers, which isn't a real engagement rate, and the API said so directly in _warnings instead of quietly shipping something that looks like a valid float. A tool that types engagement_rate: float | None and reads _warnings before trusting a non-null value is the whole pattern in one field.

Three nulls, three different reasons. language is null because a one-line, emoji-heavy bio can't be language-detected. url and joined_at are null because TikTok's profile endpoint just doesn't expose them. Same type, str | None, completely different cause, which is exactly what the next section is about.

Why do two platforms return such different nulls from the same Author model?

The tool for YouTube channels reuses the exact same Author model. Only the endpoint and the handle change:

@agent.tool
async def get_youtube_channel(ctx: RunContext[SocialCrawlDeps], handle: str) -> Author:
    async with httpx.AsyncClient() as client:
        r = await client.get(
            "https://www.socialcrawl.dev/v1/youtube/channel",
            params={"handle": handle},
            headers={"x-api-key": ctx.deps.api_key},
        )
        r.raise_for_status()
        payload = r.json()
        return Author.model_validate(payload["data"]["author"])

That's the whole payoff of typing the model once: the same Author.model_validate() call validates TikTok and YouTube, no platform-specific branch anywhere. Here's what came back for mkbhd, captured 19/08/2026:

{
  "success": true,
  "platform": "youtube",
  "endpoint": "/v1/youtube/channel",
  "data": {
    "author": {
      "id": "UCBJycsmduvYEL83R_U4JriQ",
      "username": "@mkbhd",
      "display_name": "Marques Brownlee",
      "verified": null,
      "followers": 21100000,
      "following": null,
      "posts_count": 1841,
      "likes_count": null,
      "url": "https://www.youtube.com/channel/UCBJycsmduvYEL83R_U4JriQ",
      "private": null,
      "joined_at": "2008-03-21",
      "ext": {
        "country": "US",
        "followers_approximate": true,
        "madeForKids": false,
        "hiddenSubscriberCount": false
      }
    },
    "computed": {
      "engagement_rate": null,
      "language": "en",
      "content_category": "other",
      "estimated_reach": null
    }
  },
  "credits_used": 1,
  "credits_remaining": 169324,
  "request_id": "req-XL6AZaE9dMhnQv9w",
  "cached": false
}

Compare it to the TikTok response. verified, following, likes_count, and private are all populated for MrBeast and all null here: YouTube's channel endpoint doesn't expose them the same way. And followers: 21100000 carries ext.followers_approximate: true. YouTube rounds subscriber counts above 1,000 to three significant figures (source), so that number is a rounded estimate, and the response says so instead of pretending precision it doesn't have.

This is the general shape of the problem, not a YouTube quirk. YouTube's API calls this field subscriberCount, TikTok's calls it follower_count, Instagram's calls it followers_count: three names, two casings, and one platform that only exposes it on a different API tier (TikTok, Instagram). One typed Author model, with the right fields marked optional, absorbs all of it.

How do you write Pydantic AI tools that return validated models, not raw dicts?

None of the top Pydantic AI examples on the web validate the response a tool receives from an upstream API. The closest attempt returns a bare dict, and in one case stringifies the whole record before handing it to the model. That inverts the framework's promise: the least trustworthy part of the chain, an LLM's paraphrase, ends up the only part that's schema-checked.

Here's the "before": a tool that returns whatever came back, unchecked.

@agent.tool
async def search_youtube_before(ctx: RunContext[SocialCrawlDeps], query: str) -> list[dict]:
    async with httpx.AsyncClient() as client:
        r = await client.get(
            "https://www.socialcrawl.dev/v1/youtube/search",
            params={"query": query, "includeExtras": "true"},
            headers={"x-api-key": ctx.deps.api_key},
        )
        r.raise_for_status()
        return r.json()["data"]["items"]

That runs fine. Nothing downstream knows a field is missing until it crashes on it. Here's the "after": typed models for the archetype this endpoint returns, validated before anything reaches the agent.

class PostContent(BaseModel):
    text: str
    media_urls: list[str] | None = None
    thumbnail_url: str | None = None
    duration_seconds: int | None = None


class PostAuthor(BaseModel):
    username: str
    display_name: str
    verified: bool | None = None


class Engagement(BaseModel):
    views: int | None = None
    likes: int | None = None
    comments: int | None = None
    shares: int | None = None
    saves: int | None = None


class Flags(BaseModel):
    nsfw: bool | None = None
    spoiler: bool | None = None
    pinned: bool | None = None
    deleted: bool | None = None


class Computed(BaseModel):
    engagement_rate: float | None = None
    language: str | None = None
    content_category: str | None = None
    estimated_reach: int | None = None


class Post(BaseModel):
    id: str
    url: str
    content: PostContent
    author: PostAuthor
    engagement: Engagement
    flags: Flags
    published_at: str
    ext: dict | None = None


class PostResult(BaseModel):
    post: Post
    computed: Computed


@agent.tool
async def search_youtube(ctx: RunContext[SocialCrawlDeps], query: str) -> list[PostResult]:
    async with httpx.AsyncClient() as client:
        r = await client.get(
            "https://www.socialcrawl.dev/v1/youtube/search",
            params={"query": query, "includeExtras": "true"},
            headers={"x-api-key": ctx.deps.api_key},
        )
        r.raise_for_status()
        return [PostResult.model_validate(item) for item in r.json()["data"]["items"]]

One real result, captured 19/08/2026 for the query "pydantic ai tutorial":

{
  "post": {
    "id": "zcYtSckecD8",
    "url": "https://www.youtube.com/watch?v=zcYtSckecD8",
    "content": {
      "text": "How to Build AI Agents with PydanticAI (Beginner Tutorial)",
      "media_urls": null,
      "thumbnail_url": "https://i.ytimg.com/vi/zcYtSckecD8/hq720.jpg?...",
      "duration_seconds": 2036
    },
    "author": {
      "username": "@daveebbelaar",
      "display_name": "Dave Ebbelaar",
      "verified": null
    },
    "engagement": { "views": 79564, "likes": 1770, "comments": 81, "shares": null, "saves": null },
    "flags": { "nsfw": null, "spoiler": null, "pinned": null, "deleted": false },
    "published_at": "2024-12-05T08:01:50-08:00",
    "ext": { "content_type": "video", "channel_id": "UCn8ujwUInbJkBhffxqAPBVQ" }
  },
  "computed": {
    "engagement_rate": 0.023264,
    "language": "en",
    "content_category": "other",
    "estimated_reach": 95477
  }
}

engagement.shares and engagement.saves are null on all 18 results in this search, every single one. That's not a gap in this particular video, it's structural: YouTube's Post archetype has no concept of shares or saves at this endpoint. A field typed int | None that is always None for a platform is something a tool needs to represent honestly, not silently default to 0, which would falsely claim "zero shares" instead of "not tracked here".

Comments are a different archetype entirely, and it shows in what computed carries:

class CommentAuthor(BaseModel):
    username: str
    display_name: str
    verified: bool | None = None


class CommentEngagement(BaseModel):
    likes: int
    replies: int


class CommentFlags(BaseModel):
    pinned: bool | None = None
    deleted: bool | None = None


class Comment(BaseModel):
    id: str
    url: str | None = None
    parent_id: str | None = None
    post_id: str
    text: str
    author: CommentAuthor
    engagement: CommentEngagement
    flags: CommentFlags
    published_at: str


class CommentComputed(BaseModel):
    language: str | None = None


class CommentResult(BaseModel):
    comment: Comment
    computed: CommentComputed

A real comment from the same video:

{
  "comment": {
    "id": "UgwCYYsrgXoc9Ly9fGB4AaABAg",
    "url": null,
    "parent_id": null,
    "post_id": "zcYtSckecD8",
    "text": "About the temperature, is setted in run. Example:\nselected_columns=select_columns_agent.run_sync(...)\nIt makes sense, as in your workflow you can need to use the same Agent many times running with different temperature",
    "author": {
      "username": "@marcopancotti8505",
      "display_name": "@marcopancotti8505",
      "verified": null
    },
    "engagement": { "likes": 16, "replies": 0 },
    "flags": { "pinned": null, "deleted": false },
    "published_at": "2025-02-02T13:06:08Z"
  },
  "computed": { "language": "en" }
}

comment.url is null on every comment on this page: YouTube's comment model has no permalink field the way Reddit does, so url: str | None is correct. But code that assumes every object has a working URL will crash or silently drop rows here. author.verified is likewise always null on commenters, because YouTube exposes verification on channels, not on the people commenting on them.

Three archetypes, one API key, three different computed shapes: Author gets engagement_rate, language, content_category, and estimated_reach; Post gets the same four; Comment gets language and nothing else. That's the argument for one typed model per archetype rather than a single loose schema stretched to cover all three.

Six sources, one query, four different engagement shapes

One call proves this at scale. /v1/search/everywhere fans a single query across multiple sources in one request, at a flat 20 credits regardless of how many actually return data:

from typing import Annotated, Literal

from pydantic import Field, TypeAdapter


class HNEngagement(BaseModel):
    source: Literal["hackernews"]
    points: int
    comments: int


class RedditEngagement(BaseModel):
    source: Literal["reddit"]
    score: int
    num_comments: int


class GitHubEngagement(BaseModel):
    source: Literal["github"]
    comments: int
    reactions: int


class TwitterEngagement(BaseModel):
    source: Literal["twitter-ai-search"]
    views: int
    likes: int
    reposts: int
    replies: int
    comments: int
    saves: int


SourceEngagement = Annotated[
    HNEngagement | RedditEngagement | GitHubEngagement | TwitterEngagement,
    Field(discriminator="source"),
]
engagement_adapter = TypeAdapter(SourceEngagement)


def parse_engagement(item: dict):
    return engagement_adapter.validate_python({"source": item["source"], **item["engagement"]})


@agent.tool
async def search_everywhere(ctx: RunContext[SocialCrawlDeps], query: str) -> list[dict]:
    async with httpx.AsyncClient(timeout=30) as client:
        r = await client.get(
            "https://www.socialcrawl.dev/v1/search/everywhere",
            params={
                "query": query,
                "lookback_days": 180,
                "sources": "hackernews,github,youtube,reddit,twitter-ai-search",
            },
            headers={"x-api-key": ctx.deps.api_key},
        )
        r.raise_for_status()
        return [
            {"title": item["title"], "url": item["url"], "engagement": parse_engagement(item)}
            for item in r.json()["data"]["items"]
        ]

For the query "pydantic ai agents" on 19/08/2026, that call actually ran across hackernews, github, youtube-hashtag, youtube, reddit, twitter-ai-search, six sources, sources_failed: {}, coverage: 1, 40 fused and ranked items. The top-ranked result, an HN thread on the framework itself:

{
  "candidate_id": "https://ai.pydantic.dev",
  "source": "hackernews",
  "title": "Pydantic.ai: Python agent framework from Pydantic team",
  "url": "https://ai.pydantic.dev/",
  "source_items": [
    {
      "item_id": "43006835",
      "author": "alexdong",
      "published_at": "2025-02-10T23:47:19Z",
      "date_confidence": "low",
      "engagement": { "points": 5, "comments": 1 },
      "metadata": {
        "top_comments": [
          {
            "score": null,
            "excerpt": "Looks interesting but I do wonder what the use cases and advantages are here...",
            "author": "NomDePlum",
            "url": "https://news.ycombinator.com/item?id=43007167",
            "date": "2025-02-11T00:18:24.000Z"
          }
        ]
      }
    }
  ]
}

The reason this needed a discriminated union rather than one dict[str, int], straight from the same response:

Source	engagement shape
Hacker News	{points, comments}
Reddit	{score, num_comments}
GitHub	{comments, reactions}
X / twitter-ai-search	{views, likes, reposts, replies, comments, saves}

Code that assumes engagement.likes exists on every item breaks silently on Hacker News and GitHub the moment it runs. Validating each source into its own model at the boundary catches that at parse time, not three calls deep into someone else's rate limit.

One honest caveat: this is a single captured run, not a benchmark. Scores and titles will drift as new content gets published, and youtube-hashtag succeeded here but returned zero items, a legitimate empty result, structurally different from a source that never ran at all. A tool that can't tell those two states apart will misreport its own coverage. If a field ever arrives as the wrong type entirely, that's a ValidationError inside the tool, and what the model sees next is covered in the FAQ below.

How do you start using this?

Sign up for a free key with starting credits, then set it as SOCIALCRAWL_API_KEY. Every call in this post is reproducible with your own:

curl -H "x-api-key: $YOUR_KEY" "https://www.socialcrawl.dev/v1/tiktok/profile?handle=mrbeast"

The exact numbers will have moved on (MrBeast's follower count changes daily), but the response shape and the null pattern won't.

From there: the explorer lets you see a raw response before you write a model for it, which is worth doing before you commit to a schema. The TikTok and YouTube reference pages document the unified schema per endpoint, including which fields are always null on a given platform. Every parameter used in the code above (includeExtras, lookback_days, sources) is documented there too, so you're not guessing at what else a tool could ask for.

If you'd rather not write direct HTTP calls at all, MCP is the alternative transport: same schema, no httpx boilerplate. And if you want to turn any of this into a standing job instead of a one-off script, that's a different post: building an agent that monitors social accounts on a schedule.

Frequently asked questions

Does Pydantic AI validate the data a tool returns?

No, not automatically. Pydantic validates a tool's arguments against its function signature, but return values aren't checked against your type annotation: "Tools can return anything that Pydantic can serialize to JSON" (docs). If you want the upstream response itself validated, call Model.model_validate() on it inside the tool, which is what every example above does.

How do I pass an API key into a Pydantic AI tool?

Through deps_type and RunContext, not a global variable or a hardcoded string. Define a small dataclass holding the key, pass an instance via agent.run_sync(prompt, deps=SocialCrawlDeps(api_key=...)), and read it inside any tool as ctx.deps.api_key. Every example in this post reads that value from the env var SOCIALCRAWL_API_KEY.

What happens when a tool's arguments fail validation?

A ValidationError becomes a RetryPromptPart sent back to the model automatically, and each retry consumes one unit of the retry budget (Agent(retries=N), default 1) (docs). The same repair loop fires if your tool body raises ModelRetry deliberately, for example after an upstream 429. Exhaust the budget either way and the run raises UnexpectedModelBehavior instead of silently returning something wrong.

How is Pydantic AI different from LangChain?

Typed deps_type, RunContext[T], and output_type on one agent object, all checkable statically before anything runs, is the distinctive combination. Typed tools alone aren't unique to Pydantic AI: the OpenAI Agents SDK uses Pydantic-powered validation too (docs). LangGraph is a different shape entirely, a graph and state-machine model built around checkpointing. See our framework field guide for the fuller comparison.

How many credits does a call like this cost?

In this run, single-object calls (TikTok profile, YouTube channel, YouTube search, YouTube comments) cost 1 credit each. /v1/search/everywhere is a flat 20 credits regardless of how many of its sources actually return data. Total spend across all five calls in this post: 24 credits.