# Web page schema (/docs/schema/web-page)



{/*
  AUTO-GENERATED. Do not hand-edit.
  Source: packages/social-api/src/docs/generate-schema-pages.ts
  Regenerate with: pnpm -F @repo/social-api generate:docs
  Field tables derive from the canonical Zod schemas (packages/social-api/src/schemas/canonical.ts);
  availability + equivalence derive from the registry fieldMaps.
  */}

Web page schema [#web-page-schema]

Every SocialCrawl endpoint that returns a web page gives you this exact shape, whatever the source platform. Write your parser once and the same code reads web page data from every platform below. That is the unified schema: one contract instead of a dozen raw upstream formats.

Field reference [#field-reference]

| Field                  | Type     | Nullable | Description                                          |
| ---------------------- | -------- | -------- | ---------------------------------------------------- |
| `url`                  | `string` | Yes      | Requested URL.                                       |
| `final_url`            | `string` | Yes      | Final resolved URL after redirects.                  |
| `status_code`          | `number` | Yes      | HTTP status code observed while fetching the page.   |
| `scrape_id`            | `string` | Yes      | Opaque scrape identifier for follow-up interactions. |
| `fetched_at`           | `string` | Yes      | Fetch timestamp when reported.                       |
| `content.markdown`     | `string` | Yes      | Markdown body.                                       |
| `content.html`         | `string` | Yes      | HTML body.                                           |
| `content.raw_html`     | `string` | Yes      | Raw HTML body.                                       |
| `content.summary`      | `string` | Yes      | Generated summary.                                   |
| `media.screenshot_url` | `string` | Yes      | Screenshot URL when requested.                       |
| `media.audio_url`      | `string` | Yes      | Audio URL.                                           |
| `media.video_url`      | `string` | Yes      | Video URL.                                           |
| `extraction`           | `object` | Yes      | Structured extraction result when requested.         |
| `fetch.cache_state`    | `string` | Yes      | Upstream cache state when reported.                  |
| `fetch.cached_at`      | `string` | Yes      | Upstream cache timestamp when reported.              |
| `fetch.proxy_tier`     | `string` | Yes      | Proxy tier used for the fetch.                       |

Platform availability [#platform-availability]

1 platform returns the `Web page` shape. **yes** means the platform populates the field (the value may still be null); a blank means the platform never provides it, so it is always null.

<div className="overflow-x-auto">
  | Platform     | `url` | `final_url` | `status_code` | `scrape_id` | `fetched_at` | `content.markdown` | `content.html` | `content.raw_html` | `content.summary` | `media.screenshot_url` | `media.audio_url` | `media.video_url` |
  | ------------ | ----- | ----------- | ------------- | ----------- | ------------ | ------------------ | -------------- | ------------------ | ----------------- | ---------------------- | ----------------- | ----------------- |
  | Web Scraping | yes   | yes         | yes           | yes         | yes          | yes                | yes            | yes                | yes               | yes                    | yes               | yes               |
</div>

Machine-readable schema [#machine-readable-schema]

Validate responses programmatically against the JSON Schema (2020-12):

[`/schemas/web-page.json`](/schemas/web-page.json)

Point a validator (Ajv, `jsonschema`, or your framework's) at that URL, or hand it to an agent so it can check the shape without a live call.

Returned by [#returned-by]

Endpoints with the `WebPage` / `WebPageList` archetype return this shape:

[Web Scraping](/platforms/web)

See how the same fields map to each platform's raw upstream names in the [cross-platform field equivalence table](/docs/schema/field-equivalence.md).