SocialCrawl

Wikipedia

Search Wikipedia articles in any language, then read one article in full as plain text

Wikipedia is the free encyclopedia, published as a separate Wikipedia for each language. These endpoints search articles in one language and read one article in full as plain text. Both use the canonical Post shape: the page id is post.id, the title is post.ext.title, and the Wikipedia-only facts sit in post.ext.wikipedia.

Base URL: /v1/wikipedia/...

Wikipedia exposes 2 routes here.

Quickstart

Search articles

Pass search terms as query. language picks the Wikipedia (en by default).

cURL
curl "https://www.socialcrawl.dev/v1/wikipedia/search?query=python%20programming&language=en" \
  -H "x-api-key: YOUR_API_KEY"

Read one article

Pass its title (with language when it is not English), or the article's link as url.

cURL
curl "https://www.socialcrawl.dev/v1/wikipedia/article?title=Python%20(programming%20language)" \
  -H "x-api-key: YOUR_API_KEY"
Response (trimmed)
{
  "success": true,
  "data": {
    "post": {
      "id": "23862",
      "url": "https://en.wikipedia.org/wiki/Python_(programming_language)",
      "content": { "text": "Python is a high-level, general-purpose programming language. ..." },
      "ext": {
        "title": "Python (programming language)",
        "wikipedia": { "language": "en", "is_disambiguation": false, "revision_id": "1386743958" }
      }
    }
  }
}

Questions people ask

Is the article text HTML or wikitext?

Neither. post.content.text is the article as plain text. Section headings stay on their own lines as == Heading ==, so you can split the text into sections.

What happens with a redirect or an ambiguous title?

A redirect is followed to the article it points to. A title with several meanings returns the disambiguation page, with post.ext.wikipedia.is_disambiguation true; pick the meaning you want from its text and read that article.

Can I read the same topic in another language?

Yes. Each language is its own Wikipedia, so pass language (for example de or ja) with that Wikipedia's title, or pass the article's link as url and the language comes from the link.

What does a title that does not exist return?

A 404, which is not charged.

All endpoints

2 endpoints available.

EndpointPathCredit Tier
Get a Wikipedia article as plain text/v1/wikipedia/articlestandard (1cr)
Search Wikipedia articles/v1/wikipedia/searchstandard (1cr)

How far back and how deep

Each list returns one page at a time. Send cursor for the next page, or max_items or max_pages to collect several pages in one call. A page size appears only where it has been verified. How paging works

PathRows per pageDate windowOne call collects
/v1/wikipedia/searchabout 10since and until, applied to each page fetchedup to 5,000 rows or 50 pages; limit up to 50

Platform notes

Content is published under Creative Commons. Wikipedia text is licensed CC BY-SA; credit Wikipedia and link the article when you republish it.

Times are ISO 8601 in UTC. post.ext.wikipedia.updated_at is when the page last changed.

Next steps