SocialCrawl

Use SocialCrawl with Gemini

Give a Gemini agent social media data, with the price shown before it spends.

The Google GenAI SDK can call plain Python functions for the model, so two small functions give a Gemini agent SocialCrawl data. SocialCrawl publishes no Gemini package, and the SDK needs none.

Setup

Install the Google GenAI SDK and requests, then set your keys. Create the SocialCrawl key in the dashboard under API Keys; it looks like sc_your_key_here.

Terminal
pip install google-genai requests
export SOCIALCRAWL_API_KEY=sc_your_key_here
export GEMINI_API_KEY=your_gemini_key

Connect

Pass the functions in tools. The SDK builds each declaration from the type hints and the docstring, calls the function when the model asks, and sends the result back. The functions take params as a query string such as handle=nasa, which keeps the declaration simple. maximum_remote_calls is the step limit.

Python
import os

import requests

BASE = "https://www.socialcrawl.dev"
HEADERS = {"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]}


def estimate_cost(endpoint: str, params: str) -> dict:
    """Quote the credits a SocialCrawl call will hold. Free. Call it before call_endpoint.

    Args:
        endpoint: The endpoint as platform/resource, for example tiktok/profile.
        params: The call's query string, for example handle=nasa.
    """
    r = requests.get(
        f"{BASE}/v1/utility/estimate?id={endpoint}&{params}", headers=HEADERS, timeout=30
    )
    body = r.json()
    return body.get("data", body)


def call_endpoint(platform: str, resource: str, params: str) -> dict:
    """Run one SocialCrawl call. Paid, so quote it with estimate_cost first.

    Args:
        platform: For example tiktok.
        resource: For example profile or profile/videos.
        params: The call's query string, for example handle=nasa.
    """
    r = requests.get(
        f"{BASE}/v1/{platform}/{resource}?{params}", headers=HEADERS, timeout=60
    )
    body = r.json()
    if body.get("success"):
        return body["data"]
    err = body.get("error", {})
    return {"error": err.get("type"), "message": err.get("message"), "status": r.status_code}
from google import genai
from google.genai import types

client = genai.Client()
MODEL = os.environ.get("GEMINI_MODEL", "gemini-2.5-flash")

config = types.GenerateContentConfig(
    system_instruction="Call estimate_cost and report the hold before any call_endpoint. If a call returns INSUFFICIENT_CREDITS, tell the user and stop.",
    tools=[estimate_cost, call_endpoint],
    automatic_function_calling=types.AutomaticFunctionCallingConfig(maximum_remote_calls=5),
)

To declare tools by hand instead, a file from /agent-tools/ works after you rename input_schema to parameters, and you then run the function calls yourself.

Read a profile and its posts

Quote the call first, then let the model run it. The estimate is free and its hold is the most the call can take.

Python
quote = estimate_cost("tiktok/profile", "handle=nasa")
print("hold:", quote["hold"], "valid:", quote["valid"])

response = client.models.generate_content(
    model=MODEL,
    contents="Read the TikTok profile for nasa and its latest videos, then summarise them.",
    config=config,
)
print(response.text)

Keep spend in check

Call estimate_cost before any paid call and show the answer to the person who owns the key. maximum_remote_calls caps the loop, and max_credits stops a list walk where you set it. A 402 means the account is out of credits. Tell the user it is an offer to top up and stop, because a retry cannot succeed. See Errors and Credits.

Ground an answer in social data

Ask a question that needs current posts, and the model searches before it answers. Keep the quote in the prompt so it prices the search first.

Python
response = client.models.generate_content(
    model=MODEL,
    contents="Quote, then run a TikTok video search for espresso at home (tiktok/search, query=espresso at home). "
    "Summarise what the top videos say.",
    config=config,
)
print(response.text)

Resources

On this page