SocialCrawl

Use SocialCrawl with Mastra

Give a Mastra agent social media data, with the price shown before it spends.

Mastra agents take tools built with createTool, so two small tools over the REST API give an agent SocialCrawl data. SocialCrawl publishes no Mastra package, and the wrapper below needs none.

Setup

Install Mastra core and zod, then set your keys. Create the SocialCrawl key in the dashboard under API Keys; it looks like sc_your_key_here. The model key depends on the provider you name in model.

Terminal
npm install @mastra/core zod
export SOCIALCRAWL_API_KEY=sc_your_key_here
export ANTHROPIC_API_KEY=your_anthropic_key

Connect

createTool takes an id, a description, a zod inputSchema and an execute function that receives the validated input. The agent below gets both tools.

TypeScript
import { Agent } from "@mastra/core/agent";
import { createTool } from "@mastra/core/tools";
import { z } from "zod";

const API_KEY = process.env.SOCIALCRAWL_API_KEY;
if (!API_KEY) throw new Error("Set SOCIALCRAWL_API_KEY");

async function sc(path: string, params: Record<string, string | number>) {
  const url = new URL(`https://www.socialcrawl.dev${path}`);
  for (const [key, value] of Object.entries(params ?? {})) url.searchParams.set(key, String(value));
  const res = await fetch(url, { headers: { "x-api-key": API_KEY! } });
  const json = await res.json();
  if (!json.success) return { error: json.error.type, message: json.error.message };
  return json.data;
}

const paramsSchema = z.record(z.string(), z.union([z.string(), z.number()]));

export const estimateCost = createTool({
  id: "estimate_cost",
  description: "Quote the credits a SocialCrawl call will hold. Free. Call it before call_endpoint and tell the user the answer.",
  inputSchema: z.object({ endpoint: z.string().describe("platform/resource, e.g. tiktok/profile"), params: paramsSchema }),
  execute: async ({ endpoint, params }) => ({ quote: await sc("/v1/utility/estimate", { id: endpoint, ...params }) }),
});

export const callEndpoint = createTool({
  id: "call_endpoint",
  description: "Run one SocialCrawl call, for example platform tiktok and resource profile. Paid, so quote it first. A 402 means out of credits: tell the user and stop.",
  inputSchema: z.object({ platform: z.string(), resource: z.string(), params: paramsSchema }),
  execute: async ({ platform, resource, params }) => ({ data: await sc(`/v1/${platform}/${resource}`, params) }),
});

export const agent = new Agent({
  id: "social-researcher",
  name: "social-researcher",
  instructions:
    "Read public social media data with SocialCrawl. Call estimate_cost and report the hold before any call_endpoint.",
  model: "anthropic/claude-sonnet-5-5",
  tools: { estimateCost, callEndpoint },
});

Read a profile and its posts

Quote the call yourself, then run the agent. maxSteps is the step limit.

TypeScript
console.log(await sc("/v1/utility/estimate", { id: "tiktok/profile", handle: "nasa" }));

const response = await agent.generate(
  "Read the TikTok profile for nasa and its latest videos, then summarise them.",
  { maxSteps: 5 },
);
console.log(response.text);

Keep spend in check

Call estimate_cost before any paid call and show the answer to the person who owns the key. maxSteps caps the loop, and max_credits stops a list walk where you set it. A 402 means the account is out of credits. Tell the user it is an offer to top up and stop, because a retry cannot succeed. See Errors and Credits.

Resources

On this page