Use SocialCrawl with AutoGen
Give an AutoGen agent social media data, with the price shown before it spends.
AutoGen can attach an MCP server to an agent through McpWorkbench, so the hosted SocialCrawl server gives an agent its tools with no function to write. SocialCrawl publishes no AutoGen package, and the one install line below is AutoGen's own.
Setup
Install AutoGen with its MCP and OpenAI extras, then set your keys. Create the SocialCrawl key in the dashboard under API Keys; it looks like sc_your_key_here.
pip install -U autogen-agentchat "autogen-ext[mcp,openai]"
export SOCIALCRAWL_API_KEY=sc_your_key_here
export OPENAI_API_KEY=your_openai_keyConnect
Point a workbench at the hosted server at https://mcp.socialcrawl.dev/mcp and send your key as a bearer header. The server brings a free estimate tool, socialcrawl_estimate, so the system message can ask for the price first. max_tool_iterations is the step limit.
import asyncio
import os
from autogen_agentchat.agents import AssistantAgent
from autogen_ext.models.openai import OpenAIChatCompletionClient
from autogen_ext.tools.mcp import McpWorkbench, StreamableHttpServerParams
server = StreamableHttpServerParams(
url="https://mcp.socialcrawl.dev/mcp",
headers={"Authorization": f"Bearer {os.environ['SOCIALCRAWL_API_KEY']}"},
)
SYSTEM = (
"Read public social media data with SocialCrawl. Call socialcrawl_estimate and report the cost "
"before any paid call. If a call says the account is out of credits, tell the user and stop."
)
async def main() -> None:
model_client = OpenAIChatCompletionClient(model="gpt-4.1-mini")
async with McpWorkbench(server) as workbench:
agent = AssistantAgent(
"researcher",
model_client=model_client,
workbench=workbench,
system_message=SYSTEM,
reflect_on_tool_use=True,
max_tool_iterations=5,
)
result = await agent.run(task="Read the TikTok profile for nasa and summarise it.")
print(result.messages[-1].content)
await model_client.close()
asyncio.run(main())Read a profile and its posts
The run above asks the agent to quote before it reads, so the estimate appears in the tool calls that result.messages records. Widen the task to ask for the latest videos as well, for example Read the TikTok profile for nasa and its latest videos, then summarise both.
Keep spend in check
Call estimate_cost before any paid call and show the answer to the person who owns the key. max_tool_iterations caps the loop, and max_credits stops a list walk where you set it. A 402 means the account is out of credits. Tell the user it is an offer to top up and stop, because a retry cannot succeed. See Errors and Credits.
Researcher and writer
Put a second agent in a round-robin team. Only the researcher gets the workbench, so only the researcher can spend credits, and the writer ends the run by saying DONE.
from autogen_agentchat.conditions import TextMentionTermination
from autogen_agentchat.teams import RoundRobinGroupChat
async def team_run() -> None:
model_client = OpenAIChatCompletionClient(model="gpt-4.1-mini")
async with McpWorkbench(server) as workbench:
researcher = AssistantAgent(
"researcher", model_client=model_client, workbench=workbench,
system_message=SYSTEM, reflect_on_tool_use=True, max_tool_iterations=5,
)
writer = AssistantAgent(
"writer", model_client=model_client,
system_message="Write a four-sentence brief from the researcher's notes, then say DONE.",
)
team = RoundRobinGroupChat([researcher, writer], termination_condition=TextMentionTermination("DONE"))
result = await team.run(task="Read the TikTok profile for nasa and brief me on it.")
print(result.messages[-1].content)
await model_client.close()
asyncio.run(team_run())