SocialCrawl

Use SocialCrawl with Dify

Pull social media data into Dify without writing a server.

You can call SocialCrawl from a Dify workflow with an HTTP Request node. One GET request returns structured data for the next node. Setup takes three steps: set the header, add the node, route the failure.

Before you start

Create an API key in the dashboard under API Keys. The HTTP Request node has its own authorization setting, so enter the key there and not in the URL or a prompt. Prove the key with a call that costs nothing:

Check the key (0 credits)
curl "https://www.socialcrawl.dev/v1/credits/balance" \
  -H "x-api-key: sc_your_key_here"

A reply with "success": true and your balance in data.balance means the key works.

Connect

Add an HTTP Request node. Set the method to GET and the URL to https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio. For authorization, choose API Key with a custom header, name it x-api-key, and enter your key.

To turn a whole endpoint into a tool the model can pick, Dify can also import an OpenAPI specification. Our specifications are listed in AI agents.

Read the answer

Every response has the same shape. Branch on success. A list puts its rows in data.items, and pagination.next_cursor is the token for the next page.

Response (trimmed)
{
  "success": true,
  "data": { "author": { "username": "charlidamelio" } },
  "credits_used": 1,
  "credits_remaining": 99
}

To read the next page of a list, call again with next_cursor as the cursor query parameter, and stop when pagination.has_more is false. See Pagination.

When it stops

Dify nodes have an error handling setting with a fail branch. Turn it on for the HTTP Request node and connect the fail branch to an end node that returns the message, so a refused call reaches the user as text instead of a stopped run.

When credits run out, the answer is a 402 with error.type set to INSUFFICIENT_CREDITS. Read error.message for the text and error.details.checkout_url for the link that opens checkout. When the link is absent, error.details.top_up_url opens the billing page.

Never retry a 402, because it fails until credits are added. The node's retry setting suits a 429 or a 502, 503 or 504, and not a 402. For a 429, wait the seconds in the Retry-After header. See Errors.

What a run costs

It depends on the endpoint, and some endpoints are priced by what they return. Endpoint pricing lists each one, and Credits explains cache hits and refunds. A workflow that runs per user message calls the API each time. GET /v1/utility/estimate quotes a call for 0 credits.

On this page