SocialCrawl

Use SocialCrawl with Retool

Pull social media data into Retool without writing a server.

You can call SocialCrawl from a Retool app with a REST API resource. A query on that resource feeds a table. Setup takes three steps: create the resource, write the query, bind a table.

Before you start

Create an API key in the dashboard under API Keys. Enter it once in the resource's header settings, so every query uses it and no app page shows it. Prove the key with a call that costs nothing:

Check the key (0 credits)
curl "https://www.socialcrawl.dev/v1/credits/balance" \
  -H "x-api-key: sc_your_key_here"

A reply with "success": true and your balance in data.balance means the key works.

Connect

Create a REST API resource and set these fields:

FieldValue
Base URLhttps://www.socialcrawl.dev/v1
Headersx-api-key = your key

Then add a query on the resource with method GET, the path /tiktok/profile/videos, and the URL parameter handle set to mrbeast. Change the path and the parameters to call any other endpoint.

Read the answer

Every response has the same shape. Branch on success. A list puts its rows in data.items, and pagination.next_cursor is the token for the next page.

Response (trimmed)
{
  "success": true,
  "data": { "items": [{ "id": "7654638524729216287" }] },
  "pagination": { "next_cursor": "sc.eyJ2Ijoy…", "has_more": true },
  "credits_used": 1,
  "credits_remaining": 99
}

Bind a table to {{ query1.data.data.items }}, where query1 is your query's name. To read the next page, send next_cursor back as the cursor URL parameter, and stop when has_more is false. See Pagination.

When it stops

A failed query raises a notification for app editors by default. To show it to the people who use the app, add an event handler to the query's Failure section that shows a notification, and read {{ query1.error }} for the details. Retool sends no email.

When credits run out, the answer is a 402 with error.type set to INSUFFICIENT_CREDITS. Read error.message for the text and error.details.checkout_url for the link that opens checkout. When the link is absent, error.details.top_up_url opens the billing page. Run the query once with a failing key to see where your version places the body.

Never retry a 402, because it fails until credits are added. Retry a 429 after the seconds in the Retry-After header, and a 502, 503 or 504 after a short backoff. See Errors.

What a run costs

It depends on the endpoint, and some endpoints are priced by what they return. Endpoint pricing lists each one, and Credits explains cache hits and refunds. A query set to run when the page loads calls the API for every user who opens the app. GET /v1/utility/estimate quotes a call for 0 credits.

On this page