Testing Your Integration

Start with a zero-credit request. It confirms that your test key reaches SocialCrawl and lets you record the balance before any billed test.

curl --include "https://www.socialcrawl.dev/v1/credits/balance" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY"

{
  "success": true,
  "platform": "meta",
  "endpoint": "/v1/credits/balance",
  "data": {
    "balance": 100,
    "recent_deductions": 0
  },
  "credits_used": 0,
  "credits_remaining": 100,
  "request_id": "req-abc123",
  "cached": false
}

The balance value depends on your account. Check that the status is 200, success is true, credits_used is 0, and X-Credits-Used is 0. Keep the request_id in your test log.

There is no sandbox unless verified otherwise

SocialCrawl does not currently provide a separate customer sandbox contract. Tests use the normal API and follow the same billing, caching, and rate-limit rules as production requests.

Do not treat a successful fixture test as proof that an upstream source is available. Do not treat a successful real request as a permanent availability guarantee. Use fixtures for deterministic parser checks and one budgeted request to verify the current end-to-end path.

Use a separate capped test key

Create a key for development or pre-production and keep it separate from the production key. Store it in a server-side secret named SOCIALCRAWL_API_KEY. Do not put it in source control, browser code, test snapshots, or logs.

Set the test key's credit cap to the smallest amount that covers the planned requests. Record the starting balance, expected maximum charge, and ending balance. Rotate or revoke the key if it appears in test output. See Authentication for key handling and Credits for billing behavior.

Start with free and low-cost checks

Use GET /v1/credits/balance first. It costs 0 credits and does not call an upstream source.

Then make one real data request. This verifies authentication, transport, normalization, and your response parser against the current service. The following profile request has a current maximum cost of 1 credit, but a cache hit costs 0. Confirm the current cost in Endpoint pricing before adding it to an automated suite.

curl --include "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY"

Budget 1 credit for this check. Read X-Credits-Used and credits_used instead of assuming whether the response was billed. Keep repeated upstream checks out of unit and pull-request test loops.

Test response fixtures, nulls, and dropped rows

Store small response fixtures in your own test suite. Redact keys and avoid copying personal data that your test does not need. This synthetic fixture covers the five states your parser must distinguish:

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile/videos",
  "data": {
    "items": [
      {
        "post": {
          "id": "7351234567890123456",
          "url": "https://www.tiktok.com/@charlidamelio/video/7351234567890123456",
          "content": {
            "text": "Launch day",
            "media_urls": null,
            "thumbnail_url": null,
            "duration_seconds": null
          },
          "author": {
            "username": "charlidamelio",
            "display_name": null,
            "avatar_url": null,
            "verified": true
          },
          "engagement": {
            "views": 1200,
            "likes": 100,
            "comments": 12,
            "shares": null,
            "saves": null
          },
          "flags": {
            "nsfw": null,
            "spoiler": null,
            "pinned": false,
            "deleted": false
          },
          "published_at": "2026-09-01T12:00:00.000Z"
        },
        "computed": {
          "engagement_rate": null,
          "language": null,
          "content_category": null,
          "estimated_reach": null
        }
      }
    ],
    "dropped": 1,
    "_warnings": ["One source row could not be normalized."]
  },
  "credits_used": 1,
  "credits_remaining": 99,
  "request_id": "req-fixture-success",
  "cached": false,
  "pagination": {
    "next_cursor": null,
    "has_more": false,
    "page_size": 1
  }
}

Assert each state separately:

1. A present value, such as post.author.username, passed the endpoint schema.
2. computed.language: null means the canonical field exists but no defensible value is available.
3. An optional field may be absent. For example, this fixture omits post.ext. Test key existence before reading it.
4. data.dropped counts source rows rejected during list normalization. Record it for every page.
5. data._warnings contains advisory partial-data or transformation conditions. The response remains valid.

Follow the complete envelope rules in Response schema. A fixture proves that your parser handles a known shape. It does not prove current upstream availability or current source values.

For list endpoints, send pagination.next_cursor back unchanged as the cursor parameter. Stop only when pagination.has_more is false, equivalently when next_cursor is null. Test the one-page and multi-page cases with fixtures. See Pagination for the full contract.

Keep error fixtures too. Branch on error.type, not on message text.

{
  "success": false,
  "error": {
    "type": "RATE_LIMITED",
    "message": "Request rate limit exceeded",
    "status": 429,
    "doc_url": "https://www.socialcrawl.dev/docs/errors#rate-limited"
  },
  "credits_used": 0,
  "credits_remaining": 99,
  "request_id": "req-fixture-error"
}

if (!body.success) {
  const retryAfter = response.headers.get("retry-after");
  const retryableTypes = new Set([
    "RATE_LIMITED",
    "CONCURRENCY_LIMIT",
    "IDEMPOTENCY_IN_PROGRESS",
    "UPSTREAM_ERROR",
    "INTERNAL_ERROR",
    "SERVICE_UNAVAILABLE",
  ]);
  const retryAfterSeconds = Number(retryAfter);
  const hasTransientRetryAfter =
    retryAfter !== null &&
    Number.isFinite(retryAfterSeconds) &&
    retryAfterSeconds >= 0;
  const isTransient =
    retryableTypes.has(body.error.type) &&
    (body.error.type !== "SERVICE_UNAVAILABLE" || hasTransientRetryAfter);

  if (isTransient) {
    return retryAfterDelay(retryAfter);
  }

  throw new Error(`SocialCrawl request failed: ${body.request_id}`);
}

Add fixtures for every error type your integration handles. A SERVICE_UNAVAILABLE response is transient only when it carries Retry-After. A withdrawn endpoint returns the same type without that header and must not be retried. Use Errors for the complete contract and keep every retry loop bounded.

Test cache hits and forced misses

Run an identical request twice against a cache-enabled endpoint and inspect X-Cache on each response. An exact repeat may return HIT, but a hit is not guaranteed. The entry may be absent, expired, or unavailable for that endpoint.

curl --include "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY"

curl --include "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY"

When X-Cache is HIT, assert that cached is true and credits_used is 0. For a successful MISS, assert that cached is false and reconcile the normal charge with the endpoint price. A failed miss may be uncharged or refunded. Branch on error.type and trust the returned credits_used value instead of treating every miss as billable.

Test a forced miss only when a current upstream fetch is necessary. Cache-Control: no-cache skips the lookup and makes a normal billable request. Reserve the endpoint's full listed cost before running it.

curl --include "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  -H "Cache-Control: no-cache"

On a successful forced miss, assert X-Cache: MISS, cached: false, and the normal endpoint charge. If the request fails, assert billing from its error envelope instead. Cache-Control: no-store alone does not bypass the cache. See Caching for the complete request-header behavior.

Test retries with idempotency

Use an Idempotency-Key when a retry must not create a second charge or a second stateful operation. Send the same request twice with the same key.

IDEMPOTENCY_KEY="$(uuidgen)"

curl --include "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  -H "Idempotency-Key: $IDEMPOTENCY_KEY"

curl --include "https://www.socialcrawl.dev/v1/tiktok/profile?handle=charlidamelio" \
  -H "x-api-key: $SOCIALCRAWL_API_KEY" \
  -H "Idempotency-Key: $IDEMPOTENCY_KEY"

Generate the UUID once per test run and reuse it for exactly these two identical requests. The replay is scoped to the account and remains replayable for 24 hours. It reports credits_used: 0 and idempotent_replay: true. Reusing the key with a changed path, parameters, or body is rejected. Generate a new key for each distinct operation.

Do not add an idempotency key to a GET request that uses Accept: text/event-stream. GET server-sent event streams do not support replay keys. Request JSON when you need idempotent replay. See Response schema for storage limits and error cases.

Test webhook signatures and duplicate delivery

There is no webhook test-fire endpoint. Test the signing routine first with a fixed raw-body fixture, a known timestamp, and a test signing secret. Assert that your handler:

• verifies the HMAC against the exact raw request bytes before JSON parsing;
• rejects an invalid signature;
• rejects a timestamp outside the documented replay window;
• accepts the first valid delivery; and
• treats a repeated event or run identifier as a no-op.

Use the exact algorithm and replay-window guidance in Webhooks or Billing webhooks. Do not copy a re-serialized JSON body into the verifier.

Run one real delivery test only when its scheduled call or billing event is acceptable. Register a public HTTPS receiver, trigger a real event you control, and capture the raw body plus signature header. Replay that captured pair against your verifier offline. For Monitor and asynchronous run deliveries, record run_id before starting side effects. For billing deliveries, record the (event, created) pair. Treat a duplicate key as a no-op.

Clean up stateful test data

Delete only resources with a documented cleanup route. Verify each identifier and scope before sending a request.

• Cancel an unfinished web job with DELETE /v1/web/jobs/{job_id}.
• Stop future checks for a web monitor with DELETE /v1/web/monitors/{monitor_id}. Past checks remain readable.
• Close a browser session and settle its hold with DELETE /v1/web/sessions/{session_id}.
• Cancel a cohort query with DELETE /v1/cohort-queries/{queryId}.
• Delete a cohort and its members, queries, and results with DELETE /v1/cohorts/{cohortId}.

Do not invent a cleanup call for a stateless read. Keep billing receipts and request IDs in your own test record even after stateful data is removed.

Pre-production checklist

• The test key is separate from production and has a small credit cap.
• The suite lists each endpoint, its current price, and its maximum planned calls.
• Success and error fixtures cover values, null, optional absence, dropped rows, and warnings.
• Pagination tests stop on has_more: false and return the cursor unchanged.
• Retry tests branch on error.type, use bounded delays, and preserve the request cursor or body.
• Cache tests inspect X-Cache, cached, and credits_used.
• Forced-miss tests are explicitly budgeted as billable requests.
• Idempotency tests replay an identical request and reject changed payloads.
• Webhook tests verify the raw body, replay window, and duplicate-delivery handling where webhooks are used.
• Stateful test resources are cancelled, closed, or deleted through verified routes.
• Logs correlate status, request_id, error.type, cache state, credits, and retry count.
• Production secrets never enter the test environment or test artifacts.

Next, apply these checks to your release controls in the Production checklist.