Caching

Every successful response from a cache-enabled SocialCrawl endpoint is cached, and the cache is shared. Cache hits are free, so caching is the main lever for controlling both spend and latency. This page covers the default behavior, how to read the cache signals, how long data stays fresh, and how to force a live fetch when you need one. A minority of endpoints disable caching in the registry and run live on every request; each endpoint's exact cache contract is stated on its own page and in llms.txt.

Caching is on by default

There is nothing to enable or configure. When you call any /v1/* endpoint:

1. If a warm copy exists for the same endpoint and the same parameters, you get it instantly for 0 credits (X-Cache: HIT).
2. If not, we fetch live from the source, bill the endpoint's normal cost (X-Cache: MISS), return the result, and store it for the next caller.

You never pay twice for the same data within its freshness window.

Reading the cache signals

Every response tells you whether it came from cache, in both the body and the headers:

Cache hits are free

A cache hit deducts 0 credits. No transaction row is written and your balance is unchanged. We already paid the upstream provider when we fetched the original, so charging again for the same bytes would be double-billing. (This is the same principle as idempotent replays, which are also free. See Credits.)

Freshness windows

How long an entry stays warm depends on the kind of data. After the window passes, the next call is a live MISS that re-warms the entry for the next caller:

Shorter windows on fast-moving data (comments, search) keep results current; longer windows on slower-moving data (profiles, analytics) maximise free hits. Single-video transcripts are immutable once published, so they stay warm for 30 days (empty transcripts are never cache-written).

These are the category defaults, and a handful of endpoints override them where the data justifies it. Instagram music search caches for 6 hours rather than the 2-minute search default because a music catalogue is stable for hours; Google Finance quotes cache for 60 seconds rather than the 30-minute analytics default because a live price is not; Google News search caches for 5 minutes. Around 35 endpoints do not cache at all. Do not assume a category default when you are budgeting a specific job — the effective window for a given endpoint is stated on that endpoint's page, and in llms.txt for machine consumers.

The cache is shared

The cache key is derived from the endpoint and its sorted, normalised query parameters — not from your API key and not from your account. Three consequences follow:

• Shared hits. Any identical request already served inside the freshness window comes back to you as a free hit, whether it was your own earlier call, another job on your account, or another customer's traffic. Public platform data is the same bytes for everyone, so we pay the upstream once and bill the fetch once.
• Freshness is a property of the entry, not of your traffic. "Nobody on my team has called this handle recently" does not mean you are getting a live fetch. When you need a guaranteed-live read, send Cache-Control: no-cache.
• Junk params do not bust it. Unknown query parameters are stripped before the key is built, so a cache-buster like ?_cb=12345 does not force a fresh call, and it cannot accidentally turn a free hit into a paid miss. The header below is the only supported way to bypass the cache.

Force a fresh fetch

When you need guaranteed-live data, send the standard Cache-Control: no-cache request header:

GET /v1/tiktok/profile?handle=charlidamelio HTTP/1.1
x-api-key: sc_...
Cache-Control: no-cache

The request skips the cache lookup and fetches live from the source. It is billed at the endpoint's normal cost (a forced MISS, so X-Cache: MISS and a non-zero X-Credits-Used), and the fresh result is written back to the cache. The pattern is "pay once to refresh, then ride free hits until the window expires."

Details worth knowing:

• The header affects only the single request it rides on. It never disables caching for your key.
• Only the no-cache directive triggers it. Cache-Control: no-store on its own does not. A composite value like no-store, no-cache does, and matching is case-insensitive.
• There is no query-parameter equivalent (for example ?fresh=1), by design, so caching cannot be defeated by accident or by a value hardcoded into an integration.
• Idempotency-Key takes precedence: a replayed idempotent request returns the stored response even if it also carries Cache-Control: no-cache. See Idempotent requests.

When should I bypass the cache?

Most workloads should leave caching on and let hits accrue. Reach for Cache-Control: no-cache only when staleness is a correctness problem, for example:

• Reconciliation or audit jobs that must reflect the source at the exact moment of the call.
• Reading data back immediately after a user action or webhook changed it upstream.
• Debugging a field-map or freshness question where you want to compare cached against live.

For everything else, reusing identical requests within the freshness window is the cheapest and fastest path.

Try it interactively

Cache-Control and Idempotency-Key are listed as optional headers on every endpoint in the API Reference "Try it" panel, so you can experiment with force-refresh against your own key and watch the X-Cache and X-Credits-Used headers change.