Computed fields

Most APIs hand you the raw numbers and walk away. SocialCrawl runs every payload through a transformer that attaches a computed block alongside the upstream data — the same shape on every platform, so a TikTok creator's engagement rate is directly comparable to an Instagram one.

Four fields live in computed:

Every value is either a real number or honestly null. We never substitute 0 for "we don't know" — the difference matters when you're sorting or filtering. If a value was forced into range (typically engagement_rate exceeding 1.0), an explanatory string lands in data._warnings so you can see it happened.

Computed fields attach to two archetypes:

• Author — data.computed on profile responses.
• Post — data.computed on single-post responses and on every item of a PostList.

CommentList, Audience, Transcript, and SearchResult archetypes don't carry a computed block.

engagement_rate

A normalised, comparable engagement signal in the range [0.0, 1.0]. Rounded to 6 decimals.

Author variant (profiles)

engagement_rate = author.likes_count / author.followers

Returns null when followers is 0 or likes_count is absent. "Zero engagement" and "we don't have the data" are different states; we report the second one honestly.

Instagram fallback. Instagram's profile payload never populates author.likes_count. When you call a profile endpoint on Instagram, we instead read up to ~12 recent posts embedded in the same response and compute:

engagement_rate = (mean(post_likes) + mean(post_comments)) / followers

A post counts only if both likes_count and comments_count are present. Zero usable posts → null. This fallback is Instagram-only today.

Post variant (single posts and PostList items)

engagement_rate = (likes + comments + shares) / views

Returns null when views is missing or 0. This is intentional — pre-views-era tweets and platforms that don't report views must be honest. The previous implementation fell back to divisor = 1, which silently surfaced the numerator (e.g. 26,573) as an engagement rate. That's worse than null.

One formula, everywhere. This is the same calculation on every platform. There is no per-platform variant of the post-level formula. What changes between platforms is not the formula, it's which inputs the platform exposes:

• likes, comments, shares are treated as 0 when absent. If a platform does not report a shares count (Instagram and YouTube do not), the numerator simply becomes likes + comments. We do not invent a shares number, and we do not null the whole rate just because one addend is missing. This is the single most common reason a hand-recomputed rate won't match ours: if you reproduce the formula assuming a non-null shares on Instagram or YouTube, your result will differ from the value we return, because on our side shares contributed 0.
• views is treated as null when absent, and a null divisor makes the whole rate null. Unlike the addends, a missing views is not coerced to 0 (dividing by zero is meaningless), so the rate is honestly null.

Why your recomputation might not match ours (per-platform)

If you validate our engagement_rate by recomputing it from the raw fields, this table explains every case of disagreement. In each case the value we return is correct; the mismatch comes from an input you may not have that we also don't (or vice versa).

The small share of rows where the raw math exceeds 1.0 are clamped to 1.0 with a _warnings note (see below); this is a fraction of a percent of rows and does not affect the population.

Clamping to [0, 1]

If the raw computation exceeds 1.0 (common when likes_count > followers on accounts that lost followers, or when an old tweet's likes count outweighs its under-reported views), the field is pinned to 1.0 and a warning is appended:

{
  "data": {
    "computed": { "engagement_rate": 1.0 },
    "_warnings": [
      "computed.engagement_rate: value exceeded 1.0 (raw: 1.42); clamped"
    ]
  }
}

The defensive lower clamp at 0 exists for the same reason but rarely fires — our arithmetic can't produce negatives from positive inputs.

language

ISO 639-1 two-letter code (en, ko, ja, pt-BR is not used — we emit pt).

• Input on Author: author.bio.
• Input on Post: post.content.text.
• Returns null if the input text is missing, non-string, or shorter than 10 trimmed characters. The 10-character floor is a confidence gate — anything shorter is more likely to misclassify than help you.

Detection strategy

Two passes:

1. Unicode fast-path for non-Latin scripts. If the input contains characters in these ranges, we return the language code directly:

   • Korean (가–힯, Hangul Jamo) → ko
   • Japanese (Hiragana / Katakana) → ja
   • CJK Unified Ideographs → zh
   • Arabic → ar
   • Devanagari → hi
   • Thai → th

2. Trigram classification via franc-min for everything else. franc returns und when below its internal confidence threshold; we map that to null rather than guessing.

Supported codes

If franc's ISO 639-3 code maps to one of the codes below, you get the two-letter form. Anything else collapses to null — we'd rather emit nothing than leak an obscure 3-letter code through the public surface.

ar  bg  ca  cs  da  de  el  en  es  fa  fi  fr  he  hi  hu
id  it  ja  ko  nl  no  pl  pt  ro  ru  sv  th  tr  uk  vi  zh

33 codes total. If you need a language we don't surface, let us know.

content_category

One of 14 hand-curated categories or "other". Returns null if the input text is missing or < 10 chars (same gate as language).

Matching rules

• Short keywords (≤3 characters, single word) — e.g. "ai", "tv", "dj" — require an exact token match. They match "building with ai" but not "hair" or "said".
• Longer or multi-word keywords use a Unicode word-boundary regex. "machine learning" matches inside "I love machine learning!" but doesn't bleed into adjacent words.
• The highest-scoring category wins. Ties resolve to whichever category was scored first (alphabetic by category name as authored in the source).

If you build product on top of content_category, treat it as a rough first-pass classifier, not a taxonomy. It's keyword-based — fast, deterministic, and noisy. For nuanced classification, layer your own model on top of the bio/text fields.

estimated_reach

A rough upper-bound estimate of how many distinct accounts a post or profile is reaching. Always an integer ≥ 0, or null.

Author variant

estimated_reach = round(followers * engagement_rate * 0.1)

The * 0.1 factor is a conservative damping based on typical impression-to-reach ratios. Because engagement_rate is clamped to [0, 1], the output is bounded by followers / 10 — there's no extra ceiling to apply.

Returns null whenever engagement_rate is null. We don't manufacture a reach number from a missing engagement signal.

Post variant

estimated_reach = round(views * 1.2)

For posts, views is already a stronger reach signal than impressions, so we apply a modest multiplier to estimate unique-account reach (assuming a small fraction of repeat views).

Returns null whenever views is missing or 0.

Caveats

This is a heuristic, not a measurement. It's useful for:

• Sorting posts or creators by approximate reach when the platform doesn't expose reach directly.
• Estimating order-of-magnitude impact for influencer outreach.

It is not useful for:

• Forecasting paid-media ROI.
• Comparing reach across platforms with very different view-counting rules (TikTok's auto-loop views vs YouTube's 30-second threshold, for example).

Example response

{
  "success": true,
  "platform": "tiktok",
  "endpoint": "/v1/tiktok/profile",
  "data": {
    "author": {
      "username": "mrbeast",
      "followers": 95000000,
      "likes_count": 8200000000,
      "bio": "I want to make the world a better place before I die."
    },
    "computed": {
      "engagement_rate": 1.0,
      "language": "en",
      "content_category": "lifestyle",
      "estimated_reach": 9500000
    },
    "_warnings": [
      "computed.engagement_rate: value exceeded 1.0 (raw: 86.32); clamped"
    ]
  },
  "credits_used": 1,
  "credits_remaining": 8431
}

(The likes_count / followers ratio is unrealistically high here because TikTok reports cumulative lifetime hearts against current followers — a known idiosyncrasy that triggers the clamp warning.)

When you shouldn't trust the value

Three patterns mean "look at the warnings before using this number":

1. _warnings mentions clamped — the raw value blew through [0, 1]. Decide whether you want the clamped value or to recompute from the raw upstream fields yourself.
2. computed.engagement_rate is null on a Post archetype — the post lacks a views field. Pre-2020 tweets, some Reddit endpoints, and a few Facebook surfaces are common offenders.
3. computed.language is null on a clearly-textual bio — the text was probably under 10 characters or was an emoji-only string. Read author.bio directly to confirm.

See also

• Response schema → Standard envelope — where computed sits in the response.
• Response schema → data._warnings[] — how warning strings are surfaced.
• Endpoint pricing — every endpoint and what each call costs.