SocialCrawl

Lanes

The platforms a listener can watch, how each orders what it returns, the shortest schedule it takes, its page cap and the release that first fetches it

A lane is the search a listener runs on one platform. The dry run prices every lane below today; the last column says which release first fetches it on a schedule. Pick lanes with platforms and, where a platform has more than one, search_in.

cURL
curl -X POST "https://www.socialcrawl.dev/v1/listening/streams?dry_run=1" \
  -H "x-api-key: YOUR_API_KEY" -H "Content-Type: application/json" \
  -d '{"keywords": ["Acme"], "platforms": ["reddit", "naver"], "search_in": {"reddit": ["posts", "comments"], "naver": ["news"]}, "schedule": {"every": "1h", "name": "Acme"}}'

Each lane comes back in data.stream.lanes with its ordering and label, and in data.estimate.by_platform with its cost.

The launch lanes

Platform (id)search_inLaneOrderShortest everyPages per searchFirst fetched in
Reddit (reddit)postsreddit/searchNewest first30m3LRel2a
YouTube (youtube)postsyoutube/search/advancedTop results only1h3LRel2a
Reddit (reddit)commentsreddit/search/commentsNewest first30m3LRel2b
X (twitter, also x)poststwitter/search/tweetsNewest first30m3LRel2b
Naver (naver)newsnaver/news/searchNewest first30m3LRel2b
Naver (naver)blognaver/blog/searchTop results only1h3LRel2b
Google News (news)postsgoogle_news/searchTime window1h1LRel2b
TikTok (tiktok)poststiktok/searchTop results only1h1LRel2b
Facebook (facebook)postsfacebook/search/postsTop results only1h2LRel2b
Threads (threads)poststhreads/searchTop results only1h1LRel2b
LinkedIn (linkedin)postslinkedin/search/postsTop results only6h3LRel2b

Naver searches blog and news when you name neither. Every other platform searches posts by default. Each lane costs its normal API price per page, listed on endpoint pricing.

What does the order mean?

  • Newest first. The platform returns the newest posts first. These lanes take every: "30m". Once listeners run (LRel2a), a check reads pages until it reaches a post the listener has already seen, up to the page cap, and a page with no new post is refunded (pricing).
  • Top results only. The platform returns what it ranks highest, not every new post. A check reads its pages and keeps the posts it has not seen before, so it samples. These lanes take 1h or slower (6h on LinkedIn), and every page is billed.
  • Time window. Google News returns one page per refresh window: title, URL, publisher and time, with no article body and not every article. It is not a media-monitoring feed.

A lane counts as newest first only once its order has been measured on production. YouTube and Naver blog are read as top results until their date order is shown to hold over three pages; the dry run prices them that way today.

What a listener will not do

  • It does not backfill. For posts from before it is created, run discover with since set to the start of the window you need, then create the listener, and deduplicate on evidence.id.
  • It does not watch Instagram: no Instagram lane is ready.
  • Comments under the posts it matches arrive in LRel3. The Reddit comments lane above is a keyword search of comments, which is a different job.
  • Threads is not ready for Korean (Hangul) or Malay keywords. A group in those is refused for Threads with why: "lane_not_ready_for_language" and a next_step naming the lanes that are ready.

On this page