Collect every review and watch for new ones
Pull a business's review history once, then collect only the reviews written since, with a bulk job that refreshes itself from the earlier job
Send the businesses you care about, and get back their reviews as one file. A week later, send the earlier job's id and get only the reviews written since. The example reads two Yelp businesses, and the same steps work for Tripadvisor places, App Store apps and Google Play apps.
Measured on 22/11/2026: a bulk job on 2 Yelp businesses with since=2026-11-01 read 6 reviews in 9 pages and was charged 3 credits. A refresh of that job right after it found 0 new reviews and was charged 3 credits.
Start in five minutes
Set your key once. Every block on this page reads it.
export SOCIALCRAWL_API_KEY=sc_your_key_herePaste this prompt into your coding agent:
Read https://www.socialcrawl.dev/docs/recipes/every-review-and-new-ones.md and follow it to collect the Yelp reviews of the businesses la-folie-san-francisco-2 and zj8Lq1T8KIC5zwFief15jg written since 2026-11-01. Use the key in SOCIALCRAWL_API_KEY. Check the price in the roster first and show me what the job reserves. Send an Idempotency-Key, wait for the job with the status route at most once a minute, and never raise max_reviews_per_item without asking me.Or read the newest page of one business yourself, which needs no job:
curl -G "https://www.socialcrawl.dev/v1/yelp/business/reviews" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
--data-urlencode "id=zj8Lq1T8KIC5zwFief15jg" \
--data-urlencode "limit=49" \
--data-urlencode "sort=newest"How it works
- A single page call returns the newest reviews of one business, newest first.
- A bulk job collects the history of many businesses in the background and delivers it as files.
- You read the job's status for free until it ends, then download one combined file.
- You store each review once by its id.
- Next week you start a new job with
refresh_ofset to the earlier job, and it collects only what is newer.
What you need
| Call | What it gives | Credits |
|---|---|---|
GET /v1/yelp/business/reviews | One page of a business's reviews, newest first | Billed per page, see the Yelp page |
GET /v1/tripadvisor/reviews | A place's newest reviews, as many as depth asks | See the Tripadvisor page |
POST /v1/yelp/business/reviews/history | One business's whole history as a background job | See the Yelp page |
POST /v1/prism/reviews/bulk | The history of many businesses in one job | Reserved up front, settled to the pages read, see the roster |
| The job's status route (Step 4) | The job's progress, then its download links | Free to read |
The same bulk job takes Tripadvisor places, App Store apps and Google Play apps through its platform field.
Steps
Step 1: read the newest page
Send sort=newest and limit=49 for the largest page. The reviews come back newest first, each with its id, rating, text, date and the owner's reply when there is one.
curl -G "https://www.socialcrawl.dev/v1/yelp/business/reviews" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
--data-urlencode "id=zj8Lq1T8KIC5zwFief15jg" \
--data-urlencode "limit=49" \
--data-urlencode "sort=newest" \
--data-urlencode "since=2026-11-15"import json, os, urllib.parse, urllib.request
query = {"id": "zj8Lq1T8KIC5zwFief15jg", "limit": "49", "sort": "newest", "since": "2026-11-15"}
url = "https://www.socialcrawl.dev/v1/yelp/business/reviews?" + urllib.parse.urlencode(query)
req = urllib.request.Request(url, headers={"x-api-key": os.environ["SOCIALCRAWL_API_KEY"]})
page = json.load(urllib.request.urlopen(req, timeout=120))
for row in page["data"]["items"]:
print(row["review"]["published_at"], row["review"]["rating"]["value"], row["review"]["id"])sincestops the walk at the first older review, becausesort=newestis in date order.- On 22/11/2026 a page without
sinceheld 49 reviews from 21/11/2026 back to 04/06/2026 for 1 credit. Withsince=2026-11-15it held 2 reviews, ended withstopped_at: sinceand cost 1 credit. - For Tripadvisor,
GET /v1/tripadvisor/reviewstakes the place'surl_pathand adepth. On 22/11/2026depth=40returned 40 reviews of one landmark for 1 credit, andhas_morewasfalse.
Step 2: start a history job for one business
For a single business with a long history, send its id or link to the history route.
curl -X POST "https://www.socialcrawl.dev/v1/yelp/business/reviews/history" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
-H "content-type: application/json" \
-H "Idempotency-Key: reviews-history-1" \
-d '{"id": "zj8Lq1T8KIC5zwFief15jg", "since": "2024-11-01"}'It answers 202 with a job_id. Submitting holds the most the request can read and the job settles to the pages it read. Read the reviews 100 at a time with the history route with the job id added to its path, and offset and limit as query parameters, which costs 0 credits. This page's runs recorded the bulk job in step 3 instead, so this step has no measured run.
Step 3: collect many businesses in one bulk job
curl -X POST "https://www.socialcrawl.dev/v1/prism/reviews/bulk" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
-H "content-type: application/json" \
-H "Idempotency-Key: reviews-bulk-1" \
-d '{"platform": "yelp", "items": ["la-folie-san-francisco-2", "zj8Lq1T8KIC5zwFief15jg"], "since": "2026-11-01"}'itemsis a JSON array of business ids, aliases or links. Send a real array, not a string.sinceanduntiltake a UTC day asYYYY-MM-DD. Any other field in the body is a free400.- The job answers
202and reserves credits in blocks, never the whole job at once. On 22/11/2026 the reservation was 25 credits. - An
Idempotency-Keymakes a retried submit return the same job without a second reservation. Retry safely explains the header.
Step 4: follow the job
Read the status route, GET /v1/prism/reviews/bulk/ followed by the job id, about once a minute until data.job.status is no longer queued or running. Reading is free.
JOB_ID=job_your_job_id # data.job_id from Step 3
API=https://www.socialcrawl.dev/v1
curl -sS "$API/prism/reviews/bulk/$JOB_ID?limit=10" \
-H "x-api-key: $SOCIALCRAWL_API_KEY"{
"data": {
"job": {
"job_id": "job_JLdwzTaMDkww1OKlX9D3E",
"status": "succeeded",
"items_done": 2,
"items_total": 2,
"pages_read": 9,
"reviews_collected": 6,
"credits_charged": 3
},
"combined_url": "https://...",
"manifest_url": "https://..."
}
}- The job ends as
succeededand settles to the pages it read. The 25 credits it reserved became 3 charged. - On 22/11/2026 each job took a few minutes, so read the status once a minute and do not poll faster.
- Download links are minted fresh on every read and expire after an hour, so read the status again when a link has expired.
combined_urlisnullwhen the job found no reviews.
Step 5: store each review by its id
The combined file is gzipped JSON lines. The first line of each business has type: business, and every review line has type: review, the item it belongs to and a review object with its id. Insert each review on its id and skip the ones you hold.
Step 6: refresh next week
Start a new job with the same platform and items and add refresh_of with the earlier job's id.
JOB_ID=job_your_first_job_id # data.job_id of the earlier job
curl -X POST "https://www.socialcrawl.dev/v1/prism/reviews/bulk" \
-H "x-api-key: $SOCIALCRAWL_API_KEY" \
-H "content-type: application/json" \
-H "Idempotency-Key: reviews-bulk-2" \
-d '{"platform": "yelp", "items": ["la-folie-san-francisco-2", "zj8Lq1T8KIC5zwFief15jg"], "refresh_of": "'"$JOB_ID"'"}'It reads the same items again, from the day the earlier job finished. The earlier job must still be kept, which is 7 days from its submit. On 22/11/2026 the refresh ran right after the first job, so it found 0 new reviews and was charged 3 credits.
Store only what is new
The review id is the key, and the newest stored date is your watermark. Save the id of each finished job next to your reviews, because the next refresh needs it. A job older than 7 days cannot be refreshed. Start a fresh job with since set to your newest stored date instead.
What it costs
A bulk job reserves credits in blocks and settles to the pages it read. A business that cannot be read costs nothing. The roster states the price of each call.
What the runs measured on 22/11/2026:
| Run | Reviews | Credits |
|---|---|---|
Yelp newest page, limit=49 | 49 | 1 |
Yelp newest page with since=2026-11-15 | 2 | 1 |
Tripadvisor, depth=40 | 40 | 1 |
Bulk job, 2 Yelp businesses, since=2026-11-01 | 6 | 3 charged, 25 reserved |
Refresh of that job with refresh_of | 0 | 3 charged, 25 reserved |
If you collect for many businesses every week, read Volume costs and email socialcrawl@ridio.io with your monthly volume.
Tips, and what not to use
- A job pauses when your balance cannot cover the next reservation. It keeps what it has delivered and resumes by itself once you add credits.
- A Tripadvisor place id that does not exist can answer an empty page instead of an error. Read the place from
GET /v1/tripadvisor/searchfirst. - A business with accented letters in its link is safest sent by its 22-character business id. Read the id from
GET /v1/yelp/search. - Do not use this for products. For product reviews, use
GET /v1/prism/product-reviews. - To pay only when something is new, use Listening or Monitors.
Full program and agent brief
Next steps
Poll an account daily
Fetch only the posts an account published since your last run, bounded by a date on the first run and by the newest post you hold on every run after
Reviews across sites
Find one business on Yelp, Google, Tripadvisor and Trustpilot and read its newest reviews with sentiment already labelled
