Running a Bulk Routine Batch from a JSONL File with the Clay CLI

clay routines runs start --bulk rows.jsonl runs a routine over a file of rows. A 3-row Company Domain batch completed in a few polls and spent 3 credits, the same as the estimate computed before it started.

What you will build

A script that estimates the batch cost, checks the balance, starts a bulk run, waits for it, downloads the results, and prints the credits spent.

estimate_batch  ->  require_credits  ->  runs start --bulk
                                              ↓
                                    poll runs get until status != in_progress
                                              ↓
                                 resultUrl -> curl -> JSONL results -> credits_spent

AI Prompt

Using the Clay CLI, run a routine over a JSONL file and report what it cost.

Requirements:
- Each line of the file is `{ "id": "<string>", "inputs": { ... } }`. Inline runs accept up to 100 items; larger batches use `--bulk`.
- `clay routines runs start <routineId> --bulk <file>` returns `{ routineRunId, mode: "bulk", status: "in_progress", fileId }`.
- `clay routines runs get <routineRunId>` returns `status`, `total`, and `finished`. When the run is complete it also returns `resultUrl`, a presigned download URL that expires after 900 seconds.
- Download the results with `curl` and match rows by `id`. The order of lines in the results file is not the input order.
- Do not store or print `resultUrl`. It contains a temporary security token.
- Read `clay credits balance` before the run and a few seconds after it completes.
- Run the verification step below before finishing.

Using the Clay CLI, run a routine over a JSONL file and report what it cost.

Requirements:
- Each line of the file is `{ "id": "<string>", "inputs": { ... } }`. Inline runs accept up to 100 items; larger batches use `--bulk`.
- `clay routines runs start <routineId> --bulk <file>` returns `{ routineRunId, mode: "bulk", status: "in_progress", fileId }`.
- `clay routines runs get <routineRunId>` returns `status`, `total`, and `finished`. When the run is complete it also returns `resultUrl`, a presigned download URL that expires after 900 seconds.
- Download the results with `curl` and match rows by `id`. The order of lines in the results file is not the input order.
- Do not store or print `resultUrl`. It contains a temporary security token.
- Read `clay credits balance` before the run and a few seconds after it completes.
- Run the verification step below before finishing.

Prerequisites

The clay CLI on PATH, authenticated via clay login
jq, curl, and bc
The estimate_batch and require_credits functions from the two previous examples, saved as estimate.sh and guard.sh
A workspace with credits. This example spends 3 credits.

Note: JSON samples below are trimmed to the fields relevant to each step. Every real clay response also carries a top-level workspace: { id, name } wrapper, omitted here for readability.

1. Create the input file

printf '%s\n' \
  '{"id":"a","inputs":{"Company Name":"Notion"}}' \
  '{"id":"b","inputs":{"Company Name":"Figma"}}' \
  '{"id":"c","inputs":{"Company Name":"Linear"}}' > rows.jsonl

printf '%s\n' \
  '{"id":"a","inputs":{"Company Name":"Notion"}}' \
  '{"id":"b","inputs":{"Company Name":"Figma"}}' \
  '{"id":"c","inputs":{"Company Name":"Linear"}}' > rows.jsonl

2. Run the batch

source guard.sh
source estimate.sh
R=function:t_0tjqm9rUjfGY4MunGDM
estimate_batch $R rows.jsonl
require_credits 3 || exit 1
b0=$(clay credits balance | jq .balance)
rid=$(clay routines runs start $R --bulk rows.jsonl | jq -r .routineRunId)
for _ in $(seq 1 30); do
  sleep 4
  clay routines runs get "$rid" > /tmp/bulk.json
  s=$(jq -r .status /tmp/bulk.json)
  [ "$s" != "in_progress" ] && break
done
echo "status=$s total=$(jq .total /tmp/bulk.json) finished=$(jq .finished /tmp/bulk.json)"
curl -s "$(jq -r .resultUrl /tmp/bulk.json)" | jq -c .
rm /tmp/bulk.json
sleep 5
echo "credits_spent=$(( b0 - $(clay credits balance | jq .balance) ))"

source guard.sh
source estimate.sh
R=function:t_0tjqm9rUjfGY4MunGDM
estimate_batch $R rows.jsonl
require_credits 3 || exit 1
b0=$(clay credits balance | jq .balance)
rid=$(clay routines runs start $R --bulk rows.jsonl | jq -r .routineRunId)
for _ in $(seq 1 30); do
  sleep 4
  clay routines runs get "$rid" > /tmp/bulk.json
  s=$(jq -r .status /tmp/bulk.json)
  [ "$s" != "in_progress" ] && break
done
echo "status=$s total=$(jq .total /tmp/bulk.json) finished=$(jq .finished /tmp/bulk.json)"
curl -s "$(jq -r .resultUrl /tmp/bulk.json)" | jq -c .
rm /tmp/bulk.json
sleep 5
echo "credits_spent=$(( b0 - $(clay credits balance | jq .balance) ))"

Real output on 2026-10-09:

rows=3 per_run=1 estimated_credits=3
ok: need 3 credits, have 393066
status=complete total=3 finished=3
{"id":"c","status":"complete","result":{"Domain":"linear.app"}}
{"id":"a","status":"complete","result":{"Domain":"notion.com"}}
{"id":"b","status":"complete","result":{"Domain":"figma.com"}}
credits_spent=3

rows=3 per_run=1 estimated_credits=3
ok: need 3 credits, have 393066
status=complete total=3 finished=3
{"id":"c","status":"complete","result":{"Domain":"linear.app"}}
{"id":"a","status":"complete","result":{"Domain":"notion.com"}}
{"id":"b","status":"complete","result":{"Domain":"figma.com"}}
credits_spent=3

The estimate and the spend were both 3. The results arrived in the order c, a, b.

Verify the result

clay routines runs get <routineRunId> | jq '{status, total, finished}'

clay routines runs get <routineRunId> | jq '{status, total, finished}'

Expected: {"status":"complete","total":3,"finished":3}.

How it works

The CLI uploads the file, and the service runs each row against the routine. runs get reports progress as finished out of total. When the status is complete, the response carries a presigned resultUrl that points to a JSONL file with one result per input id. The batch charge is the sum of the per-row charges.

Common issues

Results are not in input order

The same 3-row file returned its results in the order c, b, a in one run and c, a, b in another. Join on id.

The result URL expires

The presigned URL carried X-Amz-Expires=900, which is 15 minutes. Download the file as soon as the status is complete, and avoid logging the URL, which includes a session token.

A bulk file with an invalid row fails after upload

A row missing a required input starts a run that ends as validation_failed. See Confirming That Invalid Inputs Cost Nothing.

Next steps

verification:
  status: verified
  tested_at: "2026-10-09"
  product_version: "clay CLI 1.9.1"
  command: "clay routines runs get <routineRunId> | jq '{status, total, finished}'"
  expected_result: "{\"status\":\"complete\",\"total\":3,\"finished\":3}"

verification:
  status: verified
  tested_at: "2026-10-09"
  product_version: "clay CLI 1.9.1"
  command: "clay routines runs get <routineRunId> | jq '{status, total, finished}'"
  expected_result: "{\"status\":\"complete\",\"total\":3,\"finished\":3}"