Running a Clay Enrichment from Cron with the Clay CLI

A cron job runs with a minimal environment: a short PATH, no login session, and no terminal. This example runs a bulk enrichment from a shell script under env -i with only /usr/bin:/bin on the PATH, authenticated with CLAY_API_KEY. The run completed, wrote a dated results file, and logged one JSON line.

What you will build

A script enrich.sh and a cron entry that starts it. The script checks a credit floor, starts a bulk run, polls it, downloads the results, and appends a JSON line to job.log. Any Clay failure is logged with the CLI's exit code and exits with the same code.

cron -> run.sh (loads key + absolute clay path) -> enrich.sh
                                                      ↓
                          clay credits balance  (floor check, exit 7 if too low)
                          clay routines runs start --bulk accounts.jsonl
                          clay routines runs get <id>   (poll up to 10 minutes)
                          curl <resultUrl> -> out/<date>.jsonl
                                                      ↓
                                                  job.log

cron -> run.sh (loads key + absolute clay path) -> enrich.sh
                                                      ↓
                          clay credits balance  (floor check, exit 7 if too low)
                          clay routines runs start --bulk accounts.jsonl
                          clay routines runs get <id>   (poll up to 10 minutes)
                          curl <resultUrl> -> out/<date>.jsonl
                                                      ↓
                                                  job.log

AI Prompt

Write a shell script that runs a Clay bulk enrichment from cron using the Clay CLI.

Requirements:
- cron starts jobs with a minimal environment, typically PATH=/usr/bin:/bin. Call the CLI by an absolute path (CLAY_BIN) and load CLAY_API_KEY from a file that only the job's user can read.
- With only CLAY_API_KEY set, these commands work: `clay credits balance`, `clay routines runs start`, and `clay routines runs get`. `clay routines list` and `clay routines runs list` return auth_forbidden.
- Check `clay credits balance | jq .balance` against a floor. Below the floor, log a skipped line and exit 7 without starting a run.
- `clay routines runs start function:t_0tjqm9rUjfGY4MunGDM --bulk accounts.jsonl` returns { routineRunId, mode: "bulk", status: "in_progress", fileId }. Each file line is { "id": "<string>", "inputs": { "Company Name": "<name>" } }. The routine is Company Domain and costs 1 credit per row.
- Poll `clay routines runs get <id>` every 5 seconds for up to 10 minutes. A complete bulk run includes resultUrl, a presigned URL for the results that expires after 900 seconds. Download it with curl and never log it.
- When any clay command fails, log its exit code and error.code to job.log and exit with that code. Exit codes: 2 validation, 3 auth, 4 rate limited, 5 network, 6 not found.
- Test it from an empty HOME with `env -i HOME=<dir> PATH=/usr/bin:/bin`, which approximates cron's environment.
- Run the verification step below before finishing.

Write a shell script that runs a Clay bulk enrichment from cron using the Clay CLI.

Requirements:
- cron starts jobs with a minimal environment, typically PATH=/usr/bin:/bin. Call the CLI by an absolute path (CLAY_BIN) and load CLAY_API_KEY from a file that only the job's user can read.
- With only CLAY_API_KEY set, these commands work: `clay credits balance`, `clay routines runs start`, and `clay routines runs get`. `clay routines list` and `clay routines runs list` return auth_forbidden.
- Check `clay credits balance | jq .balance` against a floor. Below the floor, log a skipped line and exit 7 without starting a run.
- `clay routines runs start function:t_0tjqm9rUjfGY4MunGDM --bulk accounts.jsonl` returns { routineRunId, mode: "bulk", status: "in_progress", fileId }. Each file line is { "id": "<string>", "inputs": { "Company Name": "<name>" } }. The routine is Company Domain and costs 1 credit per row.
- Poll `clay routines runs get <id>` every 5 seconds for up to 10 minutes. A complete bulk run includes resultUrl, a presigned URL for the results that expires after 900 seconds. Download it with curl and never log it.
- When any clay command fails, log its exit code and error.code to job.log and exit with that code. Exit codes: 2 validation, 3 auth, 4 rate limited, 5 network, 6 not found.
- Test it from an empty HOME with `env -i HOME=<dir> PATH=/usr/bin:/bin`, which approximates cron's environment.
- Run the verification step below before finishing.

Prerequisites

The clay CLI installed on the machine that runs cron
jq, curl, and awk
A Public API key from clay api-keys create, which requires a signed-in machine once. How the key behaves in a minimal environment is covered in Running the Clay CLI on a Headless Machine with an API Key.
A workspace with at least 10 credits. The test spends 3.

Note: The workspace object that every clay response includes is left out of the samples.

1. Create the job directory

mkdir -p ~/clay-cron && cd ~/clay-cron
printf '%s\n' \
  '{"id":"a","inputs":{"Company Name":"Notion"}}' \
  '{"id":"b","inputs":{"Company Name":"Figma"}}' \
  '{"id":"c","inputs":{"Company Name":"Linear"}}' > accounts.jsonl

mkdir -p ~/clay-cron && cd ~/clay-cron
printf '%s\n' \
  '{"id":"a","inputs":{"Company Name":"Notion"}}' \
  '{"id":"b","inputs":{"Company Name":"Figma"}}' \
  '{"id":"c","inputs":{"Company Name":"Linear"}}' > accounts.jsonl

2. Store the key and the CLI path

umask 077
printf 'export CLAY_API_KEY=%s\nexport CLAY_BIN=%s\n' "<your key>" "$(command -v clay)" > ~/.clay-cron.env

umask 077
printf 'export CLAY_API_KEY=%s\nexport CLAY_BIN=%s\n' "<your key>" "$(command -v clay)" > ~/.clay-cron.env

With umask 077 in effect, only your user can read the file. command -v clay is run from your interactive shell, where clay is on the PATH, so the script gets an absolute path.

3. Write the script

Write the job script to ~/clay-cron/enrich.sh and make it executable:

#!/bin/bash
# Nightly Clay enrichment for cron. Needs CLAY_API_KEY and CLAY_BIN (absolute path to the clay binary) in the environment.
set -uo pipefail
cd "$(dirname "$0")"
mkdir -p out
ROUTINE=function:t_0tjqm9rUjfGY4MunGDM   # Company Domain, 1 credit per row
FLOOR=${MIN_CREDITS:-10}
log() { printf '{"at":"%s","event":"%s"%s}\n' "$(date -u +%FT%TZ)" "$1" "${2:+,$2}"

# Run clay and print its stdout. On failure, log the CLI's own error code and return its exit code;
# callers append `|| exit $?`, because an exit inside $(...) only leaves the subshell.
clay_json() {
  local out code
  out=$("$CLAY_BIN" "$@" 2>err.txt); code=$?
  if [ $code -ne 0 ]; then
    log failed "\"command\":\"$1 $2\",\"exit\":$code,\"error
    return $code
  fi
  printf '%s' "$out"
}

out=$(clay_json credits balance) || exit $?
balance=$(printf '%s' "$out" | jq .balance)
if awk -v b="$balance" -v f="$FLOOR" 'BEGIN { exit !(b < f) }'; then
  log skipped "\"reason\":\"balance $balance is below $FLOOR\""
  exit 7
fi

out=$(clay_json routines runs start $ROUTINE --bulk accounts.jsonl) || exit $?
rid=$(printf '%s' "$out" | jq -r .routineRunId)
for _ in $(seq 1 120); do
  sleep 5
  run=$(clay_json routines runs get "$rid") || exit $?
  status=$(printf '%s' "$run" | jq -r .status)
  [ "$status" != "in_progress" ] && break
done
if [ "$status" = "in_progress" ]; then log timeout "\"run\":\"$rid

curl -fsS "$(printf '%s' "$run" | jq -r .resultUrl)" -o "out/$(date -u +%F).jsonl"
log finished "\"run\":\"$rid\",\"status\":\"$status\",\"

#!/bin/bash
# Nightly Clay enrichment for cron. Needs CLAY_API_KEY and CLAY_BIN (absolute path to the clay binary) in the environment.
set -uo pipefail
cd "$(dirname "$0")"
mkdir -p out
ROUTINE=function:t_0tjqm9rUjfGY4MunGDM   # Company Domain, 1 credit per row
FLOOR=${MIN_CREDITS:-10}
log() { printf '{"at":"%s","event":"%s"%s}\n' "$(date -u +%FT%TZ)" "$1" "${2:+,$2}" >> job.log; }

# Run clay and print its stdout. On failure, log the CLI's own error code and return its exit code;
# callers append `|| exit $?`, because an exit inside $(...) only leaves the subshell.
clay_json() {
  local out code
  out=$("$CLAY_BIN" "$@" 2>err.txt); code=$?
  if [ $code -ne 0 ]; then
    log failed "\"command\":\"$1 $2\",\"exit\":$code,\"error\":$(jq -c '.error.code' err.txt 2>/dev/null || echo null)"
    return $code
  fi
  printf '%s' "$out"
}

out=$(clay_json credits balance) || exit $?
balance=$(printf '%s' "$out" | jq .balance)
if awk -v b="$balance" -v f="$FLOOR" 'BEGIN { exit !(b < f) }'; then
  log skipped "\"reason\":\"balance $balance is below $FLOOR\""
  exit 7
fi

out=$(clay_json routines runs start $ROUTINE --bulk accounts.jsonl) || exit $?
rid=$(printf '%s' "$out" | jq -r .routineRunId)
for _ in $(seq 1 120); do
  sleep 5
  run=$(clay_json routines runs get "$rid") || exit $?
  status=$(printf '%s' "$run" | jq -r .status)
  [ "$status" != "in_progress" ] && break
done
if [ "$status" = "in_progress" ]; then log timeout "\"run\":\"$rid\""; exit 1; fi

curl -fsS "$(printf '%s' "$run" | jq -r .resultUrl)" -o "out/$(date -u +%F).jsonl"
log finished "\"run\":\"$rid\",\"status\":\"$status\",\"total\":$(printf '%s' "$run" | jq .total),\"finished\":$(printf '%s' "$run" | jq .finished)"

Save this as ~/clay-cron/run.sh and chmod +x it. It is the line cron runs:

#!/bin/sh
. "$HOME/.clay-cron.env"
exec /bin/bash "$HOME/clay-cron/enrich.sh"

#!/bin/sh
. "$HOME/.clay-cron.env"
exec /bin/bash "$HOME/clay-cron/enrich.sh"

4. Test it with cron's environment

Copy the job into a throwaway home directory, so no stored clay login session can answer for a missing or invalid key, and run it with env -i:

T=$(mktemp -d)
mkdir "$T/clay-cron"
cp ~/clay-cron/enrich.sh ~/clay-cron/run.sh ~/clay-cron/accounts.jsonl "$T/clay-cron/"
cp ~/.clay-cron.env "$T/"
cd "$T" && env -i HOME="$T" PATH=/usr/bin:/bin /bin/sh -c "$T/clay-cron/run.sh"
cat "$T/clay-cron/job.log"
ls "$T/clay-cron/out"
wc -l "$T"/clay-cron/out/*.jsonl

T=$(mktemp -d)
mkdir "$T/clay-cron"
cp ~/clay-cron/enrich.sh ~/clay-cron/run.sh ~/clay-cron/accounts.jsonl "$T/clay-cron/"
cp ~/.clay-cron.env "$T/"
cd "$T" && env -i HOME="$T" PATH=/usr/bin:/bin /bin/sh -c "$T/clay-cron/run.sh"; echo "exit=$?"
cat "$T/clay-cron/job.log"
ls "$T/clay-cron/out"
wc -l "$T"/clay-cron/out/*.jsonl

5. Add the cron entry

crontab -e

crontab -e

0 6 * * * /bin/sh $HOME/clay-cron/run.sh

0 6 * * * /bin/sh $HOME/clay-cron/run.sh

This runs the job at 06:00 every day. The crontab entry itself was not installed during this test, so the scheduled start is described and not observed. The step 4 command reproduces cron's short PATH and an otherwise empty environment.

6. Verify the result

Real output on 2026-10-12 from the step 4 commands. The home directory was empty, so no stored clay login session could be used:

## normal run
exit=0
{"at":"2026-10-13T02:00:27Z","event":"finished","run":"run_0tmtpkfyYgu7xpiSwWf","status":"complete","total":3,"finished":3}
2026-10-13.jsonl
       3 clay-cron/out/2026-10-13.jsonl

## invalid key
exit=3
{"at":"2026-10-13T02:00:28Z","event":"failed","command":"credits balance","exit":3,"error":"auth_invalid"}

## credit floor above the balance (MIN_CREDITS=99999999)
exit=7
{"at":"2026-10-13T02:00:28Z","event":"skipped","reason":"balance 398123.5 is below 99999999"}

## normal run
exit=0
{"at":"2026-10-13T02:00:27Z","event":"finished","run":"run_0tmtpkfyYgu7xpiSwWf","status":"complete","total":3,"finished":3}
2026-10-13.jsonl
       3 clay-cron/out/2026-10-13.jsonl

## invalid key
exit=3
{"at":"2026-10-13T02:00:28Z","event":"failed","command":"credits balance","exit":3,"error":"auth_invalid"}

## credit floor above the balance (MIN_CREDITS=99999999)
exit=7
{"at":"2026-10-13T02:00:28Z","event":"skipped","reason":"balance 398123.5 is below 99999999"}

The first section is the step 4 commands. The other two sections repeat them in a fresh throwaway home, once with CLAY_API_KEY set to clay_scoped_invalid and once with export MIN_CREDITS=99999999 added to ~/.clay-cron.env.

The check is a set of properties: the normal run exits 0 with one finished line whose total and finished are both 3 and a dated results file with 3 lines; the invalid key exits 3, logs failed with auth_invalid, and writes no results file; the credit floor exits 7 and logs skipped, and no run starts.

How it works

run.sh exists because cron does not read your shell profile, so CLAY_API_KEY and the CLI's location must come from a file. enrich.sh calls clay only through clay_json, which logs any non-zero CLI exit and returns the same code. Each call site ends in || exit $?, because an exit inside $(...) leaves only the subshell and the script would otherwise carry on with empty values. A scheduler that alerts on non-zero exits then sees the CLI's own code.

Common issues

`clay: command not found`

Under cron's PATH, clay installed in ~/.local/bin is not found, and the shell exits with 127. A run with env -i PATH=/usr/bin:/bin and clay credits balance reproduced this on 2026-10-12. Set CLAY_BIN to the absolute path.

A stored login hides a bad key

If the user who runs cron has also run clay login, the stored session is used and CLAY_API_KEY is ignored. A key that has expired or been deleted would not show up until the stored session expires. Run the job as a user without a stored login, or test with an empty HOME as above.

The key file needs restricted permissions

~/.clay-cron.env holds a secret. umask 077 creates it with mode 600. Keep the key out of the crontab line, because crontab -l prints every line.

Overlapping runs

This script does not take a lock. A run that takes longer than the interval can overlap the next one. Use flock where it exists, or a lock directory created with mkdir, which fails if the directory is already there.

Polling gives up after 10 minutes

The script logs a timeout line and exits 1. The log line holds the routineRunId, so clay routines runs get <id> can fetch the run later. In a REST test during this batch, a run kept going after the script stopped polling, and a later fetch by id returned all rows.

Next steps

verification:
  status: verified
  tested_at: "2026-10-12"
  product_version: "clay 1.9"
  command: "env -i HOME=$HOME PATH=/usr/bin:/bin /bin/sh -c $HOME/clay-cron/run.sh"
  expected_result: "Exit 0 with one finished line (total and finished of 3) and a dated 3-line results file; an invalid key exits 3 with a failed line; a credit floor above the balance exits 7 with a skipped line."

verification:
  status: verified
  tested_at: "2026-10-12"
  product_version: "clay 1.9"
  command: "env -i HOME=$HOME PATH=/usr/bin:/bin /bin/sh -c $HOME/clay-cron/run.sh"
  expected_result: "Exit 0 with one finished line (total and finished of 3) and a dated 3-line results file; an invalid key exits 3 with a failed line; a credit floor above the balance exits 7 with a skipped line."