Failing a CI Job on Clay CLI Exit Codes with the Clay CLI

The Clay CLI exits with a different code for each kind of failure, so a CI step can retry the transient ones and fail fast on the rest. This example wraps a routine run in a script that retries exit codes 4 and 5, and turns the others into a CI error annotation. Five situations were run from an empty home directory with only CLAY_API_KEY set.

What you will build

A script clay-ci.sh that starts an inline routine run, polls it, and exits with the CLI's own exit code on failure, plus a GitHub Actions workflow that calls it.

clay routines runs start <routine> --input input.json
    exit 0  -> poll `runs get` until status != in_progress
    exit 4 or 5 -> wait (retryAfter, or 3s x try), retry up to 3 times
    exit 2, 3, 6, 1 -> print ::error::, exit with the same code
final status complete -> exit 0, anything else -> exit 1

AI Prompt

Write a shell script for CI that starts a Clay routine run with the Clay CLI and fails the job with a useful message.

Requirements:
- The CLI prints errors as JSON on stderr: { "error": { "code", "message", "details"? } }, and exits with: 0 success; 1 generic or unrecoverable (server_error, quota_exceeded, conflict), do not retry; 2 validation_error; 3 auth_required, auth_invalid, or auth_forbidden; 4 rate_limited (details.retryAfter in seconds); 5 network_error or network_timeout; 6 not_found.
- Retry only 4 and 5, at most 3 tries, waiting details.retryAfter seconds when present and otherwise 3 seconds times the try number. Every other non-zero code fails immediately and the script exits with the same code.
- Print failures as GitHub Actions annotations: `::error::<message>`.
- `clay routines runs start <routineId> --input <file>` returns { routineRunId, mode: "inline", status: "in_progress" }. `clay routines runs get <routineRunId>` returns status, total, and finished. Poll every 5 seconds for up to 5 minutes. The job succeeds only when the final status is "complete".
- Authenticate with the CLAY_API_KEY environment variable. With only that variable, `routines runs start` and `routines runs get` work, and most other command groups return exit 3.
- Run the verification step below before finishing.

Write a shell script for CI that starts a Clay routine run with the Clay CLI and fails the job with a useful message.

Requirements:
- The CLI prints errors as JSON on stderr: { "error": { "code", "message", "details"? } }, and exits with: 0 success; 1 generic or unrecoverable (server_error, quota_exceeded, conflict), do not retry; 2 validation_error; 3 auth_required, auth_invalid, or auth_forbidden; 4 rate_limited (details.retryAfter in seconds); 5 network_error or network_timeout; 6 not_found.
- Retry only 4 and 5, at most 3 tries, waiting details.retryAfter seconds when present and otherwise 3 seconds times the try number. Every other non-zero code fails immediately and the script exits with the same code.
- Print failures as GitHub Actions annotations: `::error::<message>`.
- `clay routines runs start <routineId> --input <file>` returns { routineRunId, mode: "inline", status: "in_progress" }. `clay routines runs get <routineRunId>` returns status, total, and finished. Poll every 5 seconds for up to 5 minutes. The job succeeds only when the final status is "complete".
- Authenticate with the CLAY_API_KEY environment variable. With only that variable, `routines runs start` and `routines runs get` work, and most other command groups return exit 3.
- Run the verification step below before finishing.

Prerequisites

The clay CLI on the runner's PATH
jq
A Public API key stored as a CI secret named CLAY_API_KEY. The key itself comes from clay api-keys create, as in Running the Clay CLI on a Headless Machine with an API Key.

Note: Samples leave out the workspace object that the CLI adds to each response.

1. Write the script

Write the wrapper to clay-ci.sh and make it executable:

#!/bin/bash
# Usage: clay-ci.sh <routine-id> <input.json>
# Starts an inline run, waits for it, and turns the Clay CLI exit code into a CI decision.
set -uo pipefail
ROUTINE=$1; INPUT=$2
MAX_TRIES=3

# Retry only the codes that can clear on their own: 4 (rate limited) and 5 (network).
clay_retry() {
  local try=1 out code
  while :; do
    out=$(clay "$@" 2>err.txt); code=$?
    case $code in
      0) printf '%s' "$out"; return 0 ;;
      4|5) [ $try -ge $MAX_TRIES ] && break
           wait=$(jq -r '.error.details.retryAfter // empty' err.txt 2>/dev/null); sleep "${wait:-$((try 
      *) break ;;
    esac
  done
  report $code "$*"
  exit $code
}

report() {
  local msg; msg=$(jq -r '.error.message // empty' err.txt 2>/dev/null)
  case $1 in
    2) echo >&2 "::error::Clay rejected the input: $msg" ;;
    3) echo >&2 "::error::Clay credential missing, expired, or not allowed for this command: $msg" ;;
    4|5) echo >&2 "::error::Clay still unreachable or rate limited after $MAX_TRIES tries: $msg" ;;
    6) echo >&2 "::error::Clay routine not found, or not enabled for the API and CLI: $msg" ;;
    *) echo >&2 "::error::Clay failed with exit $1, which is not retried: $msg" ;;
  esac
}

out=$(clay_retry routines runs start "$ROUTINE" --input "$INPUT") || 
rid=$(printf '%s' "$out" | jq -r .routineRunId)
for _ in $(seq 1 60); do
  sleep 5
  run=$(clay_retry routines runs get "$rid") || exit $?
  [ "$(printf '%s' "$run" | jq -r .status)" != "in_progress" ] && break
done
printf '%s' "$run" | jq -c '{routineRunId, status, total, finished}'
[ "$(printf '%s' "$run" | jq -r .status)" = complete ]

#!/bin/bash
# Usage: clay-ci.sh <routine-id> <input.json>
# Starts an inline run, waits for it, and turns the Clay CLI exit code into a CI decision.
set -uo pipefail
ROUTINE=$1; INPUT=$2
MAX_TRIES=3

# Retry only the codes that can clear on their own: 4 (rate limited) and 5 (network).
clay_retry() {
  local try=1 out code
  while :; do
    out=$(clay "$@" 2>err.txt); code=$?
    case $code in
      0) printf '%s' "$out"; return 0 ;;
      4|5) [ $try -ge $MAX_TRIES ] && break
           wait=$(jq -r '.error.details.retryAfter // empty' err.txt 2>/dev/null); sleep "${wait:-$((try * 3))}"; try=$((try + 1)) ;;
      *) break ;;
    esac
  done
  report $code "$*"
  exit $code
}

report() {
  local msg; msg=$(jq -r '.error.message // empty' err.txt 2>/dev/null)
  case $1 in
    2) echo >&2 "::error::Clay rejected the input: $msg" ;;
    3) echo >&2 "::error::Clay credential missing, expired, or not allowed for this command: $msg" ;;
    4|5) echo >&2 "::error::Clay still unreachable or rate limited after $MAX_TRIES tries: $msg" ;;
    6) echo >&2 "::error::Clay routine not found, or not enabled for the API and CLI: $msg" ;;
    *) echo >&2 "::error::Clay failed with exit $1, which is not retried: $msg" ;;
  esac
}

out=$(clay_retry routines runs start "$ROUTINE" --input "$INPUT") || exit $?
rid=$(printf '%s' "$out" | jq -r .routineRunId)
for _ in $(seq 1 60); do
  sleep 5
  run=$(clay_retry routines runs get "$rid") || exit $?
  [ "$(printf '%s' "$run" | jq -r .status)" != "in_progress" ] && break
done
printf '%s' "$run" | jq -c '{routineRunId, status, total, finished}'
[ "$(printf '%s' "$run" | jq -r .status)" = complete ]

2. Add the workflow

name: enrich
on:
  schedule:
    - cron: "0 6 * * *"
jobs:
  enrich:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./clay-ci.sh function:t_0tjqm9rUjfGY4MunGDM input.json
        env:
          CLAY_API_KEY: ${{ secrets.CLAY_API_KEY }}

name: enrich
on:
  schedule:
    - cron: "0 6 * * *"
jobs:
  enrich:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: ./clay-ci.sh function:t_0tjqm9rUjfGY4MunGDM input.json
        env:
          CLAY_API_KEY: ${{ secrets.CLAY_API_KEY }}

This workflow was not run on GitHub. It assumes the clay CLI is already installed on the runner, for example by an earlier step, and that input.json is in the repository.

3. Run the five situations locally

Save this as ci-scenarios.sh. It runs clay-ci.sh from an empty home directory with only CLAY_API_KEY set:

#!/bin/bash
# Runs clay-ci.sh against five situations, from an empty HOME and an empty directory.
CL_DIR=$HOME/.local/bin
S=/private/tmp/claude-501/-Users-tpc-macbook-air-001/35e347f0-a12c-4f63-931a-0fcfbdb7b864/scratchpad
W=$(mktemp -d -p $S); EH=$(mktemp -d -p $S)
cp ~/clay-work/_batch-7/cli/clay-ci.sh $W/; cd $W
KEY=$(grep ^CLAY_API_KEY= ~/clay-work/_batch-7/.env | cut -d= -f2)
echo '{"items":[{"id":"r1","inputs":{"Company Name":"Stripe"}}]}' > ok.json
echo '{"items":[{"id":"r1","inputs":{}}]}' > bad-input.json
CD=function:t_0tjqm9rUjfGY4MunGDM
run() { # label, extra env, routine, input
  local label=$1 extra=$2 routine=$3 input=$4
  out=$(env -i PATH=$CL_DIR:/usr/bin:/bin HOME=$EH CLAY_API_KEY=${KEY_OVERRIDE:-$KEY} $extra ./clay-ci.sh $routine $input 2>&1); code
  printf '%s\nexit=%s\n%s\n\n' "## $label" "$code" "$out"
}
run "ok" "" $CD ok.json
run "input missing a required field" "" $CD bad-input.json
run "unknown routine" "" function:t_doesnotexist ok.json
KEY_OVERRIDE=clay_scoped_invalid run "invalid key" "" $CD ok.json
run "network unreachable" "HTTPS_PROXY=http://127.0.0.1:9" $CD ok.json

#!/bin/bash
# Runs clay-ci.sh against five situations, from an empty HOME and an empty directory.
CL_DIR=$HOME/.local/bin
S=/private/tmp/claude-501/-Users-tpc-macbook-air-001/35e347f0-a12c-4f63-931a-0fcfbdb7b864/scratchpad
W=$(mktemp -d -p $S); EH=$(mktemp -d -p $S)
cp ~/clay-work/_batch-7/cli/clay-ci.sh $W/; cd $W
KEY=$(grep ^CLAY_API_KEY= ~/clay-work/_batch-7/.env | cut -d= -f2)
echo '{"items":[{"id":"r1","inputs":{"Company Name":"Stripe"}}]}' > ok.json
echo '{"items":[{"id":"r1","inputs":{}}]}' > bad-input.json
CD=function:t_0tjqm9rUjfGY4MunGDM
run() { # label, extra env, routine, input
  local label=$1 extra=$2 routine=$3 input=$4
  out=$(env -i PATH=$CL_DIR:/usr/bin:/bin HOME=$EH CLAY_API_KEY=${KEY_OVERRIDE:-$KEY} $extra ./clay-ci.sh $routine $input 2>&1); code=$?
  printf '%s\nexit=%s\n%s\n\n' "## $label" "$code" "$out"
}
run "ok" "" $CD ok.json
run "input missing a required field" "" $CD bad-input.json
run "unknown routine" "" function:t_doesnotexist ok.json
KEY_OVERRIDE=clay_scoped_invalid run "invalid key" "" $CD ok.json
run "network unreachable" "HTTPS_PROXY=http://127.0.0.1:9" $CD ok.json

bash ci-scenarios.sh

bash ci-scenarios.sh

4. Verify the result

Output of the five scenarios on 2026-10-12:

## ok
exit=0
{"routineRunId":"run_0tmtmd9eNHU6B3PMDao","status":"complete","total":1,"finished":1}

## input missing a required field
exit=2
::error::Clay rejected the input: Invalid inputs for item r1: Company Name: Missing required field: Company Name

## unknown routine
exit=6
::error::Clay routine not found, or not enabled for the API and CLI: Routine not found

## invalid key
exit=3
::error::Clay credential missing, expired, or not allowed for this command: Authentication required. Run `clay login` to sign in again.

## network unreachable
exit=5
::error::Clay still unreachable or rate limited after 3 tries: Unable to connect. Is the computer able to access the url?

## ok
exit=0
{"routineRunId":"run_0tmtmd9eNHU6B3PMDao","status":"complete","total":1,"finished":1}

## input missing a required field
exit=2
::error::Clay rejected the input: Invalid inputs for item r1: Company Name: Missing required field: Company Name

## unknown routine
exit=6
::error::Clay routine not found, or not enabled for the API and CLI: Routine not found

## invalid key
exit=3
::error::Clay credential missing, expired, or not allowed for this command: Authentication required. Run `clay login` to sign in again.

## network unreachable
exit=5
::error::Clay still unreachable or rate limited after 3 tries: Unable to connect. Is the computer able to access the url?

The check is a set of properties: the successful run prints a final status of complete and exits 0, the input missing a required field exits 2, the unknown routine exits 6, the invalid key exits 3, and the unreachable network exits 5 after three tries. Each failure prints one ::error:: line that names the problem.

How it works

clay_retry runs the command, and a case on the exit code decides what happens next. Codes 4 and 5 sleep and loop. Every other non-zero code leaves the loop, calls report, and exits with the code it received, so the CI system's step result carries the same number the CLI produced.

report reads the CLI's JSON error from err.txt on stderr, so the annotation carries the server's own message. The exit code identifies the class of failure, and the message identifies the cause.

Common issues

Exit codes 1, 3, and 6 are not retried

Exit 1 covers quota_exceeded, which the CLI help says not to retry, and plan_upgrade_required. Exit 3 will not change on a retry because the credential is wrong. Exit 6 means the routine id is wrong or the routine is not enabled for API and CLI use.

Exit 4 was not triggered here

No rate-limited response could be provoked, so the branch that waits for retryAfter follows the CLI help and has not run. The wait of 3 seconds times the try number applies to exit 5, which was.

A stored login hides a bad key

On a runner or laptop where clay login has completed, an invalid CLAY_API_KEY still authenticates through the stored session. The scenarios above use an empty home directory for that reason.

The status check does not read rows

The script only reads status, total, and finished, so the 20-row default page size of runs get does not affect it. A step that does read rows should pass --limit 100 and follow --cursor.

A complete run can contain rows with no result

The script succeeds when the run's status is complete. A row that completes with an empty result is not an error to the CLI. Check the rows you need after the run if empty results should fail the job.

Next steps

verification:
  status: verified
  tested_at: "2026-10-12"
  product_version: "clay 1.9"
  command: "bash ci-scenarios.sh"
  expected_result: "The successful run exits 0 with status complete; missing input exits 2; unknown routine exits 6; invalid key exits 3; unreachable network exits 5 after three tries."

verification:
  status: verified
  tested_at: "2026-10-12"
  product_version: "clay 1.9"
  command: "bash ci-scenarios.sh"
  expected_result: "The successful run exits 0 with status complete; missing input exits 2; unknown routine exits 6; invalid key exits 3; unreachable network exits 5 after three tries."