Clay AI agent benchmark: open-weight vs closed models

With Clay, the open-weight Kimi K3 kept pace with the closed models in our benchmark: in Factory it averaged 83.0 of 100 across five GTM data tasks, within 0.2 points of the top closed-model setup, and passed 11 of 15 runs, second only to Claude Code with Opus 5.5. Kimi K3 posted the two highest job change averages, while the closed models led on CMOs and target lists, where Opus found CMOs for under 3 Clay credits per company.

By The Prompting Company · Snapshot November 5, 2026

The question

Our Clay benchmark ran one open-weight model, Kimi K3, in two harnesses: OpenCode with Kimi K3 and Factory with Kimi K3, called OpenCode Kimi and Factory Kimi below. Five setups used closed models: Claude Code with Opus 5.5, Claude Code with Sonnet 5, Codex with GPT-6 Sol, Pi with GPT-5.6 Luna and Grok CLI with Grok 4.6. Clay credits are measured as the change in the Clay workspace credit balance.

Task by task

Primary score out of 100, averaged over three runs.

*Includes one run that reached the 60-minute limit and counts as 0.

Where Kimi K3 does best with Clay

Job changes. Factory Kimi averaged 94.7 and OpenCode Kimi 93.7, the two highest averages of seven agents. Both ran Clay's Enrich Person and Find Contact Details once per LinkedIn profile in every run and caught 42 of the 45 people who had moved. Factory Kimi's second run scored 96, the best single job change run in the benchmark.

Work emails. Both harnesses found 93 of 100 verified emails in every run, the same top score as four closed-model setups. Factory Kimi's first run spent 334.5 credits, the lowest of all 21 work email runs, and OpenCode Kimi had the fastest median time, 5.1 minutes.

Single-run highs on company data. Factory Kimi's first company data run was the only run in the benchmark to score 100 on domain, HQ country and headcount, and its 79 on funding was the best single funding score.

Overall. Factory Kimi's 83.0 average sits within 0.2 points of the top average, and its 11 passed runs are second only to Opus's 14.

Where the closed models do best with Clay

CMOs. Opus used Clay's free people search to choose each CMO, then ran Work Email for that one person: 74 to 82 of 100 for 285 to 293 credits, under 3 per company.

Target lists. Opus listed 91 to 94 qualifying companies per run, and Codex 82 to 88 for 10 to 31 credits, under 20 credits for a 100-company list in two of its runs. Both confirmed each company was listed in its own name and headquartered in the US before adding it.

Company data. Codex averaged 98.3 on domain, HQ country and headcount by checking candidates in Clay's company search before enriching, and Opus read funding from Clay's company news for 73 to 77 in every run.

The smallest bills. Codex used the fewest Clay credits, 1,313 for one average run of all five tasks, and Pi had the lowest model cost, $0.25 for all 15 runs.

Which fits which job?

Open-weight and closed models both did strong work with Clay, and the same Clay method gave close to the same result whichever model ran it. For the two Kimi K3 harnesses side by side, see "Does the agent harness matter when using Clay with the same model?", and for all seven agents, "Which AI coding agent gets the most out of Clay?".

How we tested

Seven agents ran each of five tasks three times, one agent at a time, in a fresh sandbox with the same Clay key, the same 19 saved Clay routines and a 60-minute limit. A timeout counts as 0, and platform failures were rerun. Each task had 100 records. Model cost is as reported by the test platform. Tasks, prompts and graders are at github.com/promptingcompany/clay-bench.

Try it yourself

The Clay Agent Plugin ships Clay's skills to your agent up front, including the habits that keep credit use down; setup is one prompt (about 15 minutes), and there is no additional cost to use it. Install the Clay Agent Plugin in the harness you use, open-weight or closed, or give your agent a Clay API key and the Clay Public API docs. Agent work uses the same credits as in-product work (Clay pricing).

Common questions

Open-weight or closed models: which works better with Clay? Both work well: open-weight Kimi K3 in Factory averaged 83.0 of 100, within 0.2 points of the top closed-model setup, and led on job changes, while closed models led on CMOs and target lists.

Where does Kimi K3 fit best with Clay? Job change checks, where it posted the two highest averages, 94.7 and 93.7, and work emails, where it found 93 of 100 in every run.

Does an open-weight model use more Clay credits? Kimi K3 spent 3,541 to 3,560 credits for one average run of all five tasks, between Opus's 2,913 and Sonnet's 3,797.