Exmergo logoExmergo
All benchmarks

Published by dbt Labs

ADE-bench

Analytics Engineering Benchmark

dbt Labs' 75-task analytics-engineering benchmark, scored on whether the dbt project's tests actually pass.

75 tasks1 published runADE-bench on GitHub

What ADE-bench measures

ADE-bench hands an agent a real dbt project running on DuckDB and asks it to fix a broken model, build a new one, or extend the semantic layer. Each of the 75 tasks is graded by running the project's own tests afterwards, so the benchmark measures completed engineering work rather than plausible-looking SQL. Tasks span eight domains, and there is no partial credit: a task counts only when the resulting project genuinely builds and passes.

Latest result ·

76.0%57 / 75 tasks resolved with Claude Sonnet 5

Dex + Sonnet 5 leads at 76%, for about 2.5x less than Fable 5 and ~17% less than Opus 4.8. With Dex, accuracy holds in a 72–76% band across all three models while cost ranges from $36 to $92, so the practical call is to run an inexpensive model.

For context, dbt's published agent skills reported 58% on this benchmark with Opus 4.6.

Read the full 21 July 2026 run→

Run history

Every ADE-bench run Exmergo has published, newest first. Each run keeps its own permanent page, so earlier results stay citable.

75 tasks
76.0%57 / 75 resolved

On dbt Labs' 75-task analytics-engineering benchmark, Dex + Claude Sonnet 5 resolves more tasks than any other run we measured, and beats dbt's own published agent skills.

Best run: Claude Sonnet 5 with dex

Read the run→

ADE-bench questions

What is ADE-bench?
ADE-bench is dbt Labs' Analytics Engineering Benchmark: a 75-task suite that hands an agent a real dbt project running on DuckDB and asks it to fix a broken model, build a new one, or extend the semantic layer. The agent's work is then scored on whether the project's tests actually pass, so it measures completed engineering work rather than plausible-looking SQL.
Who publishes ADE-bench?
dbt Labs publishes and maintains ADE-bench as an open benchmark at github.com/dbt-labs/ade-bench. It is a third-party suite, not one Exmergo authored, which is what makes results on it comparable across tools.
How is ADE-bench scored?
Each of the 75 tasks is graded pass or fail by running the dbt project's own tests after the agent finishes. Accuracy is the share of tasks resolved across the full suite. There is no partial credit, so a task counts only when the resulting project genuinely builds and passes.
How does Dex perform on ADE-bench?
In Exmergo's 21 July 2026 run, Dex with Claude Sonnet 5 resolved 57 of 75 tasks (76%), the strongest of the four configurations measured, at about $36 in model spend. For context, dbt's own published agent skills reported 58% on this benchmark with Opus 4.6.
Does adding Dex actually change the result, or is it the model?
Both were measured. The same Claude Sonnet 5 loop with no plugin resolved 53 of 75 tasks (70.7%), against 57 of 75 (76%) with Dex supplied as the exmergo/dex skill plugin. Accuracy also held in a 72-76% band across three different models with Dex, while cost ranged from $36 to $92, which is what makes running an inexpensive model the practical call.

Results measured by Exmergo on dbt Labs’ ADE-bench.

Exmergo logo
Exmergo