As AI models evolve from simple chatbots into reasoning agents that write code, use tools and solve complex problems, traditional benchmarks are no longer enough. The community needs dynamic, rigorous evaluations — built by the people who use these models in the real-world.
That’s why we launched Kaggle Benchmarks. Since then, the global AI community has created more than 10,000 evaluation tasks, creating the trustworthy, transparent public leaderboards that help labs measure and accelerate AI progress.
Today, we are taking the next step by launching local development for Kaggle Benchmarks.
Use Kaggle Benchmarks from your local development environment
Until now, creating evaluation tasks meant working exclusively in Kaggle’s web-based notebook editor, instead of developers’ preferred stack to build with.
Our new update enables developers to create, validate, push, run and download tasks directly from their local development environments like Antigravity, VSCode, Cursor and coding agents. This update is designed to meet developers where they work, making the journey from idea to evaluation faster and more intuitive.
Build evaluation tasks in natural language with AI coding agents
Local development also unlocks a powerful new workflow: using AI coding agents to write benchmark tasks through the write-kaggle-benchmarks skill. This skill comprises a set of structured instructions that teaches a coding agent how to build tasks using the kaggle-benchmarks SDK and the Kaggle CLI.
To add this skill to your agent, simply ask your agent to:
Once installed, you can describe an evaluation in plain language and get a working task on Kaggle. For example, you can tell your agent:
These powerful capabilities are driven by the new commands that we have built for Benchmarks in the Kaggle CLI.
