Microsoft Open Sources code-testing-generator: a Polyglot Unit-Test Agent That Hits 92.1% Task Completion Versus 78.9% for Stock Copilot

Microsoft has open sourced code-testing-generator, a polyglot agent that writes unit tests and then proves they work. It ships in the dotnet-test plugin inside the MIT-licensed dotnet/skills repository.

The agent targets a gap that coding assistants usually leave open. A prompt like ‘generate unit tests’ does not say which framework, file location or assertions to use. code-testing-generator settles those decisions by reading the repository before it writes anything. It then plans, writes, runs and checks the tests it produces. On Microsoft’s internal 152-task benchmark, it completed 140 tasks against 120 for stock GitHub Copilot. Both setups used the same model and prompts.

Is it deployable

Yes. It is an agent definition along with skills, not a hosted service, so it runs inside your existing coding agent and code stays local.

• Company stage: viable from solo maintainers upward. Startups and mid-market teams gain most, because the agent supplies repository research a small team has no time to encode. Enterprises can fork the language guidance to match internal frameworks.
• Industries: regulated or audit-heavy software estates — financial services, healthcare, insurance, public sector — plus platform teams paying down legacy test debt.
• Applications: backfilling tests on untested modules, generating tests for a pull-request diff, raising coverage before a release gate, and standardising conventions across polyglot monorepos.

What the agent actually does

It coordinates work through a Research-Plan-Implement (RPI) pipeline. It searches the repository for code needing tests, detects the language and test framework, reads existing tests for conventions, and finds the real build and test commands. That last step targets a specific failure: a test project that builds locally but never runs in CI because nothing registered it.

The agent then picks one of three strategies. Direct writes and validates tests immediately. Single pass runs one cycle. Iterative repeats it for large scopes or coverage targets. It never modifies production code, and avoids tests that call external URLs, bind ports or depend on timing.

The verification gate

Before reporting completion, the agent runs five checks. It reasons about small code changes that should make the tests fail, a lightweight form of mutation testing. It looks for weak or missing assertions. It maps every requested scenario to a test. It builds the full workspace and runs the full suite. It confirms the repository’s own test command discovers the new tests.

Benchmark results

On Microsoft’s internal benchmark of 152 tasks from real repositories, the agent completed 140 (92.1%) versus 120 (78.9%) for stock GitHub Copilot on the same model and prompts (63% fewer failures).

The gain is concentrated. On 89 vague prompts, the agent resolved 79 (88.8%) against 59 (66.3%), cutting failures from 30 to 10. On 63 detailed prompts, both scored 61 (96.8%). On 15 tasks targeting a specific diff, the agent passed all 15 and stock Copilot passed none.

Notably, the agent generated 2.3% fewer tests (6,963 vs 7,129) at effectively identical line coverage (72.4% vs 72.2%). Average task time was 359 seconds against 380. Token use per completed task was 3.2% higher.

On 45 .NET tasks, Claude Opus 4.8 reached 43/45 with the agent versus 35/45 stock; GPT-5.5 reached 41/45 versus 36/45. On the harder external SWE Atlas benchmark, completion was 16/44 versus 12/44.

Explainer: how the agent turns one prompt into verified tests

Key Takeaways

• Open source, MIT-licensed, polyglot unit-test agent from Microsoft’s .NET team.
• Research-Plan-Implement pipeline replaces one-shot generation with repository-aware planning.
• 92.1% vs 78.9% task completion against stock Copilot on the same model.
• Gains come almost entirely from vague prompts and diff-targeted requests.
• Fewer tests, same coverage, 5.5% faster — reliability, not volume.

Check out the Technical details and Repo. Also, feel free to follow us on Twitter and don’t forget to join our 150k+ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.

Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us

Michal Sutter is a data science professional with a Master of Science in Data Science from the University of Padova. With a solid foundation in statistical analysis, machine learning, and data engineering, Michal excels at transforming complex datasets into actionable insights.