GitHub Copilot Evals: repeatable tests for AI agents, now in public preview

GitHub Copilot Evals lets teams write repeatable tests for AI coding skills, custom agents and MCP tools. It checks both the result and the path the agent took, and is in public preview.

GitHub put Copilot Evals into public preview on 17 November 2026. An eval, in GitHub’s words, is “a repeatable test for AI behavior”. It is aimed at teams who have set up Copilot with their own skills, custom agents and MCP tools (connectors that let an agent use other software) and want to know whether those setups actually work.

What the changelog says

Why it matters

Most teams adopting AI coding tools tune them by feel: a prompt gets tweaked and everyone hopes it improved. Evals turn that into testing. The same shift happened to ordinary software decades ago, and it is what separates a demo from something you can depend on.

So what

If you are commissioning AI features or an AI-built product, ask your supplier how they know the AI parts keep working after a change. A good answer names a test suite, not a feeling. Our AI product work in services and AI-assisted development both treat testing as part of the build. If you are planning one, start a project.

← All research

More research.

GitHub Copilot adds Supabase, Webflow, Wix and five more connectors in public preview

GitHub Copilot now connects to eight more services, including Supabase, Webflow, Wix and Mixpanel, through verified MCP connectors in public preview. They join more than 25 existing ones.

Read ›

Gemini 4 Neon is generally available in GitHub Copilot, aimed at lower-cost agentic coding

Gemini 4 Neon is now generally available in GitHub Copilot for agentic coding. Google says its quality is close to Gemini 4 Carbon at a lower token cost per task.

Read ›

Claude Code 2.1.328 brings auto mode to Bedrock, Vertex and Foundry, and narrows /code-review to bugs

Claude Code 2.1.328 extends auto mode to Claude Opus 4.6 and Sonnet 4.6 on Amazon Bedrock, Google Cloud and Microsoft Foundry. /code-review now reports only bugs and CLAUDE.md violations.

Read ›

GitHub Copilot can now drive your browser: browser use enters public preview

GitHub Copilot browser use lets an AI coding agent control Chrome to debug web apps and run browser tests. It is in public preview across the CLI, the Copilot app, VS Code and JetBrains.

Read ›

Claude Code 2.1.327 refuses some symbolic links and lists broken setup in /status

Claude Code 2.1.327 tightens file and package permissions again and adds a /status list of broken agent, skill and hook settings. A short read for teams using AI coding tools on a live product.

Read ›

VS Code retires the Copilot Local harness in February 2027

GitHub will remove the older Local harness for Copilot agent sessions in VS Code in February 2027. Teams using AI coding agents need to move to the Copilot harness before then.

Read ›

Start a project

Building with AI? Let's talk.

Tell us what you are building. We'll give you an honest view of the right approach.

Start a projectSee our work