Kimi K3's open weights landed with a #1 coding benchmark score — and a hallucination rate nobody put in the chart

Moonshot AI's Kimi K3 went open-weight on 27 July 2026, a 2.8-trillion-parameter model that ranks #1 on the Frontend Code Arena and scores 76.8% on SWE-bench Verified, but independent testing from Artificial Analysis found a hallucination rate of roughly 51% — up from 39% on the prior Kimi K2.6 — a figure Moonshot's own benchmark release omitted.

Kimi K3’s open weights went live at 00:00 UTC on 27 July 2026 — free to download, 2.8 trillion parameters, a 1-million-token context window, and coding benchmark results that would headline any release: 76.8% on SWE-bench Verified, first place on Frontend Code Arena, and top rankings across Program Bench, SWE Marathon and several other agentic coding evaluations. On the numbers Moonshot AI chose to publish, it’s the strongest open-weight coding model released to date, and the largest open-weight release the industry has seen.

The number Moonshot didn’t put on its own benchmark chart is the one that matters most for anyone weighing whether to use it: Artificial Analysis’s independent AA-Omniscience testing measured K3’s hallucination rate at approximately 51%, up from 39% on its predecessor Kimi K2.6. That’s not a small regression dressed up as a tradeoff — it’s a majority-of-the-time failure rate on the specific measure of whether the model’s outputs are actually true, sitting directly alongside the industry’s best-scoring open coding model on task completion.

This is exactly the split this site has been tracking since the “AI code vulnerability trust gap” and “AI coding trust gap” posts earlier this month: benchmark leaderboards measure whether a model can produce plausible, working-looking code on a defined task set, not whether what it tells you about that code — the edge cases it claims to have handled, the security assumptions it states as fact — is true. A model that tops SWE-bench and hallucinates half the time isn’t a contradiction; it’s a model that’s very good at looking finished.

So what

If a lower-cost open-weight model with frontier coding benchmarks is on your shortlist for a build — your own team’s or a vendor’s — the benchmark score is not the diligence question. The diligence question is what verification sits between the model’s output and what ships: code review discipline, test coverage, and someone who actually reads what the model claims rather than trusting the confidence of the output. That verification layer is the difference between a cheap model being a genuine cost saving and being an expensive mistake discovered in production. It’s also the core of what AI-assisted development means done properly — using these tools for speed without inheriting their failure modes. If you’re evaluating an AI-heavy build and want that reviewed properly, get in touch.

← All research

More research.

GitHub Copilot SDK is becoming the Microsoft Copilot SDK: what builders of AI products should know

On 28 October 2026 GitHub said its Copilot SDK is being renamed Microsoft Copilot SDK, the programmable layer for the shared Microsoft and GitHub agent harness. No immediate action is needed.

Read ›

GitHub Agent Merge: Copilot now watches your pull requests and fixes what blocks the merge

On 28 October 2026 GitHub added Agent Merge to pull requests: a 'Watch and fix' option where Copilot repairs failing Actions workflows and merge conflicts in the cloud, optionally with auto-merge.

Read ›

Claude Code 2.1.314 adds a setting that blocks AI file writes outside the project folder

Claude Code 2.1.314 (30 October 2026) adds permissions.blockWritesOutsideWorkingDirectories, so Claude's file-editing tools refuse any path outside the project in every permission mode.

Read ›

GitHub Enterprise now includes Copilot: seats are bundled, AI usage moves to your budget

From 28 October 2026 account-team customers can get Copilot included with every GitHub Enterprise licence, with AI usage metered against an enterprise budget from the first request. Standalone Copilot is $10 per user per month.

Read ›

GitHub Copilot Fusion goes GA: one pick, many models, and a 30% launch discount

On 28 October 2026 GitHub made Copilot Fusion generally available: instead of choosing a model, developers pick an effort level and Copilot routes each task across models. Paid plans get 30% off model costs for a limited time.

Read ›

GitHub Agentic Workflows is generally available: automations written in plain Markdown

On 28 October 2026 GitHub made Agentic Workflows generally available: AI agents that triage issues, analyse CI failures and update docs, defined in Markdown and run inside GitHub Actions with read-only defaults.

Read ›

Start a project

Building with AI? Let's talk.

Tell us what you are building. We'll give you an honest view of the right approach.

Start a projectSee our work