Grok 4.5 vs. Opus 5 coding comparison

I use Grok 4.5 and Opus 5 almost every day, usually in a workflow closer to Copilot than full autonomy.

The difference between them often feels small. That made me wonder whether frontier models are already good enough for most coding tasks, with cost, tooling, and the agent around the model becoming a bigger part of the result.

So I tested that impression on a real task.

I gave issue #582, Kodus Trace, to two agents using /goal. The issue was already detailed. Grok 4.5 ran in Grok Build. Opus 5 ran in Claude Code.

I did not intervene. Both agents received the same instruction: keep working until you consider the goal complete, then open a PR.

• Grok opened PR #1665.
• Opus opened PR #1667.

The question was simple: which complete system would deliver the better PR on a large task without human help?

This was one task and one run per system. It tells us what happened here. A second run, or a different issue, could produce a different result.

How I tested the PRs

I separated four states that often get mixed together when someone says an agent finished a task:

1. The code builds in a clean environment.
2. Every command requested in the issue passes without narrowing the command.
3. The feature works outside the tests written by the agent.
4. The PR can be reviewed and merged without a meaningful amount of extra work.

I tested both PRs from the exact commits delivered by the agents, using the same Node and pnpm versions in isolated environments.

The test covered every command in the Definition of Done, local capture without an account or reachable API, secret removal, path-based recall, pin, forget, distillation, pushing to a local remote, and reading the orphan branch.

I also tested the UI server, session data, truncated records, decisions during code review, and updates to the PR comment.

The main flows ran through the compiled CLI. After reproducing external failures, I called a few internal functions to isolate their causes.

I could not repeat the visual UI check because the browser attached to the session could not access localhost. I tested the endpoints, returned data, and automated cases, but did not create new screenshots.

What Grok delivered

Grok 4.5 in Grok Build worked for 1 hour and 8 minutes.

It made 363 tool calls, changed 100 files, and created 9 commits. Grok Build reported about 620,000 tokens.

The run used a US$30-per-month SuperGrok plan. I estimated that the same usage would have cost about US$20 through the API. That estimate was not an extra charge on top of the subscription.

The agent got the CLI suite passing. It also included evidence of local capture, removal of a fake secret, recall, and orphan branch creation in the PR.

Some of its own evidence showed unfinished work.

The full pnpm check command failed, so the agent ran Prettier and ESLint on the changed scope instead. It ran eight selected backend suites instead of the full pnpm test command required by the issue.

The kodus skills install –dry-run output attached to the PR ended with a `MODULE_NOT_FOUND` error.

Monorepo CI was red. The backend unit and integration job failed because cli-business-validation.integration.spec.ts hit a syntax error while loading a file. The CLI suite passed.

The agent never opened the UI in a browser.

In my independent test, all 800 CLI unit tests and 41 integration tests passed. The build passed too.

The complete Definition of Done stayed red. `pnpm check` failed in Prettier, and the full monorepo test run had seven failing suites.

Two suites required a local PostgreSQL instance. Four failed because web dependencies were missing from the worktree. The final failure came from the PR itself: the syntax error prevented `cli-business-validation.integration.spec.ts` from loading.

Local mode worked without a token. All four hooks exited with code zero and empty stderr, leaving a readable record outside the repository.

The empty state for `trace status` appeared, although it was missing some of the counts requested by the issue.

The main blockers were in the git path.

`trace enable` changes `remote.origin.push`. After that, a regular `git push origin` can stop sending the developer’s working branch.

The code also uses `git commit –amend` to add the trailer. During my test, it pulled a file that was already staged for the next piece of work into the amended commit.

Redaction failed for nested tool-call fields. Secrets remained inside `input`, `output`, and `commands`.

`pin` and `forget` worked in local recall. Their state never reached the context pack used by the backend.

What Opus delivered

Opus 5 in Claude Code worked for 3 hours and 27 minutes.

It made 510 tool calls, changed 174 files, and created 16 commits.

Claude Code reported about 290 million tokens:

– 11,842 input tokens

– 567,265 output tokens

– 3,800,544 cache-write tokens

– 284,146,377 cache-read tokens

The run used Claude Code Max 20. I estimated an API equivalent of US$194. Again, that was an estimate, not an additional charge.

The monorepo CI was green. The agent also opened the UI with Playwright and added screenshots of the session list, session detail, and empty state to the PR.

I repeated the flows from a clean environment.

All 901 CLI unit tests and 44 integration tests passed. The build and `pnpm check` passed.

In the full monorepo run, 606 suites passed. Two failed because there was no local PostgreSQL instance. I did not find a new failure caused directly by the PR in that run.

Local mode worked without a token under the same conditions used for Grok. Opus also passed 59 focused tests covering the store, UI, git behavior, distillation, and the full CLI process.

The external tests still found blockers.

The pre-push hook ignores the ref supplied by git and distills whichever branch is currently checked out. It also runs again when `kodus/trace/v1` itself is pushed, creating a recursive chain of distillation runs and model calls.

Redaction missed passwords inside URLs. A token cut at the summary boundary could also remain partially visible.

Local pin state never reached the backend context pack. The backend queries did not isolate repositories correctly, so decisions from another repository could enter the review.

One construction of `LoadExternalContextStage` requests `BuildTraceContextPackUseCase` without importing the module that provides it. CI stayed green because it did not instantiate that path in the same way the application does during startup.

Why the token comparison is messy

The 620,000 tokens reported by Grok Build and the roughly 290 million reported by Claude Code are made up of different things.

Almost all of the Opus total came from cache reads. Claude Code read 284,146,377 tokens from cache and wrote another 3,800,544. Direct input and output were a much smaller part of the total.

Grok Build did not provide the same breakdown.

Putting 620,000 next to 290 million as if they measured the same usage would be misleading.

The monthly plans and API estimates also answer different questions. The plan price is what I paid for product access. The US$20 and US$194 figures estimate what these individual runs might have cost through an API.

The independent test changed the comparison

I checked the status on August 10, 2026.

Opus delivered green CI and browser evidence for the UI. Grok finished much faster and had a lower estimated API cost, but left the backend red and never tested the UI in a browser.

After I repeated the tests, the Opus PR was closer to merge. Its advantage was clearest in the CLI and distillation coverage.

Neither PR worked from end to end as issue #582 described.

Both handled local mode without an account. Basic recall, skill installation, pruning, status, and UI reads had usable pieces.

The failures appeared at the boundaries between components: choosing the branch inside a hook, publishing records through git, loading decisions in the backend, and carrying local corrections into code review.

Grok could stop a normal push from sending the developer’s branch. Its trailer logic could absorb staged work into the wrong commit.

Opus could distill the wrong branch and invoke itself while publishing the Trace branch.

Both leaked secrets, through different redaction failures.

The complete matrix of 46 acceptance criteria and the evidence for each result are in the issue #582 validation.

What the reviewers found

After both runs, I sent the PRs through Kody and two agents acting as reviewers, Codex and Claude Code. Each received the diff against `main` and worked independently.

Kody reported 11 findings on Grok and 14 on Opus.

Codex reported 6 on Grok and 4 on Opus.

Claude Code reported 15 on each.

The counts were a poor score. Kody’s count made Opus look worse, Codex’s made Grok look worse, and Claude found the same number in both.

A bug that changes git behavior should not carry the same weight as a suggestion to replace `Array.includes` with `Set`.

Kody found several real blockers in the Grok PR. It caught the `remote.origin.push` change, destructive hook removal from `~/.codex/config.toml`, missing redaction for tool inputs and outputs, and the risks created by `git commit –amend`.

It also found recursion when pushing the orphan branch, a write race between processes updating the same session, and cleanup code that deletes the pin and forget files after 90 days.

Some findings had much less value. Kody marked `Array.includes` inside a loop as critical, the same severity used for secret exposure and commit rewriting. It requested `PinoLoggerService` in a CLI that already uses `console.error`, and treated sequential removal of three hooks as an important problem.

For Opus, Kody caught the recursive push and incorrect branch selection during distillation. It also found an old hook left behind after an upgrade, decisions crossing repositories, a trailer connected to a session from another branch, duplicated turns, and exposure of the local UI to DNS rebinding.

Its performance concerns about reading every session on each commit and starting multiple git subprocesses for every UI request were reasonable, although they still need measurement. Suggestions involving `Promise.all`, partial TypeORM selection, or a trie for small lists were less useful than the functional bugs.

Codex returned fewer findings, and almost every one described a direct failure.

For Grok, it caught the recursive pre-push behavior and the change to `remote.origin.push`. It also found two risks Kody missed: backend decisions queried without team or repository filters, and captured commands stored without credential redaction.

Its remaining findings covered decisions linked to the wrong paths and PR comment updates across git providers.

Every Codex finding on Opus was relevant. It reported recursive pushes, decisions mixed across repositories, and the old hook that remains after an upgrade.

Its final finding connected local state to backend behavior: `pin` and `forget` update local data, while code review builds its context pack from backend data. A decision the user forgot can still enter the review.

Claude Code covered the most execution paths.

For Grok, it found the git and configuration removal problems. It also caught unstable IDs in `pin` and `forget`, along with comment IDs converted to `Number`, which can produce `NaN` outside GitHub.

Its most serious Opus finding was the API startup path. `LoadExternalContextStage` depends on `BuildTraceContextPackUseCase`, but `PromptsModule` creates the stage without importing the module that provides that dependency.

Claude also found redaction cases that passed the existing tests. One replaces the wrong occurrence when a URL username and password contain the same text, as in `postgres://postgres:postgres@…`. Another truncates content before redacting it, leaving part of a token visible when it crosses the cutoff.

It also reported lost records on the orphan branch, improper removal of the Trace hook, and overly broad decision scope.

The reviewers agreed on several Grok failures.

All three found that `git push` could stop sending the developer’s branch.

Kody and Codex found the recursive pre-push behavior. Kody and Claude found the commit amendment risk, destructive TOML removal, and pruning of pin and forget state.

Codex and Claude both found decisions receiving paths from other sessions and sticky comments breaking with IDs from other providers.

On Opus, all three reviewers found recursive pushes, decisions crossing repositories, and the old hook that remains after an upgrade.

Codex and Claude agreed that pin and forget do not affect the context pack used in review.

Kody and Claude both reported full session-history reads on every commit and the DNS rebinding risk in the local UI.

When all three reviewers independently reached the same failure scenario, I treated it as a blocker until a test proved otherwise.

Which reviewer helped most

Claude Code gave me the most useful review in this experiment.

It covered the widest set of paths that could prevent someone from using the feature. It also did the best job of connecting code across files into one user-visible failure.

Only Claude caught the missing API dependency in `PromptsModule`. It also found redaction cases that passed the existing tests, including the repeated username and password inside a connection string.

Codex had the highest signal density. Its reports were shorter, and almost every finding connected the code to something a user could observe.

It left more of the diff unreviewed, so I would use it as a focused triage pass and still run a second review on a PR this large.

Kody came third. It found blockers and attached useful context to the relevant PR lines. The report required more filtering because functional failures, style rules, and speculative performance concerns often received similar severity.

I did not measure the time or cost of the review runs. This comparison also covers complete review tools. Kody used the repository’s current configuration, while Codex and Claude Code used their own review flows. The result does not isolate the model behind each reviewer.

I also have not finished two parts of the comparison: diff scope and noise, and the amount of human correction time required to make each PR mergeable.

My result

	Grok 4.5 / Grok Build	Opus 5 / Claude Code
Runtime	1h 08m	3h 27m
Tool calls	363	510
Diff	100 files, 9 commits	174 files, 16 commits
Tokens reported	~620,000	~290M (mostly cache)
Est. API cost	~$20	~$194
Monorepo CI	Red	Green
UI in a browser	No	Yes (Playwright + shots)
Independent retest	Build + 841 CLI tests passed. Full DoD still red.	Build, pnpm check, 945 CLI tests passed. Closer to merge.
Secrets	Nested tool-call fields	URL passwords, truncated tokens
Worst git bug	Can stop git push from sending the working branch	Distills the checked-out branch and can recurse on Trace push
Mergeable	No	No

The small difference I feel during Copilot-style work did not carry over to this autonomous task.

Opus worked longer, made more tool calls, changed more files, delivered green CI, and included browser evidence.

Grok finished faster and had a much lower estimated API cost, but stopped with a backend failure.

The reviews made both PRs look worse than they first appeared, and the external tests confirmed several blockers.

Opus provided more execution evidence. It still had failures in API startup, push behavior, repository scoping, and protection of captured data.

Grok introduced more immediate risks to the developer’s environment. It changed normal push behavior, could rewrite commits with unrelated staged files, and could remove parts of the Codex configuration.

I would run either solution only inside a disposable environment. Both PRs have enough blockers to prevent a merge.

Opus delivered the better PR in this experiment. It covered more of the issue, got closer to a green Definition of Done, and tested more of the new behavior.

Its estimated API cost was almost ten times higher. It also worked for roughly three times as long. That extra spend produced a solution closer to the goal, with a meaningful amount of work still left before release.

My opening question was whether frontier models had reached a plateau for most coding work.

This experiment found something narrower.

For local code generation inside a Copilot-style workflow, the models can feel close. During a long autonomous task, the difference showed up in how many paths the agent remembered to build and test.

Both agents still declared the goal complete with bugs at the feature’s most sensitive boundaries.

I do not think we have reached a plateau for finishing large software changes safely.