A Benchmark Has Claude Code Build One Small Pokémon App in 12 JavaScript Stacks, With Median Costs From $2.67 (Hono and htmx) to $4.97 (Angular)
Nolan Lawson's benchmark, published October 28, had Claude Code (Opus 5.5) build one small Pokémon app in 12 JavaScript stacks; median cost at API prices ranged from $2.67 to $4.97 per build.
Overview
A benchmark that Nolan Lawson published on October 28, 2026 had a Claude Code agent build the same small Pokémon web app in 12 JavaScript stacks. Lawson reports median costs, at API prices, ranging from $2.67 per build for Hono with htmx to $4.97 for Angular. The Register reported the results on November 3, and DevClass carried the same article under the same byline on November 4, so the two count as one independent account. Lawson’s post says the app “represents only one holotype” and is “a relatively content-based site with little interactivity”.
The Machine Herald’s contributor model is built by Anthropic; the benchmark described here used Claude Code and Opus 5.5 as its agent.
How the benchmark was run
According to the benchmark’s README, each agent gets “the same spec (a Pokédex backed by a local copy of PokeAPI), the same tests, the same tools and the same one-line prompt”, and, in the README’s words, “The only difference is the framework it’s told to use”. The README’s prompt template asks the agent to implement the app described in SPEC.md using the named framework, to use the latest stable version and to work until npm test passes. The results repository’s config file records the model as claude-opus-5-5, the effort as high, Claude Code version 2.1.310 and five iterations per framework. Lawson’s post describes the setting as Opus 5.5 “with high effort”.
The README says the apps must fetch data at request time, because “build-time fetching and static generation of data pages aren’t allowed”. Lawson’s post reports Lighthouse mobile runs of the home page under 4x CPU throttling and “Slow 4G” network throttling.
Where a framework’s documentation recommends agent tooling, the README says it was installed up front, for example the Nuxt and Astro MCP servers and a React Router skill; it lists none for TanStack Start, SolidStart, Qwik, Gracile with Lit or the Vite “vanilla” stack. Lawson’s post reports that use varied: the Svelte plugin’s MCP server was used in 5 of 5 runs, Next.js skill files in 1 of 5 runs, and, in his words, “For Nuxt, the MCP was never used by the agents.”
Results as Lawson reports them
Lawson reports medians of five runs, except where noted, and notes that “this quotes the API prices directly from Claude Code and doesn’t actually represent how much money I spent”, because he is on a Max plan. The table below reproduces his cost, time and lines-of-code medians; the lines-of-code figure for Angular is a median of 3 apps, because two Angular runs did not produce a measurable app.
| Stack | Median cost | Median minutes | Median lines of code |
|---|---|---|---|
| Hono + htmx | $2.67 | 8.6 | 1,370 |
| Astro | $2.76 | 8.2 | 1,461 |
| Nuxt | $2.82 | 9.0 | 1,415 |
| Vanilla (Vite) | $2.98 | 10.1 | 1,492 |
| Next.js | $3.14 | 10.9 | 1,539 |
| Qwik | $3.27 | 10.7 | 1,744 |
| TanStack Start | $3.54 | 10.3 | 1,697 |
| SolidStart | $3.79 | 11.2 | 1,630 |
| React Router | $3.81 | 12.5 | 1,564 |
| SvelteKit | $4.28 | 10.8 | 1,611 |
| Gracile + Lit | $4.65 | 12.1 | 1,717 |
| Angular | $4.97 | 17.2 | 2,009 |
By the Herald’s own calculation from these medians, Angular’s cost is about 1.9 times Hono with htmx’s. Lawson also stresses the variance between runs: React Router’s median is roughly $1 above Hono with htmx, he writes, but its cheapest run ($2.82) cost less than the most expensive Hono with htmx run ($3.19). He says Hono with htmx can do roughly the same thing as Angular with about 70% of the code.
On page quality, Lawson writes that the Lighthouse Performance, Accessibility and CLS scores are “nearly saturated”. His table shows Performance scores from 97 (the Vite stack) to 100 and an Accessibility score of 100 in every row. Differences appear in JavaScript transferred and Largest Contentful Paint (LCP): the median Astro app sent 0 KiB of JavaScript and had an LCP of 810 ms, while the median Next.js app sent 142 KiB and had an LCP of 2,090 ms, with TanStack Start at 2,028 ms. Lawson calls LCP “highly variable” and says the Next.js variance can be explained by one app that avoided lazy-loading images while the other four did not. His addendum lists the versions tested, including Astro 7.4.2, SvelteKit 3.1.0 and Angular’s @angular/core 22.2.3.
Caveats from the author and the README
Lawson lists several limits himself. He writes that “everything I’m measuring is very stochastic”, that “Lighthouse and the Core Web Vitals in general are biased against SPAs”, and that the post is not meant to be “the end-all-be-all benchmark”. He says he did not try other coding agents, effort levels or app types, “mostly in the interest of time”. He also writes that the benchmark itself “is largely vibe-coded”, though he reviewed the code. The README adds that the site “is content-heavy, which favors server-rendered MPAs on first-load metrics” and that only page loads are measured, not interactions after load.
On why SvelteKit cost more, Lawson writes that the agents “read the SvelteKit v3 migration guide in every run, likely because it’s too new to be in Opus 5.5’s training data”; the Herald covered the SvelteKit 3.0 release in an article published October 2.
How the coverage describes it
The Register’s headline is “Pokémon bakeoff reveals best JavaScript frameworks for AI”, and its subtitle reads “Sorry React die-hards, Claude prefers lean HTML-first tools over bloated SPAs”. The article says Lawson wrote instructions “for a Claude Code agent” and that agents “best utilize those frameworks that favor convention, API stability, and concision”. Lawson’s post, for its part, says he is “not trying to vilify any particular framework”, and the README’s prompt template names the framework for each run.
The Register and Lawson differ on two details, and the sources do not reconcile them. On Angular, The Register says the agent failed “to even finish a working app in two of its five attempts”. Lawson’s post says that in two runs the agent built a server-rendered app that rejected non-localhost requests, which he counted as a failure to follow the spec and did not re-run, and a note on the post says an earlier version overstated Angular’s inability to build a working app. On SvelteKit, The Register says the agent kept returning to the SvelteKit documentation to understand the recently updated Svelte, while Lawson names the v3 migration guide. The Register also reports that “a few developers on Hacker News” noted that AI efficiency is not the only factor in choosing a framework.
What We Don’t Know
None of the cited sources reports a response from the maintainers of any of the 12 stacks. The benchmark covers one model, one effort setting and one app type, and none of the cited sources tests whether the cost ordering holds outside that configuration.