Analysis Reveals Hidden AI Token Costs of Popular JavaScript Frameworks

In the rapidly evolving 'agentic era' of software development, where large language models (LLMs) are increasingly taking the helm in generating code, developers are facing an unexpected new line item on their budgets: AI token costs. What was once a subjective debate about developer experience (DX) and perceived performance is now a quantifiable financial reality. A groundbreaking new benchmark by veteran web developer Nolan Lawson, published on October 28, 2026, exposes how the choice of JavaScript framework directly translates into significantly higher or lower AI inference bills, potentially impacting a project's bottom line in ways previously unimagined.

For years, the web development community has been criticized for making technology choices based on 'vibes' – frameworks promising to be 'blazing fast' or 'supercharge' development with little to no empirical evidence. As Lawson himself notes, this trend has persisted even as AI agents begin to take over coding tasks, with claims of frameworks being 'better for agents' or resulting in 'fewer mistakes' often made without benchmarks. But in a world where every generated token costs money, these 'vibes' are no longer harmless preferences; they are direct contributors to operational expenses.

The Benchmark: Pokémon App Meets AI Agents

To cut through the noise and introduce much-needed evidence, Nolan Lawson designed a straightforward yet insightful benchmark. He instructed advanced AI agents, specifically Opus 5.5 with high effort settings, to build a small but non-trivial Pokémon application using the PokéAPI. The app, featuring lists, search, and filtering functionalities, served as a 'holotype' – a content-based site with moderate interactivity.

src="https://nolanlawson.com/_astro/pokemon-app.DUO5bAVN_34jQQ.webp" alt="Screenshot of a Pokémon web app with various monsters listed, their types, etc."

A screenshot of the Pokémon application built by AI agents across various JavaScript frameworks, used to benchmark token consumption and efficiency. (Credit: Nolan Lawson)

The agents were tasked with building the app using a dozen popular JavaScript frameworks and meta-frameworks, including Next.js, Nuxt, SvelteKit, Angular, Astro, Hono+htmx, Qwik, TanStack Start, SolidStart, React Router, Gracile+Lit, and even 'Vanilla' JavaScript. Each app was then rigorously tested against a spec conformance suite and evaluated across several critical dimensions: development time, total tokens spent, lines of code, and Lighthouse performance and accessibility scores. Lawson ran five iterations for each framework (except Angular, which failed twice, yielding three runs) to account for the stochastic nature of LLMs, providing a robust median for his findings.

The Surprising Economics of Frameworks in the AI Era

The results are a stark reminder that verbosity and complexity have a direct dollar cost when AI is doing the heavy lifting. Lawson's analysis, based on Claude Code API prices, reveals significant discrepancies in build costs:

• Most Cost-Efficient: Hono+htmx ($2.67), Astro ($2.76), Nuxt ($2.82), and Vanilla ($2.98) emerged as the most economical choices for AI code generation.
• Most Expensive: SvelteKit ($4.28), Gracile+Lit ($4.65), and particularly Angular ($4.97) were notably more expensive to build.

These costs tracked closely with the total tokens consumed. Angular, for instance, required a median of 9.76 million tokens, compared to Hono+htmx's 4.26 million. Lawson attributes SvelteKit's higher cost partly to agents needing to consult its v3 migration guide, suggesting that newer frameworks or those undergoing significant changes might incur higher initial AI costs due to less established training data. Angular's cost, on the other hand, appears to be a direct function of its larger code size. Interestingly, 'Vanilla' JavaScript, despite its simplicity, surprisingly underperformed several frameworks in cost and time, indicating that a well-suited framework can indeed offer efficiency benefits even for AI agents.

Beyond Cost: Performance and Maintainability Insights

While Lighthouse performance and accessibility scores were generally high across the board for this small app, other metrics revealed deeper implications. The JavaScript bytes transferred, for example, showed Astro at a mere 0 KiB, while Next.js topped the chart at 142 KiB. This difference, though not always reflected in the overall Lighthouse performance for this specific app, hints at the potential for larger client-side bundles that could impact user experience on more complex applications.

Perhaps most illuminating for long-term project health is the 'Lines of Code' metric. Hono+htmx generated the app with just 1370 lines, whereas Angular required a hefty 2009 lines. This isn't just an aesthetic preference; as Lawson emphasizes, the total code size directly impacts future token consumption for edits and maintenance. In the agentic era, a more succinct codebase isn't just 'nicer' for developers; it translates into lower ongoing AI costs for modifications and bug fixes. The adage that frameworks are 'tools for organizing your mind' now extends to 'tools for organizing your AI's token budget.'

Lawson's benchmark, while acknowledging its limitations (one app 'holotype', stochastic LLM behavior), serves as a critical first step towards evidence-based decision-making in AI-powered development. It suggests that framework authors and the broader community must begin to consider 'AI cost' as a core metric, optimizing not just for human DX or traditional performance, but for token efficiency. The era of 'vibe-coding' is giving way to the era of 'cost-coding,' and understanding these hidden AI token costs will be paramount for developers and businesses alike.