The Pig Knuckle Papers
The Hidden AI TaxWhy cheaper tokens keep costing you more.
Per-token prices fell 67% in a year. Enterprise AI bills doubled and tripled anyway. The gap between them is a tax nobody puts on the invoice — paid in tokens you're burning without knowing it: agentic loops that never quit, context stuffed and re-sent, and prompts that discover instead of deliver. Here's where it hides, and how to stop paying it.
The bill went up when the price went down
Late 2025 was the era of tokenmaxxing — model performance and agentic autonomy at any cost. Mid-2026 is the hangover. The price of intelligence collapsed: blended per-token cost fell from $18.40 to $6.07 per million tokens in a single year, a 67% decline. And yet enterprise AI invoices didn't fall with it. They rose — two to three times over.
That's the Efficiency Paradox, and it isn't a billing error. It's the predictable result of using cheaper tokens far more wastefully: longer context windows, recursive agent loops, retrieval chains that re-fetch the same documents, and almost no centralized governance watching the meter. The unit got cheaper; the number of units went vertical.
per-token price in a year — $18.40 → $6.07 per million tokens (Q1 ’25 → Q1 ’26)
the enterprise AI invoice over the same year — bills rose as prices fell
of enterprise token spend is consumed by agentic loops alone
Where the curve bends
A human prompts. An agent spirals.
Human-in-the-loop prompting is linear and session-bounded: you ask, it answers, the meter rests. Autonomous agents don't rest. Every recursive tool call, planning step, self-correction, and re-injected context stacks onto the last. Across 2.4 billion enterprise API calls, agents consume dramatically more tokens per unit of business value than the humans they were meant to relieve.
tokens
A human developer debugging a function.
tokens
The same job as an agentic workflow — retries, replanning, context re-sent each step.
The industry stopped pretending this was free on 1 June 2026, when GitHub Copilot swapped flat-rate pricing for metered AI Credits. The all-you-can-eat AI buffet is closed. Compute is metered now, and the token bill is a bill like any other.
The soft taxes nobody logs
Not all of the waste comes from agents. Some is a quieter tax on organizational maturity. In verticals with lower prompt literacy — marketing, consulting, financial services — new users run long clarification loops: over-specified prompts, persona preambles north of 1,000 tokens, and regeneration after regeneration of rejected output. Monitoring from Helicone and LangSmith puts up to 25% of token spend in non-technical departments on this discovery, not on production work.
Reasoning models add their own surcharge. The o3, o4-mini, and GPT-5.4 Thinking classes generate extended internal chain-of-thought — billable tokens the reader never sees. Essential for multi-step logic; ruinous for simple work, where cost-to-value ratios routinely exceed 10:1 on tasks a small model could handle: JSON extraction, document classification, formatting.
The most expensive tokens in the enterprise aren't the ones doing the work. They're the ones spent discovering, retrying, and second-guessing.
Token-hygiene profiles vary sharply by sector. Financial services runs hot on AI-Act compliance — verbose logging and multi-agent audit loops. Consulting and deep research lean on context-window stuffing, feeding entire document sets into million-token windows instead of building a real RAG pipeline. Marketing and comms regenerate long-form copy five to ten times chasing brand voice. And writing, publishing, and lower-tech teams show the weakest practices of all: no caching, no compression, no small-model routing.
Where the tokens actually go
Six kinds of work, one enterprise token budget. The share of spend, the waste it carries, and the single lever that pulls it back.
Recursive tool-calling loops, with context re-injected on every step.
Lever · Token budgets · SLM routing
Context-window stuffing and redundant re-retrieval.
Lever · Semantic caching · prompt compression
Multi-agent search loops and billable chain-of-thought.
Lever · Reasoning-model tiering
Multi-draft iteration and rejected outputs.
Lever · Output caching · few-shot tuning
Trial-and-error prompting and persona preambles.
Lever · Prompt libraries · upskilling
Clarification loops and discovery exchanges.
Lever · Prompt templates · training
One category dominates. Agentic workflows are 45% of enterprise token spend — nearly half the bill — and carry a high waste ratio. Govern that one line, and you've moved the whole invoice.
The diagnostic
Every token forks
At the moment of the request, each token is heading toward one of two fates — value or waste — and a single gate decides which. Same model, same price. The difference is whether anyone is governing the call.
✕ Waste
No budgets, no circuit breakers — recursive loops run up runaway spend.
Fix · Token budgets · SLM routing
✓ Value
Token budgets + circuit breakers keep the loop contained.
✕ Waste
50k+ tokens of stuffed context, re-retrieved every call — mostly noise.
Fix · Semantic cache · reranking · compression
✓ Value
Compressed, reranked, cache-hit context — only what the question needs.
✕ Waste
Trial-and-error prompts and 1,000-token persona preambles — discovery, not output.
Fix · Prompt templates · training
✓ Value
A prompt library gets the answer on the first try.
✕ Waste
An o-class model chews chain-of-thought on JSON extraction — 10:1 cost-to-value.
Fix · Model tiering · SLM routing
✓ Value
Complex logic gets the reasoning model; the rest routes to an SLM.
How to stop paying it
The tax isn't inevitable — it's ungoverned. Four moves, in order of leverage, take most of it back.
1. Govern the agents.
Agentic loops are 45% of the bill and the top waste driver. Give them token budgets and circuit breakers so a runaway loop trips instead of billing — and route sub-tasks to small models rather than firing the flagship at everything.
2. Fix retrieval before you blame the model.
Stuffing a million-token context on every call is the expensive way to be wrong. Semantic caching, reranking, and compression feed the model only what the question needs — and turn repeat questions into cache hits, not re-retrievals.
3. Tier the models to the task.
A reasoning model grinding chain-of-thought on JSON extraction runs a 10:1 cost-to-value ratio. Send genuine multi-step analysis to the o-class models; route classification, extraction, and formatting to standard or small language models.
4. Upskill the humans.
Up to a quarter of token spend in non-technical teams is discovery — trial-and-error prompts and 1,000-token persona preambles. A shared prompt library and a little training convert that discovery tax into first-try answers.
The model didn't get more expensive. We did.
The price of a token fell off a cliff. The bill went up anyway — because the savings were handed back as retries, re-retrievals, and reasoning nobody needed. The Hidden AI Tax isn't a line on the invoice; it's the gap between what intelligence costs and how carelessly we spend it.
The metered era makes that gap impossible to ignore. When compute is billed by the credit, governance stops being hygiene and becomes a line item — budgets, caches, model tiers, and prompt discipline that turn waste tokens into value tokens. The organizations that win the next year won't be the ones with the biggest models. They'll be the ones who stopped paying a tax they never knew they owed.
About the Pig Knuckle Papers: The Pig Knuckle Papers are a human-led, AI-assisted research series published by Alchemy Agentic. Each paper begins with a human question, uses Pig Knuckle, Alchemy Agentic’s flagship LLM-orchestration product, for deep research, and undergoes human review before publication. Keith Norton, co-founder of Alchemy Agentic, serves as editor and narrator. Paul Langtry is co-founder of Alchemy Agentic. ChatGPT may be used to shape approved research into the intended human voice, but humans review and approve every final piece.
Sources
- [1]GitHub Copilot Billing Change FAQ — Lantern
- [3]GitHub Copilot AI Credits: Usage-Based Billing Starts June 1, 2026 — Windows Forum
- [5]AI Cost Optimization: Cut LLM & Infrastructure Costs — Alice Labs
- [6]AI Token Costs: Why Enterprise AI Bills Keep Rising in 2026 — Optimum Partners
- [10]Claude Opus 4.5 vs Gemini 3 Pro vs GPT-5: The Ultimate Agentic AI Showdown for Developers — Klavis
- [11]LLM vs. SLM vs. FM: Choosing the Right AI Model for Enterprise Workloads — StartupHub.ai
- [12]Reduce LLM Cost and Latency: A Comprehensive Guide for 2026 — Maxim AI
- [13]GPT-5.4 vs Claude Sonnet 4.6 — honest take (r/AI_Agents)
- [15]LLM Cost Optimization: How to Cut Spend 50–90% — LeanLM
Additional research consulted
Gathered during research but not directly cited above.
- [2]GitHub Copilot Billing Is Changing: How to Prepare Before June 1 (video)
- [4]7 Tools for Tracking AI Token Usage Across Vendors in 2026 — Torii
- [7]GitHub Copilot Shifts to Usage-Based Billing June 1 2026 — LinkedIn
- [8]GitHub Copilot is moving to usage-based billing (#192948)
- [9]GitHub Copilot’s New Usage-Based Billing: What Changed, Why Developers Are Upset — GAP Velocity
- [14]Devs Sound Off on Usage-Based Copilot Pricing Change — Visual Studio Magazine