UpTrajectory Review

A coalition of AI researchers has put numbers on a quiet inequity that anyone paying an LLM bill by the token already suspects: the same prompt costs wildly different amounts depending on which language you type it in. Their study, reported by Rest of World, found that among the twenty languages with the largest web presence, less common languages consumed 1.5 to 5 times as many tokens as English for identical requests. Expand the test to two hundred languages and one model needed as many as 17 times the tokens. Tokenizers, the components that chop text into billable chunks, were trained on oceans of English and European web text. They handle those languages efficiently. They splinter Burmese, Khmer, Tibetan, and Lao into fragments, and the meter runs the whole time.

If your business serves customers in South Asia, Southeast Asia, or East Africa, or if you run a support desk, a chatbot, or a translation workflow in those markets, this is your cost structure the study is describing. Punjabi has roughly 200 million speakers, Bengali about 266 million, Urdu about 288 million. Yet a Bengali-speaking customer who asks your chatbot the same question as an English speaker burns several times the tokens, slows the response, and crowds the context window. You pay more per interaction, you serve fewer turns per session, and your service feels slower to precisely the customers you spent money to reach.

The genuinely new element here is the framing as a tax with a measurable rate. Earlier complaints about language bias in AI focused on accuracy and representation. This study prices the bias. It also punctures a comfortable assumption: the penalty does not track speaker population. It tracks digitized web presence. Latin-alphabet languages with small speaker bases often tokenize cleanly, while massive languages written in non-Latin scripts pay the steepest multiples. That inverts the usual economies-of-scale logic and makes the gap harder to dismiss as a niche problem.

The downstream effects compound. Higher per-token costs discourage companies from building or localizing AI services for exactly the markets where those services could do the most good. Slower responses and tighter context limits mean a worse product for speakers of underrepresented languages, which suppresses adoption, which shrinks the digitized corpus, which worsens the tokenizer. The researchers call this a driver of digital inequality; for an operator, it is also a product-design trap. A pricing model that looks flat per token is regressive in practice.

Watch whether the major model providers respond with tokenizer improvements, language-specific pricing, or both. A provider that ships a more efficient tokenizer for Burmese or Amharic is handing you a cost cut on your next invoice. In the meantime, audit your own usage. If you serve multilingual customers, benchmark token consumption per language on your actual prompts, and price multilingual deployments with the tax in view rather than discovering it in the bill.

“This means hundreds of millions of people billed at multiples of the English rate for the same intelligence.” — Computerworld

Takeaway: Benchmark token costs per language before deploying AI in multilingual markets; the same prompt can cost up to 17 times more in Burmese than in English.