UpTrajectory Review
Harvey, a legal AI startup, has seen its token usage skyrocket from 1 trillion in January to an expected 12 to 13 trillion by May. This dramatic increase highlights the growing reliance on AI in the legal sector, particularly for tasks like document drafting and analysis. CEO Winston Weinberg's comments suggest that while AI can enhance efficiency, the costs associated with extensive token use are becoming a pressing concern for firms.
For small business owners, especially those in legal services, this trend underscores the importance of evaluating AI investments carefully. As AI tools become more integral to operations, understanding the cost implications of token usage is crucial. Weinberg's caution against excessive AI spending mirrors broader industry sentiments, where companies are reassessing their AI strategies to avoid unsustainable expenses. This week, operators should consider how much they rely on AI and whether the benefits justify the costs.
Takeaway: Evaluate your AI usage and costs to ensure sustainable operations.
From the original item — Business Insider:
Big Event Media/Getty Images for HumanX Conference
- Legal AI startup Harvey used 1 trillion tokens in January. That jumped to 12 trillion in May, CEO Winston Weinberg said.
- “You don’t want frontier intelligence running every task. It’s too expensive,” Weinberg said in a statement to Business Insider.
- He compared the coming debate over AI spending to law firms’ billable hours.
Harvey is burning through a lot more AI tokens. Trillions more.
The legal AI startup used 1 trillion tokens in January, CEO Winston Weinberg told the “Sourcery with Molly O’Shea” podcast, released on Tuesday. Harvey’s AI product is used by law firms and corporate legal teams to help draft, review, and analyze documents.
The startup’s monthly token use has since ballooned roughly 12X.
Weinberg said the company was on pace to use 12 trillion to 13 trillion tokens in a month. A Harvey spokesperson confirmed to Business Insider that the estimate referred to May.
“The usage is getting insane,” Weinberg said.
Tokens are the chunks of text that AI models process when users type prompts and receive answers. They have also become a key measurement of AI use and billing. Generally speaking, the more complex an AI task is or powerful the AI model processing the request, the more tokens are burned.
The remarks from Harvey’s CEO come as several tech companies are paying closer attention to their AI usage costs. Business Insider recently reported that some startups are pulling back from “tokenmaxxing” after receiving high AI bills.
A few examples of the token reassessment: Coinbase is routing some prompts to cheaper models, and Uber executives have questioned whether spending on AI coding tools is translating into enough measurable product gains.
Weinberg said the next challenge for companies is deciding the right level of AI use for each job.
“You don’t want frontier intelligence running every task. It’s too expensive,” Weinberg said in a statement to Business Insider. For example, “a change of control review warrants it. A first-pass document summary doesn’t,” he wrote.
On the podcast, Weinberg compared the coming debate over AI spending to how law firms justify their bills to clients. He called it the “billable hours problem.”
“When you get a bill from a law firm, it says in six-minute increments what people did, and then the hourly rate,” Weinberg said. “Why did they do that? It’s because they’re trying to show ROI.”
As companies spend more on tokens, he said, they will face growing pressure to explain what that spending actually accomplishes.
“This is kind of like the main problem that I think the entire world is about to hit,” he said, “Which is, ‘I just spent $1 billion on tokens. Where’s my ROI?'”