LanguageLeap

As companies invest heavily in artificial intelligence, much of the focus has been on building infrastructure—data centers, chips, and computing power. But a growing concern is not just the cost of building AI, but the cost of actually using it at scale.

In this lesson, we examine how AI usage is creating new and often unexpected costs for businesses and consider whether these expenses are sustainable in the long term. As companies increase their reliance on AI tools, some are finding that usage costs can grow rapidly, raising questions about efficiency, productivity, and return on investment.

There’s no such thing as free AI

Warm-up question: Do you think AI saves companies money overall, or does it create new kinds of costs?

Listen: Link to audio [HERE]

Read:

Megan McCarty Carino: A Kai Ryssdal: This is Marketplace, I’m Kai Ryssdal. I think we mentioned Meta and Microsoft yesterday, almost 25,000 positions between the two of them that are either going away or aren’t going to be filled so that those two companies can spend more on artificial intelligence. A lot of that spending is building physical AI infrastructure: all those data centers, computer racks, and GPUs—the hardware of it. But it turns out that actually using AI is turning out to be a bigger and bigger expense for a whole lot of companies. KPMG figures companies are going to nearly double their AI usage spend in the next year to the point where it might start to rival employee salaries in some cases. Marketplace’s Megan McCarty Carino has that one.

Megan McCarty Carino: At a fireside chat at the Federal Reserve last year, OpenAI CEO Sam Altman mused about the future of AI economics.

Sam Altman: It does in fact look like we’re about to deliver on intelligence too cheap to meter.

Megan McCarty Carino: But for now, it’s definitely metered and it’s not all that cheap. The way it’s metered is with tokens.

Max Can: The way that I like to think about it is sort of like the number of turns your interaction with the AI sort of takes.

Megan McCarty Carino: Max Can is a tokenomics analyst at SemiAnalysis. He says a basic conversation with a chatbot takes a handful of turns.

Max Can: You ask the AI some question, it gives you an answer, maybe you ask like two or three more follow-ups, and that’s about it.

Megan McCarty Carino: But new AI agents might analyze hundreds of documents, talk to other agents, and keep doing these things around the clock.

Max Can: It is easily like 100x, 1000x, like maybe even 10,000x more tokens.

Megan McCarty Carino: And some companies like Meta are reportedly “token maxing,” says Daniel Newman at Futurum Group.

Daniel Newman: Employees, they’re basically being encouraged to consume as many AI tokens as possible. And the idea is there’s this direct correlation between token consumption and productivityrecent Goldman Sachs survey of large companies found many are overrunning their AI budgets by orders of magnitude, and AI spend could equal engineers’ salaries in the near future.

Ed Zitron: This doesn’t really make economical sense.

Megan McCarty Carino: Ed Zitron is a tech critic who hosts the podcast Better Offline.

Ed Zitron: I don’t think a lot of businesses are actually aware of how much this really costs.

Megan McCarty Carino: He says tech companies had been subsidizing AI; now they’re increasing prices. And most businesses aren’t really measuring return on their investment, says Brian Jabarian, an economist at the University of Chicago.

Brian Jabarian: How do you prove with hardcore data that AI has been fruitful for your business?

Megan McCarty Carino: The bigger the AI bill, the bigger the benefits needed to justify it. I’m Megan McCarty Carino for Marketplace.

Vocabulary and Phrases:

1. between the two of: combined total of two people or groups
2. rival: to be similar in size, amount, or importance
3. fireside chat: an informal public conversation or interview
4. to meter: to measure and charge based on usage
5. around the clock: continuously, 24 hours a day
6. maxing: using something to its maximum limit
7. by orders of magnitude: by a very large amount or difference
8. subsidizing: financially supporting something to reduce its cost
9. fruitful: producing positive or useful results

Fill in the Blank Use the correct word or phrase from the vocabulary list.

1. The CEO discussed future plans during a __________ at the conference.
2. The government is __________ renewable energy projects.
3. Cloud services are often __________ based on how much data you use.
4. __________ companies, over 10,000 jobs were cut this year.
5. Some users are __________ their data usage to test system limits.
6. Costs increased __________ compared to last year.
7. The partnership proved to be very __________ for both companies.
8. The system runs __________ to support global customers.
9. The new product could __________ the company’s main competitor in sales.

Comprehension Questions:

1. Why are companies cutting jobs at firms like Meta and Microsoft?
2. What does Sam Altman mean by “intelligence too cheap to meter”?
3. How is AI usage currently measured and priced?
4. Why are AI costs increasing so quickly for companies?
5. What concern do critics have about AI spending?

Discussion Questions:

1. Do you think companies fully understand the cost of using AI? Why or why not?
2. Should businesses limit AI usage, or encourage more experimentation?
3. What are the risks of relying too heavily on AI tools?
4. How can companies measure whether AI is actually “fruitful”?
5. Do you think AI will become cheaper or more expensive in the future? Why?