AI Infrastructure Spending Accelerates as Inference Costs Collapse

Remy: We've reached the tipping point for the next gen of the cloud. Over the last year, we've seen AI inference costs collapse by more than 99%, and that crash in price is fueling a massive explosion in demand. Now, to keep up, the physical foundations of the internet are being rebuilt. And since the ChatGPT moment, data center growth has accelerated from 5% to nearly 30% annually. And we're talking about a CapEx wave of over $500 billion. And while these spending looks like the .com boom, the valuations do tell a different story. So joining me to weigh in this morning is Frank Downing, director of research of AI and cloud at ARK Invest. Well Frank, great to have you here. Thank you so much for joining me. Well we know we've seen plenty of volatility in the sector. I understand you've noted that AI inference costs have dropped by a staggering percentage point in a single year. So usually when costs drop that fast it does signal a commodity race to the bottom.

But you're seeing it drive a 25 fold increase in usage. So tell us what's going on here.

Frank: Yeah. Happy to. And thanks for having me, Remy. I think, you know, stepping back to give some context. We're seeing adoption of AI. If you look at penetration relative to the internet is happening twice as fast. And on the enterprise side, I think, you know, these models are just hitting a really critical inflection point where the capability and the products built around them and the costs coming down is unlocking a lot of new demand, which is why you're seeing this, you know, 25x growth in token consumption. And I think, you know, to put that cost decline in context, we look at costs and how quickly they're coming down as a cornerstone of our research. It's what led us to invest in companies like Tesla when nobody thought electric vehicles were going to be possible. Seeing costs come down in batteries led us to the belief that electric vehicles would not only be possible, but also profitable at scale. And we think that same story is playing out in AI, with the costs coming down even more quickly. And as these costs come down, they unlock new use cases and that incentivizes more demand. And I think you can look at companies like OpenAI and Anthropic that are, you know, multi-billion dollar businesses that are tripling or 9xing year on year to see how much value is being created even as those costs come down.

Remy: Yeah. And you've been doing extensive research, big tech spending half $1 trillion on infrastructure this year. And critics, some critics that is, are calling it a bubble. So why does your research say that this is not 1998 all over again?

Frank: Yeah. No, it's a fair question. And I think to, to add, you know, context, we looked at tech spending, relative to that late 90s, early 2000 period. And you're right, the levels are at the same relative to GDP that we saw during that time period, and they're projected to go higher. I think what's interesting, and if you look at this chart, is that tech spending as a percentage of GDP has been going up since the bottoms of post .com and post great financial crisis, as technology has become such a more core part of our day to day lives and a more core part of the economy. So I'm not surprised to see that trend. And I think investors are right to question, you know, is this spending too much? I think the valuation point is important. If you look at the big tech companies today versus the big tech companies during that period of time, they're actually much more reasonable - 40x earnings relative to peaking at 100x plus. And I think obviously you can see the rolling selloffs across AI space and software in particular, as underperformed so much as an example of how skeptical the market is right now. But we think, you know, the underlying demand trends, seeing what's happening, particularly in the private markets and the amount of demand being driven for AI tools like Anthropic's Claude or OpenAI's ChatGPT is going to fuel a lot more infrastructure investment over the next five years. We think that $500 billion in spending is going to grow to $1.4 trillion by 2030.

Remy: Yeah. And when we take a look at the big names, we know that the 2026 landscape is starting to look crowded. So Nvidia has enjoyed an 85% market share and 75% gross margins. But Google and AMD are also catching up. So do you think Nvidia's crown is finally at risk, or would you say that they are untouchable?

Frank:Yeah, Nvidia's had an amazing run, and they've been investing in data center compute much longer than the rest of the field. But there's a strong incentive by their biggest customers to diversify their supply chain and find alternatives that are competitive on a performance adjusted basis. And we think AMD is a great story there where they have consistently executed well. Gaining share on Intel in the data center CPU space. They've competed with Nvidia in the consumer space for quite some time, and their chips are now competitive on some workloads. And the chips that are coming out, particularly their Helios rack and the MI450 series in the second half of this year, are going to make them even more competitive. So they've won contracts with Microsoft, Meta, OpenAI, and I think that's going to lead to a positive share gain story. We have AMD as a top pick across our funds. It's in the top ten of ARKK. And it's the number two holding in ARKW, which is our fund that has the most AI exposure.

Remy:Yeah. And Frank, before I let you go, we have about 60s here. I understand you talk about a constellation of AI driven, intelligent devices that will permeate our lives. So break this down for us.

Frank: Yeah, I think, you know, you can see AI is being delivered into every connected device that we have, whether it's your phone or your laptop. And there's a whole new wave of connected devices coming. I'm wearing my meta glasses, for example, right now, where you can now take something that looked like a toy 15 years ago when Google was testing out Google Glasses and now has, you know, some real consumer value. And I think, you know, what you can do when you put a chip in a large language model on these devices where you can get feedback and interact with your everyday surroundings is going to be quite profound over time.

And I think we're going to see every device connected, the internet become an AI device over time.

Remy: Well, Frank, we will have to leave it there, but a lot to keep our eyes on as we head into the series. So thank you so much for joining us and weighing in.

Frank: Thank you for having me.