How to forecast token spend
Recently, I sat down at a table full of CFOs, and the question came up:
How are you forecasting compute spend in the age of AI?
Most times at these rounds, the conversation is vibrant. Lots of folks eager to weigh in.
But this question left everybody stumped.
And when they did talk, it was more complaints and confessions. One CFO admitted he’d unintentionally personally spent $100k in a single month building with agentic AI.
The general mood around the table was stress.
But then something struck me: Wait…
...Is this just the classic Startup Mode/Finance Mode Problem?
The Startup Mode/Finance Mode Problem is something I discovered first when teaching financial modeling at NYU Stern.
My course was originally just for founders-
Folks without a finance background
Often engineers
And it was geared around teaching them financial modeling skills in general, from a first principles lens.
But to my surprise, experienced modelers started showing up (like, bankers, PE guys, VCs, FP&A folks, accountants etc).
I was intimidated at first, then just confused, because to my surprise they LOVED it.
I was of course delighted, but I didn't really understand what they could possibly be getting out of a beginner course.
Did they just want to steal my course material? 😬
No.
The thing they were there to learn was the Startup of it all.
On my personal path to CFO - going from engineering to operations to CFO - I'd always considered startup modeling the easiest build.
What I didn't realize is that for traditional financial folks, modeling for startups is actually the hardest.
Why?
Because there's no data.
How could you forecast with confidence when you don't have historical precedent?
How can you avoid explosion if you're forced to use cascading assumptions?
How do you set drivers without credible benchmarks?
How do you handle huge swings in possible inputs?
Startup modeling broke every rule they knew.
And that's why it felt like such a mystery.
Zooming back to that room of CFOs, I realized: Here we are again.
In a world where technology is completely transforming what's possible every day, we all need to shift into startup CFO mode.
So my recommendation to CFOs forecasting AI spend is the same advice I give to those early-stage modelers:
If you don't have enough data, you need to learn the techniques for making good bets:
Stop requiring your assumptions to be right, and learn to work with assumptions that are reasonable. A good assumption is a factor that is knowable, controllable, estimateable, or even guessable. If more than one knowledgable person would reasonably guess the same number, that's a fine working assumption.
Question your forecasting methodology - Have you exhausted all possible knowable contributing factors? Is there a leading indicator that's more predictable you can control or predict instead?
Adopt a shoot-measure-aim mentality, with a move-forward plan for a range of inputs. Startups budget on a monthly cycle for a reason! Working assumptions and constraints are made to be updated.
Swap from a budget mentality to a ratio measure. If you pair an unruly metric with a constraining one, you make it much more useful (Compute spend / employee spend, total compute & employee spend / bug close time, etc.)
And most importantly:
Stop using your forecasts as a baseline for the future, and start focusing on modeling scenarios.
In a world where there's no data and the future is more unpredictable than ever, you can't judge yourself on your ability to be RIGHT - being RIGHT is an unreasonable goal.
Instead, you want to be useful, even if you're wrong.
Because when it comes to AI, your forecasts are definitely wrong.
We all are.
How will you make decisions anyway?
Stop grading the model on whether the number was right.
Grade it on whether you have a plan for the numbers you get.
We’re all startups again.
Enjoyed reading this article? Subscribe to receive more via email here.
Know a Founder or Entrepreneur who'd love this content? Please share it!
