Ep.012 - GPU Rental Markets: The New Compute Arbitrage

Ep.012 - GPU Rental Markets: The New Compute Arbitrage

This is 3rd in the series of Build vs Buy vs Rent: The AI Infrastructure Decision Tree for Startups. The global GPU rental market grew from 3.2 billion dollars in 2023 to a projected 9.8 billion by 2025. That growth created a new category of infrastructure provider called neoclouds or GPU-as-a-Service. Neo-clouds ( GPU-as-a-Service) compete with AWS, GCP, and Azure on price and specialization. Intelligent Founder AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. CoreWeave is the largest, with trailing revenues around 3.5 billion dollars. Lambda Labs, Crusoe, RunPod, and Spheron cover different segments of the market, from enterprise multi-year contracts to developer-friendly hourly access to sustainability-focused compute. and what they all have in common is that they are cheaper than hyperscalers for GPU-intensive AI workloads. Current pricing for H100-class compute runs between 1.80 and 3.50 dollars per hour on specialist providers, with spot instances as low as 1.20 dollars per hour. AMD MI300X alternatives often come in 30 to 40 percent cheaper than Nvidia H100 for equivalent inference throughput. There are Four pricing models available: * on-demand by the hour, * reserved instances with 30 to 60 percent discounts for committed periods, * spot instances that can be interrupted, and * bare metal for teams at significant scale. The strategy maps directly to workload type. Variable traffic: on-demand. Stable production inference: reserved. Training and batch jobs: spot. High-throughput inference: bare metal. The most commonly missed cost is egress. Moving data out of a cloud environment typically costs 80 to 90 dollars per terabyte. For applications with larger inputs or outputs, egress can add 20 to 40 percent to the apparent cost of GPU rental. and It is almost never included in headline pricing comparisons. SO, the right way to use GPU rental markets is as a bridge. Validate on APIs, build observability into your utilisation patterns, then move to reserved GPU rental once your traffic is predictable enough to commit. That is the staircase: API to reserved rental to owned hardware, moving up each step only when the data justifies it. Listen to the full episode here, in Substack app, or Apple, Spotify / youtube. Thanks for reading / listening Intelligent Founder AI! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.intelligentfounder.ai/subscribe [https://www.intelligentfounder.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

Ep.011 - The Real Cost of Inference APIs: What You Are Actually Paying For

There are now over 69 providers offering LLM inference. Prices range from 14 cents per million input tokens at the cheap end to 180 dollars per million output tokens at the frontier. That is a 1,280-times gap. Most founders are somewhere in the middle without a clear reason to be there. The calculation is somewhat straightforward once you do it. lets take your daily active users, multiply by interactions per user per day, multiply by tokens per interaction - split into input and output, because output tokens are typically two to four times more expensive, and run that number through the per-token prices of three or four providers. The result is often a shock. A product with ten thousand daily active users and five AI interactions each can easily reach twenty thousand pounds per month on a frontier model, and under five hundred pounds per month on a comparable open-weight alternative. Three things determine whether the API route makes sense. Volume: below fifteen thousand pounds per month in API spend, the engineering overhead of self-hosting is not worth it. Predictability: APIs absorb traffic spikes without operational overhead, and that flexibility has real value for early-stage products. Model quality requirements: not every task needs a frontier model. Routing simple queries to smaller, cheaper models can cut inference costs by 40 to 70 percent with no user-visible quality loss. Intelligent Founder AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is how the vendor lock-in risk gets real and underestimated. When you build on a single API provider, you are locked in not just contractually but through accumulated prompt engineering work, evaluation datasets tuned to that model’s behaviour, and observability tooling built around that provider’s response format. The exit cost from a fully-loaded enterprise AI stack has been estimated at between two hundred thousand and one million dollars in re-engineering. The mitigation is simple: use an abstraction layer between your application code and the vendor API. LiteLLM and Portkey both do this. It adds minimal overhead and lets you switch providers with a configuration change. The cheapest route from day one is not the cheapest route at scale. Start tracking your cost per inference from launch. Listen to the full podcast episode for the full breakdown. This episode is second in the series Build vs Buy vs Rent. see you in the next episode. Note : APIs are sometime pronouced APees, apologies for that, I let Elevenlabs run wild. Thanks for listening/reading Intelligent Founder AI! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.intelligentfounder.ai/subscribe [https://www.intelligentfounder.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

Ep.010 — Build vs Buy vs Rent: The AI Infrastructure Decision Tree for Startups

Every AI startup hits the same wall eventually. The product is working, users are growing, and then the infrastructure bill arrives and nothing makes sense anymore. The question is not which model to use or which framework to build on. The question is where your AI actually lives, who owns it, and what happens to your margins as you scale. There are three positions available to you. You can build on hosted inference APIs, paying OpenAI or Anthropic or one of the cheaper alternatives per token. You can rent GPU compute by the hour from neocloud providers like Lambda, CoreWeave, or Crusoe. Or you can buy hardware and operate it yourself. Build, rent, buy. Three positions, very different economics at different scales. Intelligent Founder AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. Most founders treat this as a one-time decision. It is not. It is a continuous optimisation problem. and, the right answer changes as your traffic grows, as GPU prices shift, and as your model strategy matures. H100 cloud rental rates have fallen from eight dollars an hour in early 2023 to around one eighty to three fifty per hour in mid-2026. API prices have fallen too, but unevenly - there is now a 640-times gap between the cheapest viable LLM API and the most expensive frontier option. That spread is enormous, and most founders are not actively managing it. The decision is driven by three variables: your utilization rate, your workload predictability, and your engineering capacity. High sustained utilization, predictable traffic, and a team that can operate infrastructure - when all three are true, owning your compute makes economic sense. When any one is missing, you want flexibility. Here is the number that changes how you think about this. Eighty percent of AI GPU spend is now inference, not training. That means your infrastructure choice is being made primarily for production workloads, not for training runs. And for regulated sectors like aerospace, transport, healthcare, financial services - where your data goes is not a preference. It is a legal requirement. This series runs eight episodes. We cover inference API economics, GPU rental markets, the on-premises case, open source versus proprietary models, AI FinOps, sovereign AI and compliance, edge inference, and the Nvidia compute wars story. Listen to the full episode here, in Substack app, or Apple, Spotify / youtube. I’ll add a companion cost calculator and spreadsheet at intelligentfounder.ai [http://intelligentfounder.ai] soon. Thanks for listening to Intelligent Founder AI! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.intelligentfounder.ai/subscribe [https://www.intelligentfounder.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

Is Your Salary About to Come in AI Tokens? Ep.009 - The Token Economy

Jensen Huang had a week. well, I had a week too, I was traveling so missed entire last week here. Sorry. but I am back now. so lets go back to Jenson who comparatively had more? fun. definitely more interesting. The NVIDIA CEO spent seven days dropping statements at his own GPU Technology Conference, the Morgan Stanley TMT Conference, and finally on the All-In Podcast and by Thursday the internet had a new topic it couldn’t stop arguing about. The clip that went viral? Huang said and I quote. Jensen Huang: “We’re trying to.” “Let me give you the thought experiment: Let’s say you have a software engineer or AI researcher and you pay them $500,000 a year. We do that all the time.” “That $500,000 engineer, at the end of the year, I’m going to ask them, how much did you spend in tokens?” “If that person said, ‘$5,000,’ I will go ape… something else.” “If that $500,000 engineer did not consume at least $250,000 worth of tokens, I’m going to be deeply alarmed. He confirmed NVIDIA is trying to spend $2 billion annually on tokens for its engineering team. He compared engineers who don’t use AI to chip designers still insisting on paper and pencil instead of CAD software. The take was everywhere by Friday. But most of what was written about it, including the Reddit thread with 937 upvotes and 362 comments, only captured half the story. Here’s the full picture. 🎙️ In this podcast episode, we go deeper on everything above, including. * 🔁 The Jevons Paradox explained from scratch — why cheaper tokens always means more spending, not less * 🤖 What an AI agent actually is — and why it consumes 1,000x more tokens than a simple question * ⚠️ Goodhart’s Law in practice — how token burn rate becomes a metric engineers will game * 💼 The 4th pillar of compensation unpacked — what tokens as pay actually means for your financial security * 🌍 The offshoring disruption nobody’s talking about — why flat token costs globally are reshaping hiring maths * 🏢 What SK Telecom did right — and why their model is the one worth copying If you prefer to read? Here’s the breakdown from a 360 degree perspective. What’s a Token, and Why Does It Cost Money? A token is the unit of measurement for AI processing. Every word you type into an AI, every word it writes back, broken down into fragments called tokens. A sentence is roughly 20 tokens. A full document might be several thousand. Every time you run an AI model, tokens are consumed, and tokens cost money. For a simple ChatGPT query: roughly 1,000 tokens. For a research pipeline: 5,000–50,000 tokens. For an AI agent that runs autonomously » searching, coding, testing, iterating, without you pressing a single button, we’re talking hundreds of thousands of tokens per run. A fleet of agents running continuously? Billions of tokens per day. This distinction between “I asked the AI a question” and “the AI is working for me around the clock” is the entire foundation of Huang’s argument. He’s not imagining engineers typing prompts. He’s imagining engineers deploying autonomous AI workforces. Intelligent Founder AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. The Actual Thesis: Tokens as the 4th Pillar of Compensation Before Huang went viral, VC Tomasz Tunguz of Theory Ventures had already been quietly building this framework. His argument? AI inference is becoming the fourth component of engineering compensation, alongside salary, bonus, and equity. His numbers? a $375K engineer with a $100K token budget has a $475K total package. That token budget doesn’t vest or appreciate, but it enables leverage that no previous tool budget could match. Huang scaled this up to a mandate: a $500K engineer should be consuming at least $250K in tokens. Across NVIDIA’s engineering workforce, that’s a $2B annual token spend, which the company confirmed it’s actively pursuing. The framing is deliberately recruiting-adjacent. “Engineers are now asking ‘what’s my token budget?’ when evaluating offers,” Huang said at GTC. Whether or not this is universally true yet, it’s becoming true fast. [ In this newsletter you get sharp, unfiltered short essays; for full‑length, deep‑dive analysis on AI, subscribe to our companion publication, Intelligent Founder AI. ] The Conflict of Interest Is Real (But Incomplete) The Reddit critique was blunt and structurally correct: NVIDIA sells the GPUs that generate the tokens. Every dollar your engineers spend on tokens flows back, eventually, to GPU demand. Mandating token consumption at scale is demand creation by the person selling the supply. The HP printer analogy made the rounds: “HP would be deeply annoyed if its $200 printer didn’t use $600 of ink.” The Oreo CEO comparison: “Oreo cookies are as important as oxygen.” These are crude but fair. But they’re only half the story. The Jevons Paradox » An economic principle from the 19th century, explains what’s actually happening. When coal-burning technology improved and coal became cheaper, total coal consumption exploded, because efficiency unlocked entirely new applications. The same dynamic is at work with AI tokens: costs have dropped 150x since 2021, yet enterprise inference spending grew 320% in the same period. Cheaper tokens unlock agentic use cases that weren’t viable at higher prices. Agentic use cases consume tokens at orders of magnitude greater scale than simple queries. Total demand surges even as unit cost falls. This is the engine behind NVIDIA’s $1 trillion infrastructure forecast through 2027, and its $215.9B in FY2026 revenu, up 65% year on year. Huang is selling his product and accurately describing a structural shift. Both things are true. The Goodhart’s Law Problem When a measure becomes a target, it stops being a good measure. If you tell your engineers to hit $250K in token spend, some will ask “how do I produce the most value?” and some will ask “how do I hit the number?” The second group will run unnecessarily complex pipelines, use expensive frontier models where a cheaper fine-tuned model would do, leave agents running on idle tasks, and avoid caching that would make them more efficient. The technically correct objective is the inverse of what Huang is incentivizing » token minimization per outcome. Good AI-native engineering means squeezing maximum value out of minimum compute through smart model routing, prompt compression, caching, batching. Measuring raw token volume actively penalizes these skills. The metric that actually matters: Token ROI Ratio value created per dollar of inference consumed. A 10:1 ratio ($10 of revenue per $1 of tokens) is the kind of benchmark forward-looking engineering teams are building toward. That’s the measure worth adopting. The Headcount Question Nobody Is Saying Out Loud Here’s what the viral debate mostly avoided. If a token budget approaching a salary starts to become standard, CFO might l eventually ask? at what token-to-headcount ratio does the compute do enough work that we need fewer humans? And They’re already answering that question. Microsoft cut 15,000 jobs last year while committing $80B to AI infrastructure. Crypto.com laid off 12% of staff in March while revenue was growing, citing AI handling high-volume work. Block cut nearly half its workforce. Around 55,000 US tech layoffs in 2025 were directly attributed to AI-driven restructuring. Huang’s own roadmap puts NVIDIA at 75,000 employees working alongside 7.5 million AI agents? a 100:1 ratio. The “token budget as perk” framing is the friendly version of this story. The CFO version is considerably less friendly. What Smart Founders Should Do With This * Track tokens against outcomes, not as a standalone KPI. Build the denominator: what did $100 of tokens produce? A feature, a resolved ticket, a market analysis? The ratio is the signal. The volume is noise. * Treat token budgets in comp negotiations the same way you’d treat unusual equity terms. Does it vest? What happens if you leave? What’s the cash equivalent? A large non-compounding asset can obscure what you’re actually being paid. * The Jevons Paradox is your tailwind if you’re building on inference infrastructure. Costs will keep falling. Agentic deployment will keep expanding. Products that reduce token waste per outcome, or amplify team output per token consumed, are in a structurally strong position for the next three to five years. * The token economy is real. Huang is both selling chips and describing a genuine transition. The job is to understand which is which — and build accordingly. Thanks for reading Intelligent Founder AI! This post is public so feel free to share it. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.intelligentfounder.ai/subscribe [https://www.intelligentfounder.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2]

MWC 2026 Review: What Founders and Builders Need to Know - Ep.008

Mobile World Congress 2026 (Barcelona, 2–5 March) was the telecom industry's biggest annual gathering, attracting ~109,000 attendees for its 20th anniversary edition. The theme was "The IQ Era" intelligence everywhere, from radio towers to robots. Mobile technologies and services generated $7.6 trillion of economic value in 2025 (6.4% of global GDP), projected to reach $11.3 trillion by 2030. But beneath the demos and keynotes, and behind the branding, the real story is an industry sitting at a crossroads: extraordinary technology, no clear path to new revenue. If you’ haven’t read the pre-conference MWC article yet, here is the link - Now post conference, here’s what actually mattered and what it means if you’re building products, not selling phone contracts. ( total 14 points, top 3 elaborated here, and all rest in the podcast. ) 1. AI Is Eating the Network, But Nobody Knows Who Pays! Every major vendor pitched “AI-native” networks. the idea that future mobile infrastructure should be designed around machine learning from the ground up, not bolted on afterwards. Samsung showed multi-agent AI systems that manage networks end-to-end. Qualcomm launched a modem chip built specifically for “agentic AI”, software agents that act autonomously on your behalf. Deutsche Telekom demoed a system that detects, diagnoses, and fixes network problems across the entire stack using multiple AI agents working together. Almost all of this AI is about cutting costs (fixing faults faster, optimizing radio signals, automating tickets) rather than generating new revenue. The ROI gap isn’t about technology/ it’s about business model imagination. Bain & Company’s post-show report warned that the gap between telco leaders and laggards is widening fast, and any operator that can’t quantify AI business value soon needs to rethink its roadmap. If you’re building AI-powered products, maintenance platforms, agentic systems, automation tools, telcos are becoming a buyer. But expect long sales cycles and a focus on opex reduction, not top-line growth bets. The real opening may be in building the tools that help telcos prove ROI, not the AI models themselves. Intelligent Founder AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. 2. NVIDIA Is Becoming the Platform Telcos Build On A coalition of major operators and vendors committed at MWC to build future 6G networks on NVIDIA’s AI-native, open platforms. The AI-RAN Alliance, the industry group pushing GPU-accelerated radio networks hit 132 members and showed 33 demos. Ericsson demonstrated its cloud RAN software running on NVIDIA hardware with T-Mobile, proving that radio network software can be portable across different chip platforms. A separate initiative called AI-WIN (AI-Native Wireless Networks) brings together American companies to create a sovereign, NVIDIA-based AI network stack, a geopolitical as much as a technical play. What does this means? What NVIDIA did to AI is what Android did to phones, as it’s becoming the default development platform. If the mobile industry builds on CUDA (NVIDIA’s software framework), it locks in a dependency that shapes everything downstream: * who supplies hardware, * who writes the software, * who captures the value. For hardware builders and edge compute startups, this is a gravitational shift worth tracking. 3. 6G: Impressive Tech, Zero Paying Customers Multiple vendors showed off 6G technology: terahertz radio transmission, AI-native network architectures, even Li-Fi at 5 Gbps. A coalition of major operators targets commercial 6G from 2029. Ericsson claimed the first pre-standard 6G over-the-air session. The sceptic’s view is that, until someone answers “who pays for this and why?” the whole 6G story stays academic. and that exactly the view of the independent analysts too. Experienced industry are pushing hard against repeating the mistakes of 5G, which over-promised and under-delivered on new revenue. They have argued for evolving from existing 5G infrastructure rather than starting from scratch with a new, expensive core network. One analyst likened AI-RAN with GPUs in base stations to “MEC 2.0”, a reference to multi-access edge computing, which failed not on technology but on economics. and the same risk applies here as well. For founder's and builders? If your product depends on 6G capabilities (ultra-low latency, massive device density), you have at least 3–5 years before real deployment. Build for what 5G and Wi-Fi can do today, not what 6G promises for 2030. Here is everything we ‘ve covered in the podcast:- 🔹 Satellites going mainstream and disrupting telcos » Starlink Mobile, Europe’s satellite split, and the ESA × GSMA €100M convergence fund 🔹 Smart glasses as the most tangible consumer product » Google Android XR, Samsung Galaxy XR, and the emerging app platform 🔹 The API economy » GSMA Open Gateway’s progress, where it works (fraud), and where it doesn’t yet 🔹 The $1 trillion scam epidemic » GSMA’s anti-scam push and the Scam Signal API 🔹 Private 5G moving from pilots to production » $8B market, airport and port deployments, flying 5G 🔹 Tokens as the new invisible traffic » why telcos can’t see or monetise AI workloads 🔹 Physical AI doesn’t need 5G or 6G » the case for connectivity-agnostic autonomous systems 🔹 Digital sovereignty reshaping European procurement » sovereign clouds, EU funding, and new compliance expectations 🔹 Quantum-safe telecoms being built today » quantum key distribution, post-quantum cryptography, and telcos as buyers 🔹 Energy efficiency and battery-free IoT » 30%+ savings in production and energy-harvesting sensors 🔹 The revenue problem nobody solved » flat ARPU, value capture moving up the stack, and the culture gap holding telcos back The bottom line from MWC 2026 is simple: the technology is moving faster than the business models, the culture, and the courage to change. For founders and builders, that means building for what works today. 5G, Wi-Fi, and increasingly satellite, while keeping a close eye on where real funding and procurement shifts are happening. Don't bet your roadmap on telco promises that are still years from delivery. Listen to the full episode for the complete breakdown, and follow Intelligent Founder here and on your podcast platform of choice for upcoming deep dives into the signals that matter most , from sovereign AI infrastructure to the satellite power struggle shaping European connectivity. Intelligent Founder AI is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber. This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit www.intelligentfounder.ai/subscribe [https://www.intelligentfounder.ai/subscribe?utm_medium=podcast&utm_campaign=CTA_2]