Self-Hosting AI: Scaling Is the Real Problem

Self-Hosting AI: Scaling Is the Real Problem

AI is easy to use — but hard to scale. In this episode of Domesticating AI, we’re joined by Daniel Dowler (Red Hat) to break down what actually happens when you move from calling APIs to running AI systems yourself. Recorded on April 21st Most developers interact with AI through APIs — fast, simple, and pay-per-token. But behind the scenes, those systems rely on GPU scheduling, batching, and infrastructure that doesn’t behave like traditional software. We cover: * Why GPU scaling is fundamentally different from CPU scaling * Why tools like vLLM are becoming the default for high-performance inference * How Ray and Kubernetes fit into real-world AI systems * What parallelism (tensor, data, expert) actually means in practice * When self-hosting AI makes sense * When APIs are still the better choice * Claude Opus 4.7 https://www.anthropic.com/news/claude-opus-4-7 [https://www.anthropic.com/news/claude-opus-4-7] * Qwen 3.6 (Alibaba) https://qwen.ai/research [https://qwen.ai/research] * Kimi K2.6 (community discussion) https://www.reddit.com/r/LocalLLaMA/s/kvRWb7uJgM [https://www.reddit.com/r/LocalLLaMA/s/kvRWb7uJgM] * vLLM → https://github.com/vllm-project/vllm [https://github.com/vllm-project/vllm] * Ray → https://github.com/ray-project/ray [https://github.com/ray-project/ray] * Kubernetes → https://kubernetes.io [https://kubernetes.io] * Kueue → https://kueue.sigs.k8s.io [https://kueue.sigs.k8s.io] * LiteLLM → https://github.com/BerriAI/litellm [https://github.com/BerriAI/litellm] * KServe → https://kserve.github.io Daniel Dowler Platform engineer at Red Hat focused on Kubernetes and AI infrastructure. Daniel works on how modern systems support real workloads, including GPU scheduling, distributed inference, and scaling AI in production environments. He recently spoke at Machine Learning Utah on AI infrastructure and clustering. You don’t scale AI with replicas. You scale it by managing scarce compute. Subscribe on Spotify or Apple, and follow us on YouTube. 👉 Keep your AI on a leash. 🧠 News🔗 Tools & Tech Mentioned👤 Guest🎯 Key Takeaway

You’re Using AI Wrong: Build the System, Not Just the Prompt /w Lexi Pasi

Recorded: April 14, 2026 Most people using AI today are still users. They open ChatGPT, call an API, and get an answer. And honestly… it works. But that’s not the same as building with AI. In this episode of Domesticating AI, we break down the difference between AI users and AI practitioners—and why that shift matters if you want reliable systems. We’re joined by Alexandra “Lexi” Pasi, PhD, CEO of Lucidity Sciences, to talk about what it actually means to own the system around AI: * why calling an API is still user behavior * what changes when you build the harness * how agent systems actually fail (loops, cost, drift) * why switching models isn’t a reliability strategy * how to add layers—constraints, validation, and control flow * why engineering discipline matters more with AI, not less If you’ve built your first AI agent, workflow, or coding loop—this is the “now what?” episode. Alexandra Pasi is the CEO of Lucidity Sciences, where she works at the intersection of mathematics, machine learning, and real-world system design. She holds a PhD in Mathematics from Baylor University and specializes in building analytical and algorithmic systems that bring structure to complex, uncertain environments. 🔗 LinkedIn: https://www.linkedin.com/in/alexandrapasi/ [https://www.linkedin.com/in/alexandrapasi/?utm_source=chatgpt.com] 🔗 Lucidity Sciences: https://luciditysciences.com [https://luciditysciences.com] * Google TurboQuant (LLM compression research) https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/ [https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/?utm_source=chatgpt.com] * Anthropic Claude Mythos Preview (security-focused model) https://red.anthropic.com/2026/mythos-preview/ [https://red.anthropic.com/2026/mythos-preview/?utm_source=chatgpt.com] * Project Glasswing (Anthropic security initiative) https://www.anthropic.com/glasswing [https://www.anthropic.com/glasswing?utm_source=chatgpt.com] * Karpathy Autoresearch (self-improving training loop) https://github.com/karpathy/autoresearch [https://github.com/karpathy/autoresearch?utm_source=chatgpt.com] * Kitaru (durable agent execution framework) https://github.com/zenml-io/kitaru [https://github.com/zenml-io/kitaru?utm_source=chatgpt.com] * Subscribe on YouTube * Follow on Spotify & Apple Podcasts * Support the show on Patreon: 👉 https://patreon.com/DomesticatingAIPodcast [https://patreon.com/DomesticatingAIPodcast] Keep your AI on a leash. 🧾 Episode Summary👤 Guest: Alexandra “Lexi” Pasi, PhD🔗 Topics & Links Mentioned🔔 Follow & Support

Hacking AI: Why Most AI Systems Are Insecure by Default

Hosts: Miriah Peterson, Matt Sharp, Chris Brousseau Recorded: April 2026 Status: Released Most AI systems today are designed to be helpful — not secure. In this episode, we break down how AI systems actually get exploited in production: * a real supply chain attack on a widely used AI dependency * prompt injection and why it still works * image-based (multimodal) exploits * tool and agent abuse If you’re building AI — especially at a startup — you are the security team. A widely used AI dependency was compromised via a malicious .pth file: * executes automatically when Python starts * no import required * targets credentials, SSH keys, and environment variables 👉 Just installing the package was enough. This highlights a critical reality: Your AI system is only as secure as your dependencies. * Models cannot distinguish between instructions and data * External content can override system behavior * Still one of the most common AI vulnerabilities 🔗 https://learnprompting.org/docs/prompt_hacking/injection [https://learnprompting.org/docs/prompt_hacking/injection] * Hidden instructions embedded in images * AI interprets images differently than humans * Expands the attack surface significantly 🔗 https://arxiv.org/abs/2306.11698 [https://arxiv.org/abs/2306.11698] * AI systems can take real-world actions via tools * Prompt injection → API calls, data leaks, unintended execution * Agents amplify risk through autonomy and retries If you’re building AI systems today: * separate instructions from data * limit tool permissions * treat outputs as untrusted * validate everything before execution * AI systems have an internet-sized attack surface * Supply chain attacks bypass all AI safeguards * Prompt injection is a fundamental problem * AI doesn’t fail safely — it fails wherever your system is weakest * LiteLLM incident: https://github.com/BerriAI/litellm/issues/24512 [https://github.com/BerriAI/litellm/issues/24512] * Attack breakdown: https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/ [https://futuresearch.ai/blog/litellm-pypi-supply-chain-attack/] * LLM attack techniques: https://llm-attacks.org/ [https://llm-attacks.org/] * OWASP LLM Top 10: https://owasp.org/www-project-top-10-for-large-language-model-applications/ [https://owasp.org/www-project-top-10-for-large-language-model-applications/] * Gandalf challenge: https://gandalf.lakera.ai/ [https://gandalf.lakera.ai/] We’ve launched a Patreon for Domesticating AI 🎉 Get: * early access to episodes * behind-the-scenes content * bloopers and uncut moments 👉 https://patreon.com/DomesticatingAIPodcast [https://patreon.com/DomesticatingAIPodcast] * 🎥 YouTube: https://youtu.be/HTTxE7Y1sko [https://youtu.be/HTTxE7Y1sko] What’s the weirdest way an AI system has broken for you? Keep your AI on a leash.

Coding with AI: Vibe Coding vs Real Engineering (with Tyler Folkman)

AI can write code — but that doesn’t mean you should trust it. In this episode of Domesticating AI, we’re joined by Tyler Folkman (author of The AI Architect) to break down how engineers are actually using AI to build software — and why most people are still just vibe coding. * Vibe coding vs real engineering * Reasoning models vs coding models * How to plan and prompt AI effectively * When to let AI take the wheel (and when not to) * Local vs cloud coding agents * Token costs vs owning hardware * Tyler Folkman — The AI Architect [https://tylerfolkman.substack.com/] * Anthropic https://www.anthropic.com [https://www.anthropic.com] * OpenAI https://openai.com [https://openai.com] * Ollama https://ollama.com [https://ollama.com] * MiniMax-M2.5 https://ollama.com/library/minimax-m2.5 [https://ollama.com/library/minimax-m2.5] * GLM-5 https://ollama.com/library/glm-5 [https://ollama.com/library/glm-5] * AmpCode Chronicle https://ampcode.com/chronicle [https://ampcode.com/chronicle] * Andrej Karpathy on Context Engineering https://x.com/karpathy [https://x.com/karpathy] * “Human in the Loop is Tired” (add link if you have it) Domesticating AI is a bi-weekly podcast about practical AI for developers. We help you brace the feral open-source AI landscape — so you can tame it instead of getting dragged by it. contact@domesticatingai.com Spotify https://open.spotify.com/show/2WsAR4fvcXzp3vVZGVlkE2 [https://open.spotify.com/show/2WsAR4fvcXzp3vVZGVlkE2] Apple Podcasts https://podcasts.apple.com/us/podcast/domesticating-ai/id1873338950 [https://podcasts.apple.com/us/podcast/domesticating-ai/id1873338950] Are you vibe coding — or engineering with AI? Let us know your setup. Keep your AI on a leash. 🧠 What We Cover🔗 Links & ResourcesGuestModels & ToolsArticles / Mentions🎧 About the Podcast📬 Contact🔥 Follow👇 Join the Discussion

Securing Your Homelab: AI Infrastructure, Access Control & Why Docker Isn’t Isolation

Recording Date: February 27, 2026 Hosts: Miriah Peterson, Matt Sharp, Chris Brousseau Running AI locally is easier than ever. Running it securely is another story. In this episode of Domesticating AI, we break down the moment every homelab builder hits: The second you move from one machine to two machines… access becomes your first real engineering problem. We explore the real architecture questions behind self-hosting AI: * Why a dedicated machine isn’t a sandbox * Why Docker alone isn’t isolation * How homelabs evolve from Plex servers to AI infrastructure * The blast radius problem with local agents * Why networking and access control matter more than model size We also discuss the surge in local AI hardware demand and the risks of running powerful agents on machines with unrestricted access. Whether you're running OpenClaw, Ollama, a NAS, Postgres, or a home automation stack, the same rule applies: Infrastructure without containment is just risk waiting to happen. High-memory Mac Minis are seeing long shipping delays as developers rush to build local AI systems. https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-fueled-ordering-frenzy-creates-apple-mac-shortage-delivery-for-high-unified-memory-units-now-ranges-from-6-days-to-6-weeks [https://www.tomshardware.com/tech-industry/artificial-intelligence/openclaw-fueled-ordering-frenzy-creates-apple-mac-shortage-delivery-for-high-unified-memory-units-now-ranges-from-6-days-to-6-weeks?utm_source=chatgpt.com] Marketplace plugins and execution boundaries are becoming a growing security concern in agent systems. https://www.linkedin.com/posts/matthewsharp_i-use-to-do-nothing-but-post-about-clean-activity-7432832983339999232-iR04 [https://www.linkedin.com/posts/matthewsharp_i-use-to-do-nothing-but-post-about-clean-activity-7432832983339999232-iR04] Overview of risks around agent plugin ecosystems and execution boundaries. https://conscia.com/blog/the-openclaw-security-crisis/ [https://conscia.com/blog/the-openclaw-security-crisis/?utm_source=chatgpt.com] Private mesh networking used to securely access homelabs. https://tailscale.com [https://tailscale.com] Local AI coding agent framework. https://openclaw.ai [https://openclaw.ai] Local LLM runtime used for running models on personal machines. https://ollama.com [https://ollama.com] * Why people actually build homelabs * Plex, NAS, and home automation as infrastructure entry points * AI workloads vs dev workloads * Why long-running services shouldn’t live on your laptop * Networking architecture for homelabs * RBAC-style access control between machines * Secrets management mistakes developers make * Containment and blast-radius thinking for AI agents * Tailscale and private mesh networking Each host answers: If I had $0 * What I would run * What I would avoid If I had $1K * What machine I’d buy * How I’d isolate workloads If I had $5K * How I’d segment infrastructure * What monitoring I’d deploy * What I would never expose to the internet Staff Data Engineer, content creator, and founder of SoyPete Tech. Miriah focuses on practical AI systems, Go infrastructure, and self-hosted AI engineering. She is also a Google Developer Expert in Go and organizer of Go West Conf. https://soypete.tech AI engineer and co-author of LLMs in Production. Matt focuses on applied AI systems, local model infrastructure, and developer-focused AI tooling. Software engineer and AI practitioner focused on practical applications of machine learning and developer infrastructure. Domesticating AI is supported by the SoyPete Tech community. If you enjoy the show: * Subscribe on YouTube * Follow on Spotify * Join the Discord community * Share the episode with another engineer building with AI More content and tutorials: https://soypetech.substack.com 📰 News DiscussedMac Mini Shortages from Local AI DemandOpenClaw Security DiscussionOpenClaw Security Concerns (Referenced)🧰 Tools & Technologies MentionedTailscaleOpenClawOllama🏗 Topics Covered⚡ Lightning Round🎙 HostsMiriah PetersonMatt SharpChris Brousseau🤝 Sponsors