Real-world problems with STT | Klemen Simonic (Soniox) & Kwindla Kramer (Daily)

Scaling STT systems | Maxime Gaudin (CTO at Gladia)

In the Future of Voice AI series of interviews, I ask three questions to my guests: - What problems do you currently see in Enterprise Voice AI? - How does your company solve these problems? - What solutions do you envision in the next 5 years? This episode’s guest is Maxime Gaudin [https://www.linkedin.com/in/maxime-gaudin/], CTO at Gladia [https://www.gladia.io/]. Former Co-Founder & CTO at Matcha and CTO at MadKudu through its private equity acquisition. Among the first employees at Malt, where he helped scale the company from 6 to 250 people and over €12M in monthly transaction volume. He earned his Master's degree in Computer Science from INSA Lyon and Polytechnique Montréal, Canada. Throughout his career, he has built and scaled products across B2B SaaS, data intelligence, and speech AI, from early-stage founding to leading engineering organizations through hypergrowth and acquisitions. Gladia [http://www.gladia.io] was founded in 2022 by Jean-Louis Queguiner and Jonathan Soto with a mission to help companies leverage cutting-edge AI and retrieve actionable insights from audio data. Its API supports advanced speech recognition features in over 100 languages, with exceptional accuracy and asynchronous and real-time transcription. Recap Video Thanks for reading Voice AI Newsletter! Subscribe for free to receive weekly updates. Takeaways * Winning isn’t just about model quality, it is surviving brutal tradeoffs between latency, cost, and scale. * The real challenge is not training one great model, it is running it cheap enough to meet market pricing without breaking performance. * STT is getting commoditized so fast that providers have to chase better accuracy while selling at margins that keep shrinking. * Big models don’t matter if they are too expensive to run at scale. * Real-time voice AI lives or dies under a hard latency budget, and staying under 300 milliseconds leaves little room for mistakes. * The industry obsession with one model that does everything may be the wrong path if smaller specialist models can outperform it in the moments that matter. * Every model upgrade is risky because improving one language or task can make another one worse. * Testing speech systems is harder than people admit because teams know something broke, but don’t know what. * General transcription errors can be patched by an LLM, but once a name, phone number, email, or address is lost, it is gone. * The next edge in voice AI may come from tiny models trained for high-value details like PII, not from one giant model trying to handle everything. * Email addresses sound simple until real accents, pauses, corrections, and spelling cues expose how messy spoken language really is. * The companies that win enterprise voice AI will be the ones that orchestrate many narrow models well, not the ones chasing a single universal model. * Infrastructure strategy is becoming a product decision because legal rules, traffic spikes, and customer use cases all change what “best” deployment looks like. * Cloud scaling breaks in real-time spikes, like emergency calls. * Using managed infra and large DevOps teams at once wastes money. * Customers want one vendor for everything, even if quality drops. * The market will reward depth over breadth if a vendor can become truly exceptional in one painful, business-critical part of the voice stack. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit voice-ai-newsletter.krisp.ai [https://voice-ai-newsletter.krisp.ai?utm_medium=podcast&utm_campaign=CTA_1]

Promptable Speech Language Models | Dylan Fox (Founder & CEO at AssemblyAI)

In the Future of Voice AI series of interviews, I ask three questions to my guests: - What problems do you currently see in Enterprise Voice AI? - How does your company solve these problems? - What solutions do you envision in the next 5 years? This episode’s guest is Dylan Fox [https://www.linkedin.com/in/dylanbfox/], Founder & CEO at AssemblyAI [https://www.assemblyai.com/]. Dylan started AssemblyAI [https://www.assemblyai.com/] in 2017 inspired by the potential of new voice-powered products like the Amazon Alexa, as well as his experience working as a research engineer at Cisco on new AI products and features. He saw an opportunity to use new AI technology to make fundamental improvements in the way that computers can understand and extract value from voice data. AssemblyAI started in Y Combinator and has now grown into a Series C company with over $115 million in funding from notable investors like Accel, Insight Partners, and Smith Point Capital. Dylan lives in Brooklyn, NY. AssemblyAI [https://www.assemblyai.com/] builds speech language models that serve as the foundational voice AI infrastructure for next-generation voice applications. Their models deliver industry-leading speech-to-text accuracy with superhuman speech understanding capabilities including speaker detection, summarization, PII redaction, and an LLM gateway — giving developers everything they need to build sophisticated voice AI products.Universal-3 Pro, the first speech language model optimized specifically for voice AI, goes further with advanced prompting capabilities that let developers customize model behavior for their exact use case. With both async and real-time streaming support, AssemblyAI integrates directly into voice agents, AI assistants, medical scribes, real-time call analysis systems, and more. Tens of thousands of developers rely on AssemblyAI's models to power voice AI applications used by millions of end users every day. Recap Video Thanks for reading Voice AI Newsletter! Subscribe for free to receive weekly updates. Takeaways * Real-time is the new growth engine - the last ~18–20 months crossed a reliability threshold where voice use cases actually work. * The real barrier in real-time STT is not model quality, it’s running low-latency systems at massive scale without breaking. * Voice AI is quietly expanding beyond agents into robotics, consumer hardware, ambient listening, and medical scribes, which widens the market fast. * Streaming models will always be disadvantaged on “look-ahead,” so the core problem is making good calls with incomplete future context. * The old quality-vs-speed tradeoff is shrinking because hardware and model optimizations are closing the gap between streaming and batch. * The ‘98% accuracy’ claims are meaningless because benchmarks reward clean audio, not real phone chaos and edge cases. * The industry needs hard voice evals where models look bad on purpose (WER ~50%) because that’s closer to real conditions. * The bottleneck is not model quality, it’s operating low-latency voice systems at insane scale without falling over. * Pricing is used as a growth lever: $0.21 per hour, prorated by the second, with automatic volume discounts. * The “no reservations, no concurrency limits” promise is really a bet on infra superiority, not just model quality. * Dylan’s open-source take is blunt: managing your own AI infra is a tax that slows shipping and kills competitiveness. * Specialization beats multimodal generalists for reliability: a model trained 100% on STT tasks is less likely to go off the rails. * Massive training data scale, not a sudden architecture breakthrough, is the main reason accuracy jumped in the last 2–3 years. * Infrastructure is becoming the hidden moat: unlimited rate limits and no concurrency negotiations remove a major bottleneck for teams shipping voice products. * Real-world performance can move business metrics, like a 15–20% lift in voice agent booking conversions from better STT. * Dylan’s adoption forecast is aggressive: we are at the start of a 100x curve, which means today’s usage is the floor, not the peak. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit voice-ai-newsletter.krisp.ai [https://voice-ai-newsletter.krisp.ai?utm_medium=podcast&utm_campaign=CTA_1]

Real-world problems with STT | Klemen Simonic (Soniox) & Kwindla Kramer (Daily)

In the Future of Voice AI series of interviews, I ask three questions to my guests: - What problems do you currently see in Enterprise Voice AI? - How does your company solve these problems? - What solutions do you envision in the next 5 years? This episode’s guests are Klemen Simonic [https://www.linkedin.com/in/klemensimonic/], Co-Founder & CEO at Soniox [https://soniox.com/], and Kwindla Hultman Kramer [https://x.com/kwindla], Co-Founder & CEO at Daily [https://www.daily.co/]. Klemen Simonic is the CEO and Co-Founder of Soniox, where he leads the development of advanced voice AI models built for real-world performance. He brings over 16 years of experience across industry and academia, with a deep focus on artificial intelligence. He has worked on cutting-edge AI systems at Facebook, Google, Stanford University, and the University of Ljubljana. Klemen has been developing AI technologies since his undergraduate years, spanning speech, language, and large-scale knowledge systems. Kwin is CEO and co-founder of Daily, a developer platform for real-time audio, video, and AI. He has been interested in large-scale networked systems and real-time video since his graduate student days at the MIT Media Lab. Before Daily, Kwin helped to found Oblong Industries, which built an operating system for spatial, multi-user, multi-screen, multi-device computing. Recap Video Thanks for reading Voice AI Newsletter! Subscribe for free to receive weekly updates. Takeaways * Voice AI adoption is slow because real-time transcription still breaks on the most basic parts of a customer call. * Real growth is happening quietly inside call centers, but teams won’t scale until transcription stops causing cascading errors. * Even the top models fail on emails, addresses, and alphanumerics, which are the single points of failure in most B2B workflows. * Consumer-grade demos hide the reality that long, multi-turn conversations still fall apart without rigorous context control. * POC to production fails not because of LLMs, but because engineering teams underestimate context management. * A universal multilingual model can outperform single-language models by transferring entity knowledge across languages. * Mixed-language conversations are the norm worldwide, and current systems break the moment a user switches language. * Latency, accuracy, and cost must be solved at the same time; optimizing only one kills the use case. * Feeding both sides of the conversation into STT gives models more context and improves accuracy. * Domain-specific accuracy matters far more than general accuracy, and most models still fail in specialized environments. * Industry “context boosting” tricks are hacks that break at scale; native learned context inside STT is the only path forward. * Punctuation and intonation directly shape LLM reasoning, and stripping them for speed creates silent failure modes. * Voice AI is shifting from speech-to-text to full speech understanding, and models that don’t evolve won’t survive. * The future points toward fused audio plus LLM architectures that remove the brittle STT handoff entirely. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit voice-ai-newsletter.krisp.ai [https://voice-ai-newsletter.krisp.ai?utm_medium=podcast&utm_campaign=CTA_1]

Accent AI’s 85+ NPS Impact in India | James Bednar and Biju Pillai (TTEC)

In this special edition of the Future of Voice AI series of interviews, we're joined by industry vets to unpack: - How clarity became a measurable KPI for CX quality and trust - How TTEC identified and solved global voice challenges across regions - Real results: customer satisfaction, agent confidence, cost efficiency improvements and more This episode’s guests are TTEC’s James Bednar, VP of Innovation and Product, and Biju Pillai, VP of India Operations. As voice remains the most human and high-stakes channel, global contact centers like TTEC face a growing challenge: how to deliver effortless understanding across accents, environments, and expectations. In this live session, TTEC leaders and Krisp’s CEO unpack the business case for clarity — sharing how they transformed challenges into measurable wins, including how they turned fragmented communication into a unified standard across global operations. You’ll hear what worked, what didn’t, and how AI-driven voice clarity has become a core pillar of TTEC’s customer and agent experience strategy. Thanks for reading Voice AI Newsletter! Subscribe for free to receive weekly updates. Takeaways 1. Clarity drives measurable ROI. * After Krisp deployment, noise complaints dropped 76%, sales conversions rose 26%, and CSAT improved 8%. * These are not pilot numbers, they came from sustained production environments across thousands of agents. 2. Accent conversion unlocks new talent pools. * By eliminating accent barriers in real time, TTEC could hire for skill, not sound. “We don’t want to hire the right accent, we want to hire the right talent,” James said. * This reduced reliance on costly and inconsistent “voice coaching” programs, creating what Pillai called an “always-on coach.” 3. 80+ NPS from offshore delivery proves the point. * An India-based program reached 80+ NPS, with language-barrier reports cut in half (2.6% → 1.2%) and experience scores rising from 90.5% to 95.5%. * Each new Accent Conversion model release (v3.5 → v3.7) corresponded to higher NPS, peaking at 85 in September 2025. 4. Cost efficiency without quality compromise. * Offshore voice delivery using Krisp achieved ~70% cost savings versus onshore U.S. teams. “Clients that once said India isn’t where you go for voice are rethinking that.” 5. Agent wellbeing and empathy improved. * Agents reported lower fatigue, faster understanding, and higher confidence. Biju noted “calls now flow better—agents no longer overcompensate for accent or tone.” * That confidence translated into trust and empathy, making every conversation feel more human. 6. Next frontier: real-time translation and pacing intelligence. * With accent conversion now near full maturity, Krisp is launching Accent Conversion v4.0, tackling pacing and accent leakage. * Inbound accent conversion and real-time translation will soon close the loop to help both agents and customers understand each other. This isn’t just a story about cleaner audio. It’s about turning clarity into confidence, confidence into empathy, and empathy into measurable ROI. As James put it: “These use cases just work. They deliver what’s expected, with almost no effort to deploy.” This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit voice-ai-newsletter.krisp.ai [https://voice-ai-newsletter.krisp.ai?utm_medium=podcast&utm_campaign=CTA_1]

Voice AI with 100% function-calling accuracy | Will Bodewes (CEO at Phonely.ai)

In the Future of Voice AI series of interviews, I ask three questions to my guests: - What problems do you currently see in Enterprise Voice AI? - How does your company solve these problems? - What solutions do you envision in the next 5 years? This episode’s guest is Will Bodewes [https://www.linkedin.com/in/william-bodewes/], CEO at Phonely.ai [https://www.phonely.ai/]. Will Bodewes is the Co-founder and CEO of Phonely.ai, a Y Combinator–backed startup building conversational phone support powered by AI. A lifelong competitor and creator, he earned a mechanical engineering degree from UNH and launched his first company, Spoke Sound, soon after. Following AI research and travels across Africa, Asia, and the Pacific, Will combined his technical background and curiosity to take on one of tech’s toughest challenges: making AI sound human. Phonely provides AI-powered phone support agents for industries requiring fast, reliable, and human-like AI interactions. Its AI solutions reduce wait times, improve customer experiences, and enable seamless automated conversations. Recap Video Thanks for reading Voice AI Newsletter! Subscribe for free to receive weekly updates. Takeaways * Voice AI jumped from niche to movement in two years, with young builders driving it. * Reliability at scale beats clever prompts; buyers want systems that just work. * Time-to-value is the moat; months of coding kills deals. * Every AI agent succeeds only if it knows what to say, what to know, and what to do: conversation, context, and action. * Integrations are the choke point; the hard work is plumbing messy CRMs and legacy tools. * Training BPO teams to build on the platform scales better than flying in engineers. * LLMs are the latency bottleneck, so faster tokens = more human conversations. * Groq partnership delivered lower latency and beat big names on some Phonely benchmarks. * “Did the caller detect it wasn’t human?” is a better quality metric than WER. * Phonely claims 100% function-calling accuracy in production, which is what buyers actually feel. * Low ASR confidence should trigger human-like behavior (ask to spell names), not clunky links. * Capturing names, numbers, and addresses is the last-mile blocker; fix this or nothing else matters. * Cascading still wins for business logic; speech-to-speech isn’t reliably deployed in production. * Best near-term wins: customer support with tight FAQs, lead qual, and appointment setting. * Defined outcomes plus A/B testing lets agents match call-center KPIs at 50–70% lower cost. * Enterprise rollout will be gradual (2–3 years) until hallucination fear fades. * The next unlock is LLMs that talk like people while staying fast and precise. * Expect convergence where “voice-to-voice” and cascading blur, but LLMs keep the reasoning core. This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit voice-ai-newsletter.krisp.ai [https://voice-ai-newsletter.krisp.ai?utm_medium=podcast&utm_campaign=CTA_1]