AI + a16z

Labelbox [https://labelbox.com/] CEO Manu Sharma joins a16z Infra partner Matt Bornstein to explore the evolution of data labeling and evaluation in AI — from early supervised learning to today’s sophisticated reinforcement learning loops. Manu recounts Labelbox’s origins in computer vision, and then how the shift to foundation models and generative AI changed the game. The value moved from pre-training to post-training and, today, models are trained not just to answer questions, but to assess the quality of their own responses. Labelbox has responded by building a global network of “aligners” — top professionals from fields like  coding, healthcare, and customer service, who label and evaluate data used to fine-tune AI systems. The conversation also touches on Meta’s acquisition of Scale AI, underscoring how critical data and talent have become in the AGI race.  Here's a sample of Manu explaining how Labelbox was able to transition from one era of AI to another: It took us some time to really understand like that the world is shifting from building AI models to renting AI intelligence. A vast number of enterprises around the world are no longer building their own models; they're actually renting base intelligence and adding on top of it to make that work for their company. And that was a very big shift.  But then the even bigger opportunity was the hyperscalers and the AI labs that are spending billions of dollars of capital developing these models and data sets. We really ought to go and figure out and innovate for them. For us, it was a big shift from the DNA perspective because Labelbox was built with a hardcore software-tools mindset. Our go-to market, engineering, and product and design teams operated like software companies.  But I think the hardest part for many of us, at that time, was to just make the decision that we're going just go try it and do it. And nothing is better than that: "Let's just go build an MVP and see what happens." Follow everyone on X: Manu Sharma [https://x.com/manuaero] Matt Bornstein [https://x.com/BornsteinMatt] Check out everything a16z is doing with artificial intelligence here [https://a16z.com/ai/], including articles, projects, and more podcasts.

In this episode of AI + a16z, dbt Labs [https://www.getdbt.com/] founder and CEO Tristan Handy sits down with a16z's Jennifer Li and Matt Bornstein to explore the next chapter of data engineering — from the rise (and plateau) [https://roundup.getdbt.com/p/is-the-modern-data-stack-still-a] of the modern data stack [https://a16z.com/emerging-architectures-for-modern-data-infrastructure/] to the growing role of AI in analytics and data engineering. As they sum up the impact of AI on data workflows: The interesting question here is human-in-the-loop versus human-not-in-the-loop. AI isn’t about replacing analysts — it’s about enabling self-service across the company. But without a human to verify the result, that’s a very scary thing. Among other specific topics, they also discuss how automation and tooling like SQL compilers are reshaping how engineers work with data; dbt's new Fusion Engine [https://www.getdbt.com/product/fusion] and what it means for developer workflows; and what to make of the spate of recent data-industry acquisitions and ambitious product launches. Follow everyone on X: Tristan Handy [https://x.com/jthandy] Jennifer Li [https://x.com/JenniferHli] Matt Bornstein [https://x.com/BornsteinMatt] Check out everything a16z is doing with artificial intelligence here [https://a16z.com/ai/], including articles, projects, and more podcasts.

Arcjet [https://arcjet.com/] CEO David Mytton sits down with a16z partner Joel de la Garza to discuss the increasing complexity of managing who can access websites, and other web apps, and what they can do there. A primary challenge is determining whether automated traffic is coming from bad actors and troublesome bots, or perhaps AI agents trying to buy a product on behalf of a real customer.Joel and David dive into the challenge of analyzing every request without adding latency, and how faster inference at the edge opens up new possibilities for fraud prevention, content filtering, and even ad tech.Topics include: * Why traditional threat analysis won’t work for the AI-powered web * The need for full-context security checks * How to perform sub-second, cost-effective inference * The wide range of potential actors and actions behind any given visit As David puts it, lower inference costs are key to letting apps act on the full context window — everything you know about the user, the session, and your application. Follow everyone on social media: David Mytton [https://x.com/davidmytton] Joel de la Garza [https://www.linkedin.com/in/3448827723723234/] Check out everything a16z is doing with artificial intelligence here [https://a16z.com/ai/], including articles, projects, and more podcasts.

Instabase [https://instabase.com/] founder and CEO Anant Bhardwaj joins a16z Infra partner Guido Appenzeller to discuss the revolutionary impact of LLMs on analyzing unstructured data and documents (like letting banks verify identity and approve loans via WhatsApp) and shares his vision for how AI agents could take things even further (by automating actions based on those documents). In more detail, they discuss: * Why legacy robotic process automation (RPA) struggles with unstructured inputs. * How Instabase developed layout-aware models to extract insights from PDFs and complex documents. * Why predictability, not perfection, is the key metric for generative AI in the enterprise. * The growing role of AI agents at compile time (not runtime). * A vision for decentralized, federated AI systems that scale automation across complex workflows. Follow everyone on X: Anant Bhardwaj [https://x.com/anantpb] Guido Appenzeller [https://x.com/appenz] Check out everything a16z is doing with artificial intelligence here [https://a16z.com/ai/], including articles, projects, and more podcasts.

LMArena [https://lmarena.ai/] cofounders Anastasios N. Angelopoulos, Wei-Lin Chiang, and Ion Stoica sit down with a16z general partner Anjney Midha to talk about the future of AI evaluation. As benchmarks struggle to keep up with the pace of real-world deployment, LMArena is reframing the problem: what if the best way to test AI models is to put them in front of millions of users and let them vote? The team discusses how Arena evolved from a research side project into a key part of the AI stack, why fresh and subjective data is crucial for reliability, and what it means to build a CI/CD pipeline for large models. They also explore: * Why expert-only benchmarks are no longer enough. * How user preferences reveal model capabilities — and their limits. * What it takes to build personalized leaderboards and evaluation SDKs. * Why real-time testing is foundational for mission-critical AI. Follow everyone on X: Anastasios N. Angelopoulos [https://x.com/ml_angelopoulos] Wei-Lin Chiang [https://x.com/infwinston] Ion Stoica [https://x.com/istoica05] Anjney Midha [https://x.com/AnjneyMidha] TIMESTAMPS 0:04 -  LLM evaluation: From consumer chatbots to mission-critical systems 6:04 -  Style and substance: Crowdsourcing expertise 18:51 -  Building immunity to overfitting and gaming the system 29:49 -  The roots of LMArena 41:29 -   Proving the value of academic AI research 48:28 -  Scaling LMArena and starting a company 59:59 -  Benchmarks, evaluations, and the value of ranking LLMs 1:12:13 -  The challenges of measuring AI reliability 1:17:57 -  Expanding beyond binary rankings as models evolve 1:28:07 -  A leaderboard for each prompt 1:31:28 -  The LMArena roadmap 1:34:29 -  The importance of open source and openness 1:43:10 -  Adapting to agents (and other AI evolutions) Check out everything a16z is doing with artificial intelligence here [https://a16z.com/ai/], including articles, projects, and more podcasts.