From Magic to Harness: Building an AI Agent for a 25-Product SaaS Company

Beskrivelse

What happens when you put an LLM on top of a 25-product SaaS company? Ilya Zayats, CTO at Factorial, shares the story of building Factorial ONE — an AI agent designed to help users work across HR, payroll, time, talent, finance, IT, documents, permissions, and many other product surfaces. The first approach looked simple: let the model query the product directly. But the product graph was too large, the GraphQL schema did not fit in context, and the model struggled with product entities, permissions, and stable answers. The talk follows the evolution from “LLM magic” to a production agent harness: semantic tools, skills, smaller schemas, runtime context, deterministic computation, permissions, and browser-based actions. The main lesson: useful AI agents are not built by giving the model the whole world. They are built by designing the right harness — context, tools, workflows, permissions, and runtime — so the agent can safely understand the product, take actions, and create new work across it. Recorded at AI Builders Barcelona. Speaker: Ilya Zayats, CTO at Factorial Topics: AI agents, Factorial ONE, SaaS, LLMs, context engineering, tool use, skills, GraphQL, product automation, AI platform, agent harness, enterprise AI

MCP Isn’t Always the Answer: 7 Experiments Building AI Agent Interfaces at PostHog

Georgiy Tarasov, AI Product Engineer at PostHog, shares what PostHog learned from seven experiments building interfaces for AI agents — and why MCP isn’t always the right answer. As customers started asking for “all PostHog AI features in my agent,” PostHog had to rethink how its product should work outside the browser: inside Claude Code, Cursor, Codex, CLIs, MCP clients, and local agent workflows. In this talk, Georgiy walks through PostHog’s experiments with handwritten agent tools, traditional MCP, CLI-first interfaces, MCP with a CLI-like shape, code execution, dynamic toolsets, and SQL-based retrieval. He compares the trade-offs across developer experience, context bloat, tool discovery, token efficiency, latency, eval results, and real customer usage. The core lesson: your new user might not be a human in a browser tab — it might be your customer’s AI agent. 👉 Subscribe to Itnig for more conversations about real business, startups and brands. 🎙️ Want to join the Itnig podcast or sponsor one of our episodes? Appear on the podcast: https://tally.so/r/wo1Poe Sponsor the podcast: https://tally.so/r/3EERLN ABOUT ITNIG: 🐦 X - ⁠https://x.com/itnig 💡 LinkedIn -⁠ https://linkedin.com/company/itnig 📸 Instagram - ⁠https://instagram.com/itnig 💌 Newsletter - ⁠https://itnig.net/newsletter/⁠ 🌐 Web -⁠ https://itnig.net/ LISTEN TO OUR PODCAST ON: 🔊 Spotify: ⁠http://bit.ly/itnigspotify⁠ 🎙️ Apple Podcast: ⁠http://bit.ly/itnigapple ⁠00:00:00 Intro & welcome — AI Builders BCN first edition 00:01:04 Georgiy introduces himself & PostHog 00:02:14 The challenge: shipping AI agents on a complex product 00:04:37 First MCP experiments & why they didn't scale 00:05:26 Token optimization problems & early lessons 00:07:08 Rethinking the approach: sandboxing & unified interfaces 00:08:13 Deep dive: how MCP prompts & instructions really work 00:10:26 Optimizing MCP tools for different model providers 00:12:55 Experiment #1 — grouping tools by intent00:14:15 Switching to a resource-based approach 00:18:24 Advantages & limitations of the code execution approach 00:22:30 MCP with native code execution — Georgiy's favourite 00:25:15 How GitHub optimizes their MCP (grouping by intent) 00:25:43 Agent skills: what they are and why they matter 00:26:43 Building skills with regex, access controls & data 00:27:52 Benchmarking: testing with real vs synthetic data 00:29:38 Results & what actually worked in production 00:31:35 Key takeaways & closing thoughts 00:31:43 Q&A 00:36:29 End Recorded at AI Builders Barcelona. Speaker: Georgiy Tarasov, AI Product Engineer at PostHog Topics: MCP, AI agents, Claude Code, Cursor, Codex, CLI, codegen, agent interfaces, developer tools, PostHog, AI engineering

5. maj 202636 min

From Magic to Harness: Building an AI Agent for a 25-Product SaaS Company

Beskrivelse

Kommentarer

Prøv gratis i 7 dage

Alle episoder