Ep5: Multimodal AI Agents: Benchmarking, Adapting, and Adversarial attacks

Ep5: Multimodal AI Agents: Benchmarking, Adapting, and Adversarial attacks

In this episode, we dive into the multimodal AI agents, starting with the recent release of runner H [https://x.com/hcompany_ai/status/1858907025436205278] and diving into groundbreaking research, including: 04:15 VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks by [https://arxiv.org/abs/2401.13649]Jing Yu Koh et. al [https://arxiv.org/search/cs?searchtype=author&query=Koh,+J+Y] 19:18 AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations by [https://arxiv.org/abs/2411.13451]Gaurav Verma et. al. [https://arxiv.org/search/cs?searchtype=author&query=Verma,+G] 32:32 Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast by Xiangming Gu et. al. [https://arxiv.org/abs/2402.08567]

Ep4: AI Agents in Creativity and Design: Minecraft as a 3D Playground, Creating Dream Spaces and 3D Modeling

In this episode, we dive into the groundbreaking world of AI agents transforming creativity and design. We start with navigating 3D environments in Minecraft [https://venturebeat.com/games/airis-is-a-learning-ai-teaching-itself-how-to-play-minecraft/], setting the stage for more complex real-world tasks. Then, we explore how AI agents are revolutionizing 3D modeling in Blender, bringing intricate designs to life. Finally, we delve into the fascinating applications in interior design, where spatial reasoning is used to create dream spaces. Subscribe and tune in to discover new agentic applications every week. Papers covered: SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code [https://arxiv.org/abs/2403.01248] by Ziniu Hu et. al. I-Design: Personalized LLM Interior Designer [https://arxiv.org/abs/2404.02838] by Ata Çelen et. al

Ep3: What’s new in AI agents for browser & edge

In this episode, we explore various paradigms enabling AI agents to take actions on the web and on the edge. We delve into the multi-modal method used by Magentic-One [https://arxiv.org/html/2411.04468v1], recently released by Microsoft, and the reinforcement learning-based navigation method Agent Q [https://arxiv.org/abs/2012.03912] developed by MultiOn. Additionally, we discuss TinyAgent [https://arxiv.org/abs/2409.00608] from UC Berkeley, which allows agents to operate on edge devices and assist users with daily tasks.

Ep2: How AI Agents Are Shaping Hiring, Healthcare and Knowledge Work

In this episode, we dive into LinkedIn’s use of AI agents for hiring, Oracle’s clinical AI agent, and review three papers on AI agents in knowledge work, including applications in machine learning and software engineering. Papers covered: * (03:56 - 07:07) WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Tasks [https://arxiv.org/abs/2407.05291] by Léo Boisvert et al. [https://arxiv.org/search/cs?searchtype=author&query=Boisvert,+L] * (07:08 - 10:12) SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning [https://arxiv.org/abs/2410.17238] by Yizhou Chi et al. [https://arxiv.org/search/cs?searchtype=author&query=Chi,+Y] * (10:13 - 19:30) MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework [https://arxiv.org/abs/2308.00352] by Sirui Hong et al. [https://arxiv.org/search/cs?searchtype=author&query=Hong,+S]

Ep1 - Claude3.5, AgentForce, Copilot Studio, AgentC, Mixture-of-Agents, Agent-as-a-Judge, On the limits of agency

Applications covered: (0:00 - 5:15) Claude 3.5 - Anthropic AgentForce - Salesforce Copilot Studio - Microsoft AgentC - Celonis Papers covered: (5:16 - 19:28) Mixture-of-Agents Enhances Large Language Model Capabilities [https://arxiv.org/abs/2406.04692] by Junlin Wang [https://arxiv.org/search/cs?searchtype=author&query=Wang,+J] et al. (19:29 - 32:10) Agent-as-a-Judge: Evaluate Agents with Agents [https://arxiv.org/abs/2410.10934] by Mingchen Zhuge [https://arxiv.org/search/cs?searchtype=author&query=Zhuge,+M] et al. (32:11 - 42:44) On the limits of agency in agent-based models [https://arxiv.org/abs/2409.10568] by Ayush Chopra [https://arxiv.org/search/cs?searchtype=author&query=Chopra,+A] et al.