Brain Surgery for LLMs: Scaling Transformers with Embedding Modules

How AI Psychosis Destroys Infrastructure

The podcast highlights a growing concern regarding the tech industry’s obsessive reliance on artificial intelligence to manage software development. The author argues that many companies are suffering from a psychosis where they prioritize rapid recovery from errors over building inherently stable systems. By relying on automated agents to fix bugs, organizations risk ignoring underlying architectural decay and long-term structural integrity. This shift mirrors past infrastructure mistakes, where local metrics like test coverage appeared positive while global risks increased. Ultimately, the source warns that over-automation can create a catastrophe machine that functions beyond human comprehension. Decisions are being made so quickly that the semantic understanding of software is being lost to speed.

Google Gemini 3 Deep Think: Advancing Science and Engineering Reasoning

This discussion revolves around the release of Gemini 3 Deep Think, highlighting its record-breaking performance on the ARC-AGI-2 benchmark. Users compare its reasoning capabilities to rivals like Claude 4.6 and GPT-5.2, debating whether these high scores represent true intelligence or mere benchmark optimization. While some praise its long context window and visual reasoning for complex coding and research, others criticize its tendency to hallucinate and ignore instructions. The conversation also explores broader implications, such as the path toward Artificial General Intelligence (AGI) and the shifting definition of human-level reasoning. Additionally, contributors discuss the rapid pace of model releases and the potential for AI to automate professional labor.

Vibe Citing: The Hallucination Crisis at NeurIPS 2025

Recent investigations by GPTZero uncovered over 100 fabricated citations in research papers accepted for the NeurIPS 2025 conference. These "hallucinations," or vibe citations, often include fake author names like "John Doe" and non-existent paper titles that mimic legitimate academic formatting. This discovery highlights a growing reproducibility crisis fueled by a massive surge in AI-assisted submissions that has overwhelmed the peer review pipeline. While some scholars view these errors as minor technical glitches, others argue they signal academic misconduct and a fundamental breakdown in research integrity. Experts suggest that as volume increases, institutions must adopt automated verification tools to distinguish between human error and generative "slop." Ultimately, the presence of these fabrications forces a reckoning regarding the incentives for publication and the reliability of modern scientific discourse.

Brain Surgery for LLMs: Scaling Transformers with Embedding Modules

The provided research introduces STEM (Scaling Transformers with Embedding Modules), a novel architecture designed to enhance the efficiency and knowledge capacity of large language models. By replacing the traditional FFN up-projection with a token-indexed embedding lookup, the system decouples a model's total parameter count from its per-token computational cost. This static sparsity approach eliminates the need for complex runtime routing, allowing for CPU offloading and reducing inter-node communication overhead. Experiments at various scales demonstrate that STEM improves accuracy on knowledge-intensive benchmarks and strengthens performance in long-context reasoning. Furthermore, the architecture offers unique interpretability, enabling direct knowledge editing and injection by simply modifying specific embedding vectors. Ultimately, STEM provides a stable, scalable method for increasing parametric memory while maintaining high efficiency during both training and inference.

Open Responses: An Interoperable LLM Interface Specification

Open Responses is a community-governed, vendor-neutral specification designed to standardize how developers interact with large language models. By providing a unified schema and client library, it allows applications to remain interoperable across different providers like OpenAI, Anthropic, and Google. The protocol is built around an agentic loop where the model can reason, invoke tools, and manage complex workflows through a system of polymorphic items. Technical oversight is managed by a Technical Steering Committee to ensure the project remains open and competitive without single-vendor control. This architecture improves efficiency and performance by preserving reasoning states and utilizing semantic streaming events. Ultimately, the framework aims to simplify multimodal integration while offering a stable foundation for the future of AI development.