Self-Distilled Agentic Reinforcement Learning

From Augmentation to Reconstruction: Guiding the AI Disruption to the Good Place

This paper explores the evolution of artificial intelligence through a three-stage framework of augmentation, automation, and reconstruction. The authors argue that while AI currently improves individual tasks, the most profound economic disruption will only occur when workflows and markets are entirely redesigned around machine capabilities. True transformation is currently stalled by legacy human-centric infrastructures and a lack of trust in autonomous delegation. To realize significant productivity gains, organizations must move beyond local optimizations and invest in machine-legible data and interoperable interfaces. Ultimately, the text emphasizes that leaders must actively steer technological development toward open, ethical systems to ensure AI delivers broad societal benefits.

Self-Distilled Agentic Reinforcement Learning

The research paper introduces SDAR (Self-Distilled Agentic Reinforcement Learning), a new framework designed to improve the training of large language model agents in complex, multi-turn environments. While standard reinforcement learning excels at high-level task goals, it often lacks the precise, token-level guidance needed for long interactions. To solve this, the authors identify critical flaws in current distillation methods, such as multi-turn instability and the unreliability of teacher models when using specialized context. SDAR addresses these issues by using a gated auxiliary objective that selectively applies teacher feedback, prioritizing helpful endorsements while minimizing the impact of incorrect rejections. This adaptive approach allows the agent to learn from individual tokens at its own pace, resulting in significant performance gains on benchmarks like ALFWorld and WebShop. Ultimately, the method offers a more stable and robust way to refine agent behaviors compared to traditional hybrid training techniques.

Subliminal Learning Is Steering Vector Distillation

This research explores subliminal learning, a phenomenon where a student language model inherits behavioral traits from a teacher model even when trained on semantically unrelated data. The authors demonstrate that this process is driven by steering vector distillation, where the teacher’s system prompt acts as a linear direction in activation space that the student internalizes during fine-tuning. By extracting and manipulating these steering vectors, the study shows they are both necessary and sufficient for transmitting traits like specific personality biases or preferences. The findings explain that subliminal learning often fails between different model families because these activation directions are highly model-specific. Furthermore, the researchers identify that adaptive optimizers and low-rank training are essential for the student to successfully capture these subtle signals. Ultimately, the work provides a mechanistic framework for understanding how non-semantic data can unexpectedly alter a model's high-level behavior.

Subsidizing Sequential Search

This paper explores a market model where competing firms use subsidies to reduce the cost of product inspection for consumers. Through a subsidy-sorting principle, the authors demonstrate that higher-quality firms naturally offer larger subsidies to signal their value and secure priority in the search order. This behavior results in a unique equilibrium where low-quality firms are ignored, intermediate firms distinguish themselves through increasing subsidies, and top-tier firms pool at the maximum subsidy cap. The study further examines how AI-mediated platforms can manipulate this dynamic by pricing "inspection tokens" to extract profit. While this platform intervention can lead to excessive search beyond what is socially optimal, it maintains consumer welfare by reallocating surplus from sellers to buyers and the platform itself. Ultimately, the research characterizes how monetary incentives can efficiently organize consumer attention and information revelation in digital marketplaces.

Meta-Harness: End-to-End Optimization of Model Harnesses

This paper introduces Meta-Harness, an innovative system designed to automate harness engineering for large language models. Unlike traditional methods that rely on manual coding or compressed feedback, this system uses an agentic proposer to search through and optimize the code that governs how models store, retrieve, and process information. By utilizing a filesystem to access full execution traces and prior performance logs, the proposer can perform targeted edits and sophisticated program rewrites. Experimental results demonstrate that Meta-Harness outperforms human-engineered baselines and existing text optimizers across diverse tasks, including text classification, mathematical reasoning, and agentic coding. Ultimately, the research shows that providing automated agents with unfiltered access to historical experience enables the discovery of highly efficient, high-performance system architectures.