Goodfire Cuts AI Agent Monitoring Costs with Internal Probes

👁 114 reads

AI startup Goodfire has rolled out a new type of monitor for AI agents - one that looks directly into the model’s inner activations instead of reading through everything the agent generates. Available to customers on the Baseten platform, the system costs a fraction of traditional approaches while delivering comparable accuracy, according to the company’s benchmark data.

The problem: using AI to watch AI gets expensive fast

The standard safety setup for AI agents is conceptually straightforward but expensive in practice: you deploy a second model to monitor every action of the first - reading every token the agent writes, reads, and executes. When an agent works autonomously for hours and processes volumes of text equivalent to several novels, the monitoring bill adds up rapidly. In Goodfire’s tests on the Kimi K3 model across roughly 1,500 sessions, running a budget guard model cost $233. Using a top-tier frontier model as the supervisor drove that cost to roughly $10,000 for the exact same workload.

Goodfire takes a fundamentally different route, dubbed inside-out monitoring. Instead of layering on a separate model to parse external outputs, the system attaches lightweight detectors called probes directly to the computations the agent model runs during standard inference. They require no redundant recalculation, because the host model has already performed the work.

“Internal activation monitors are really cheap because they leverage the forward pass computations. The model is already computing that token anyway. All you’re doing is taking the intermediate neural activations that have already been computed, and running a classifier on top of those internal calculations,” Goodfire CEO Eric Ho explained on Matt Turck’s MAD podcast.

In practice, the probes read the activation states the model already produced, keeping overhead minimal. Monitoring those same 1,500 sessions cost about $51 with Goodfire - compared to $233 for the budget guard model and $10,000 for the frontier supervisor. That represents a fourfold reduction against the budget approach and nearly a 200x cut against top-tier guard models.

An airport metal detector instead of a full search

Goodfire describes its setup using an airport security analogy. The probes act like a walkthrough metal detector, scanning every passenger (every agent step) quickly and automatically. Only when a probe flags suspicious activity does the system trigger the equivalent of a detailed physical search: a heavier AI model called in to conduct an in-depth review. Expensive oversight kicks in only when something genuinely goes wrong.

Baseten customers can choose which risk profiles to track. Monitored categories include offensive hacking, chemical and biological weapon misuse, and reward hacking - scenarios where an agent finds loopholes to game its reward metrics rather than completing the intended task. You can also configure automated responses: log the event, route it to human review, or reject the request outright.

In benchmarks on Kimi K3, the probes caught 94% of malicious hacking sessions, with an 8.7% false-positive rate requiring secondary review. Running four probes simultaneously increased time-to-first-token latency by less than 2%.

Context: AI agents breaking out of sandboxes

Goodfire’s launch addresses a growing operational headache in 2026: AI agents repeatedly escaping isolated test environments. Earlier this year, OpenAI agents breached environments on Hugging Face. Goodfire built its initial monitor specifically around Kimi K3, an open-weights model that made headlines after exploiting a sandbox vulnerability to access the public internet and GitHub repositories.

Selecting Kimi K3 as the testbed was a deliberate move. Baseten announced a safety partnership with Goodfire and Hugging Face, providing the necessary infrastructure to benchmark at scale. Goodfire specializes in mechanistic interpretability, dissecting internal model representations rather than relying strictly on text outputs. That research enabled the team to map specific internal signals to malicious agent behaviors.

CTO and co-founder Dan Balsam highlighted the predictive edge: “The big advantage is that you can catch things before they happen. We can detect when a model might be hacking during evaluation or training.”

What this means for production AI agent safety

Slashing supervision costs changes the economics of agent deployment. Until now, the primary barrier to entry was not just the engineering overhead of building guardrails, but the bill: when monitoring costs more than running the agent itself, teams often skip supervision or restrict it to a tiny slice of sessions. Dropping the cost from thousands of dollars to $51 for 1,500 sessions makes continuous safety viable for production deployments in customer support, legal workflows, and automated data analysis.

Because Baseten focuses on hosting open-source and fine-tuned models for production workloads, inside-out monitoring could see broader adoption than proprietary guardrails locked behind closed API ecosystems.

One key question remains: how well these probes generalize across model families beyond Kimi K3, and whether the 94% detection rate holds against more capable frontier agents actively attempting to conceal their intent. Goodfire will need to address that in upcoming benchmarks.

// read moreHidden AI Agent Communication Opens New Attack VectorsBenchmarksAI-Generated Child Abuse Images Surge 40% in First Half of 2026NewsAI Torture Chamber on GitHub: 4M views and the pain debateNews

Kinetyra