The AI safety conversation in public discourse has collapsed into a binary: either you think safety is a brake on progress, or you think progress itself is dangerous. Both camps are spending their energy in the wrong place. The engineering reality is that safety mechanisms are load-bearing infrastructure — not because regulators require them, but because production AI systems that fail in unexpected ways cost money, erode trust, and in high-stakes domains, cause real harm.
We invest in the engineering side of this. Specifically in the companies building the tooling that makes AI systems auditable, predictable, and correctable in production.
Separating the Signal from the Noise
The safety debate that plays out in congressional hearings and on social media is about existential risk, model sentience, and the long-term trajectory of intelligence. That's a legitimate philosophical conversation, but it's not what's operationally relevant for an enterprise buying AI software today.
What's operationally relevant: Does this model produce harmful or misleading output under adversarial inputs? Can I audit the model's decisions when a regulator asks me to? If the model's behavior drifts after a fine-tune, will I know? Can I constrain the model's behavior to a defined scope without degrading its core capabilities?
These are engineering problems. They have engineering solutions. And the market for those solutions is large and growing fast — driven not by philosophical concern but by procurement requirements at regulated enterprises, insurance liability in autonomous systems, and the simple business risk of deploying systems you can't explain when something goes wrong.
Three Technical Areas We're Watching
Interpretability tooling. Mechanistic interpretability — the effort to understand what's actually happening inside a model at the activation level — has moved from pure research to applied tooling over the last eighteen months. Companies building on top of this work are producing tools that can attribute model outputs to specific training examples, identify circuits responsible for specific behaviors, and flag anomalous activations that precede failure modes. This is early, but the trajectory is clear.
Red-teaming infrastructure. Manual red-teaming doesn't scale. A human team can probe a few hundred edge cases before a deployment. Automated red-teaming systems can probe millions of cases and prioritize by severity. The teams building systematic adversarial testing pipelines — ones that integrate with deployment workflows, track coverage, and surface regressions when model versions change — are solving a genuine enterprise need. We've seen CISO teams at financial institutions list "adversarial test coverage" as a deployment gate for AI systems. That's procurement pull, not marketing push.
Constitutional and preference-based alignment methods. RLHF and its variants are how most production models are fine-tuned to behave appropriately. The tooling around this process — preference data collection, reward model training, policy optimization, and evaluation of the resulting behavior — is still fragmented and largely bespoke. Companies building repeatable, auditable fine-tuning workflows with explicit behavior constraints are serving a market that didn't exist at scale three years ago.
The Regulatory Tailwind
We try not to invest primarily on regulatory tailwinds — regulations change, get delayed, and sometimes die in committee. But the directional signal on AI regulation is clearer than most. The EU AI Act has passed. The US executive order on AI has pushed federal agencies to establish AI risk frameworks. Financial services regulators in multiple jurisdictions are asking banks to document AI decision-making. Healthcare regulators are treating AI diagnostic tools more like medical devices.
Each of these creates a procurement requirement. Organizations deploying AI in regulated contexts need audit trails, explainability documentation, and evidence of adversarial testing. The tooling that produces this evidence is becoming mandatory, not optional.
Safety infrastructure is increasingly the difference between a deal closing and a deal stalling at legal review. That's not a philosophical position — it's what we hear from our portfolio companies in their sales cycles.
What This Looks Like in Practice
One of our portfolio companies went into a Series A sales process at a large healthcare system. The technical evaluation went well. The legal and compliance review nearly killed the deal because the system couldn't produce a structured audit trail of model decisions. They spent six weeks building that capability reactively. The deal closed, but the lesson was clear: safety infrastructure needs to be built in, not bolted on.
That pattern repeats across verticals. The teams that treat safety as a product feature — not a research initiative or a compliance checkbox — move through enterprise procurement faster. And the companies building the infrastructure that makes that feature possible are solving a real distribution bottleneck for AI adoption.
What We're Looking For
We want to talk to founders who have firsthand experience with the failure modes — who've seen what happens when a production model behaves unexpectedly at scale, and who've built the systems to detect and correct it. The best safety infrastructure founders we've met combine research-grade depth on model behavior with practical engineering intuition about what enterprise buyers actually need.
The companies in this category that win will do two things: they'll make deployment teams feel confident shipping, and they'll make compliance teams feel confident signing off. Those are different audiences with different concerns. The product design that satisfies both is genuinely hard to get right.
Building AI safety tooling, red-teaming infrastructure, or interpretability products? We're actively looking.
