Halley
Model risk and compliance teams use Halley to deeply understand their models, surfacing the interactions, subgroup effects, and conditional patterns that other tools miss.

AI interpretability research and applications.
I design algorithms that allow humans to understand machine learning models: what they learn from their training data, how they make predictions, and where they might fail in unexpected ways.
Previously I built and interpreted models on histopathology data for cancer diagnostics, and did my PhD on model-agnostic interpretability (methods that explain a model's decisions without needing to see inside it).
Since then I've worked on AI interpretability for: failure modes of large language models (one of those, a set of tokens that broke GPT-3, made the news); scientific discovery (if a model can predict something you can't, it knows something you don't); and high-stakes applications for financial institutions where understanding model behaviour is critical.
I'm based in San Francisco.
Financial institutions use machine learning to make decisions with real consequences, and Halley allows them to understand and explain their models with unprecedented clarity.
Existing tools typically provide feature importances and score pre-specified subgroups – but that's not enough to really understand the patterns a model is doing. Halley explains the interactions a model depends on, the thresholds it leands, and the feature combinations where it systematically under- or over-predicts.
It also works on any kind of model – and this unlocks the ability to use more expressive and performant architectures that were previously out of reach due to their opacity.
Model risk and compliance teams use Halley to deeply understand their models, surfacing the interactions, subgroup effects, and conditional patterns that other tools miss.
A new method for finding interpretable structure in language models 1000x faster than sparse autoencoders.
Hypothesis-driven analysis can only ever find what you already suspect is there – here is a case for starting with the data instead.
Three publications where models trained on scientific data found novel patterns, and our interpretability methods made them legible. These findings were discovered automatically by Disco and written up with the domain experts who gathered the data.

A survey of how AI is used in science today, from systematic literature review to autonomous agents.
An efficient black-box method for attributing a language model's output back to the parts of the prompt that caused it, even if the model is hidden behind an API.
The glitch-token work: we discovered a set of very strange tokens that made GPT-3 break in unexpected ways.
Talks & interviews
A conversation with Swyx about data-first science and seeing what models learn.
On personal motivation, how AI can (and can't) aid scientific discovery, and why interpretability is important.
A segment with Brian McCullough on interpretability, AI, and seeing patterns that humans miss.
The pitch that won us the Novel AI track + many prizes at DTM.
A live session on using interpretability to surface patterns that standard analysis misses.
Press