Anthropic details how Claude automates 95% of its internal analyses

Anthropic automates 95% of its internal data analyses using Claude with 95% accuracy by combining canonical datasets, semantic layers, and Markdown skills.

Pointing an agent at a data warehouse and letting it run creates a false sense of accuracy: the warning comes from Anthropic, which describes in a feedback report how 95% of its internal analytical queries are now handled by Claude, with approximately 95% aggregate accuracy, freeing up the data team for causal modeling or forecasting.

The central thesis: analytical reliability is a problem of context and verification, not code generation, with most errors boiling down to three causes: ambiguity between the concepts in a question and the entities in the data model, outdated definitions, and failure to retrieve the correct information. The solution combines unique canonical datasets, a semantic layer that must be consulted first, and especially skills—these Markdown files read on demand: without them, accuracy plateaued at 21% on internal evaluations, compared to over 95% afterward and nearly 99% in certain domains.

The team emphasizes maintenance, as accuracy slipped from 95% to 65% in one month due to a lack of skill updates, and quantifies its trade-offs: an adversarial review by a sub-agent gained 6 accuracy points at the cost of 32% more tokens and 72% more latency. To get started, it recommends a few canonical datasets, a few dozen offline evaluations, and a minimal knowledge skill.