Of 20 codes hidden in financial filings, Manta finds 18

Empero says Manta can handle 100 million tokens in one request. Its longest published test used 88 million tokens of financial filings.

The code for the Lyon office appears in a memo near the beginning of an 88-million-token corpus. Manta finds it when asked for Lyon’s code, then gives the same code for Perth and Rotterdam. Those are its only two errors across 20 questions in the test log published by Empero. In both cases, the model retrieves a code that appears in the corpus and assigns it to the wrong office. Twenty codes across 1,269 filings

The test protocol on GitHub describes a corpus of 1,269 filings that S&P 500 companies submitted to the U.S. Securities and Exchange Commission, including annual and quarterly reports. Empero converted the filings to plain text and inserted short fictional memos at evenly spaced points between 5% and 95% of the way through the input. Each memo pairs an office with a unique code. The researchers asked one question at a time after supplying the corpus and counted an answer as correct if it contained the exact code. Each of the 20 questions names the office whose code the model must find.

At about 10.5 million tokens, Manta found 17 of the same 20 codes with one answer per question. When the researchers sampled three answers per question and counted a hit if any of them contained the right code, the score rose to 18 out of 20. A third test used 50 memos at the same context length, with similar-looking codes and office names. Manta found 43. Empero says it chose one setting after reviewing the results of the 50-code test, then ran the 88-million-token test as a follow-up check.

All 12 single-answer errors in the published results for the three tests followed the same pattern. The model returned a code from another memo in the corpus. In the 50-code test, for example, it gave Marseille’s code when asked for Bordeaux’s. The distinction matters in document research. A quote or figure can be reproduced exactly and still be attributed to the wrong source. The longest published test stops at 88 million tokens

In its announcement, Empero says Manta can take 100 million tokens in a single request. Its longest documented test used 88 million. The public files contain the source text, the memo locations, and the recorded answers, so readers can reconstruct the inputs and check the scores. Connecting distant passages or combining figures scattered across filings would test different abilities. The RULER benchmark makes that distinction by evaluating retrieval, reasoning across linked information, and aggregation separately.

Empero plans to offer an interface compatible with the OpenAI API and has opened a waitlist for limited early access. Early users can share their impressions, but publishing outputs, prompts, screenshots, or measurements requires Empero’s permission until the public release. The lab says it is now working toward a context window of one billion tokens per request.