Liquid AI tackles the last milliseconds of inference

Liquid AI introduces RIPL, an inference engine for real-time search, commerce, and financial workloads, with selected responses completed in milliseconds.

Product searches, payment authorizations, and ad selection happen within milliseconds. The systems handling these operations must process large numbers of simultaneous requests, leaving very little time for an AI model to respond. Conventional language-model inference can be difficult to integrate when a request takes hundreds of milliseconds or several seconds to complete. Liquid AI is addressing that gap with RIPL, short for Real-time Inference Pipeline, an inference engine designed to run its Liquid Foundation Models directly within live application requests. The company reports complete responses under 50 milliseconds for selected short workloads and up to 28.4 times lower 95th-percentile latency than comparison configurations using Qwen3.5 on vLLM. Target applications include search, e-commerce, advertising, financial services, and classification. Bringing Generative Models Into Transactional Systems

Most large-language-model serving infrastructure has been developed for conversational and agentic applications. These systems can tolerate several seconds of latency when generating documents, analyzing information, or carrying out complex sequences of operations. Inference engines therefore concentrate on throughput, accelerator utilization, batching, and the efficient generation of long outputs.

The constraints are different when a model operates inside an online store or financial service. A search engine must interpret a query before displaying results. An advertising platform has limited time to select relevant placements. A payment system may need to evaluate risk before authorizing a transaction. In each case, AI inference shares a latency budget with other parts of the application.

Companies have traditionally relied on specialized classifiers, gradient-boosted decision trees, and compact embedding models for these tasks. They can execute quickly, but their behavior is usually narrower than that of a general-purpose language model.

RIPL is designed to make small generative models practical inside these latency-sensitive processes. It prioritizes short or structured outputs under continuous traffic, enabling models to interpret search intent, expand queries, or classify information without introducing the delays associated with larger conversational systems. Measuring Complete Responses, Not Just First Tokens

Liquid emphasizes end-to-end latency, measured from the arrival of a request until the entire output is produced. A model may start responding quickly but still take considerably longer to generate a complete sequence of tokens.

RIPL targets the entire request path rather than only the speed of token generation after the first output appears. Its performance objective also includes maintaining low latency under sustained traffic, when hundreds or thousands of requests may arrive every second.

An important metric is p95 latency, the time within which 95% of requests are completed. Average response times can conceal significant delays among slower requests. The p95 metric makes those delays visible, particularly in applications where even a small proportion of slow responses affects service reliability.

The reported results cover different models, input lengths, output lengths, and request rates. They do not establish that every RIPL request completes within 50 milliseconds. Short workloads produce the most favorable results; longer inputs and outputs can exceed that threshold considerably. Large Latency Differences Compared With vLLM

On an NVIDIA B200 GPU, Liquid reports that LFM2.5-1.2B running through RIPL reaches 34.6 milliseconds p95 latency for a workload containing 32 input tokens and 32 output tokens at 1,000 requests per second. Under the comparison configuration, Qwen3.5-0.8B running on vLLM reaches 498 milliseconds, a difference of 14.4 times.

For a longer workload involving 256 input tokens and 128 output tokens at 400 requests per second, Liquid reports 155.8 milliseconds with RIPL, compared with 4,431 milliseconds with Qwen3.5 on vLLM, corresponding to 28.4 times lower latency.

The larger LFM2.5-2.6B, compared with Qwen3.5-2B, shows smaller but still substantial reported improvements: between 5.6 and 7.7 times lower p95 latency, depending on the tested workload.

These comparisons involve different models running on different inference engines. They therefore cannot attribute every performance gain specifically to RIPL. Model architecture, parameter counts, token processing, and serving configuration may all affect the results. Liquid also shows selected same-model comparisons between RIPL and vLLM, but has not fully disclosed the internal optimizations responsible for its performance.

The benchmarks were conducted by Liquid AI and have not been independently reproduced under equivalent configurations. Search, Advertising, and Recommendations Need Short Outputs

Several of Liquid's examples involve applications that require very little generated text. In e-commerce, LFM2.5-350M may receive a 128-token query and produce a single token identifying its intent, category, or attributes. Most of the computation involves understanding the input rather than generating a long response.

Another application is search result reranking. After an initial retrieval system identifies relevant products, a model scores individual candidates against the customer's query. Liquid reports end-to-end task completion below eight milliseconds at up to 500 requests per second for a configuration involving 1,000 input tokens and one output token.

In advertising, a model can generate short query expansions to match user searches with advertiser keywords. The demonstrated workload contains 32 input tokens and 32 output tokens, allowing the system to explore alternative formulations without generating extensive text.

These examples illustrate RIPL's primary use case: frequent, relatively narrow decisions integrated into existing services. Liquid also states that RIPL supports its d1 decision models, which are