Google opens up its architecture to run AI on ultra-low-power devices

Google expands the Coral NPU open architecture and its compiler to run AI models on connected devices, sensors, and robots.

Running artificial intelligence continuously on a smartwatch, earbuds, or an industrial sensor involves solving a problem that does not present itself in the same way in data centers: every single operation consumes part of a limited battery. Google Research is seeking to reduce this power expenditure with Coral NPU, an open hardware architecture designed to run AI models directly on small devices. The project marks a new milestone with the completion of its 32-bit architecture, the release of an open-source compiler, and the arrival of commercial components developed with Synaptics. Reducing data movement rather than increasing calculations

In an artificial intelligence accelerator, power consumption does not depend solely on the number of operations performed. A significant portion of the energy is used to move data between the memory and the computing units. This problem becomes particularly constraining for battery-powered devices, which must continuously analyze information coming from microphones, cameras, or various sensors.

The Coral NPU architecture relies on RISC-V, an open instruction set, to which Google combines specialized units for operations used by neural networks. It includes a scalar processor for program control, a vector coprocessor for parallel computing, a tightly integrated memory subsystem, and now, a matrix execution unit.

This latest component, called the Matrix Execution Unit, completes the 32-bit version of the project. It handles matrix multiplications widely used in vision models and Transformer architectures. Instead of loading the same data multiple times to perform sequential vector calculations, its engine performs operations on two-dimensional blocks by locally reusing weights and activations.

Google describes a capacity of 256 multiply-accumulate operations per cycle, with an architecture configurable up to 1 TOPS, or one trillion operations per second. The design targets an average power consumption on the order of 10 milliwatts for inference. However, this is a design target, and not a battery life measured across all devices that might integrate this technology. More complex models on connected devices

Ultra-low-power accelerators are often associated with relatively simple functions, such as keyword spotting or motion recognition. The addition of a matrix engine makes it possible to consider more complex processing, notably vision models, advanced audio functions, and small language models.

Google cites real-time speech translation on wearable devices, spatial audio filtering in earbuds, and low-latency visual perception for robots. Compact models from the Gemma family are also among the architectures to which Coral NPU could be adapted.

The approach does not consist of transferring all the capabilities of a large cloud model to a smartwatch chip. Instead, Google envisions a workload distribution across multiple compute tiers. A small local model can continuously monitor signals received by the device and handle immediate processing. When an operation requires more resources or more elaborate reasoning, it can be offloaded to a larger model, running either on the device or in the cloud.

This setup helps limit certain data transmissions and reduces reliance on a permanent connection. By itself, it does not guarantee that all information will remain local: that depends on the applications, the choices of their developers, and the processing tasks actually delegated. A unified compiler for multiple chip manufacturers

Energy consumption is a primary hurdle. Component fragmentation is another. Manufacturers use different architectures and software tools, which often forces developers to adapt their models to each new piece of hardware, sometimes to the point of rewriting math operations or modifying their quantization.

To limit these adaptations, Google is releasing Coral Compiler v1.0, an open compilation toolchain based on IREE and MLIR technologies. It enables models originating from JAX, PyTorch, or TensorFlow to be prepared for execution on the Coral NPU architecture, notably using the StableHLO and TOSA intermediate representations.

The compiler distributes operations between the main processor and the NPU, fuses certain instructions, and arranges matrix calculations according to available memory. It then translates the processing tasks into Coral’s scalar, vector, and matrix instructions. Developers also have access to an MPACT simulator and a reference FPGA design to test their software and hardware choices prior to chip fabrication.

Google is making the 32-bit architecture's RTL code open access, describing the hardware circuits so it can be used as a foundation for silicon implementation. The public repositories for the architecture and the compiler are available on GitHub. However, the ability to reuse a single toolchain does not mean that any given model will run without modification on all hardware configurations: memory constraints, operation compatibility, and performance remain specific to each chip. Synaptics turns the architecture into commercial components

The project already has its first hardware implementations thanks to Synaptics. The manufacturer presented its second generation of Astra SR200 microcontrollers, intended in particular for wearable devices, industrial sensors, and embedded perception systems. This family advertises up to 400 GOPS of compute power and integrates a previous generation of the Coral NPU technology, including the RISC-V vector capabilities.

This distinction is important: the SR200 devices should not be confused with a complete implementation of the new 32-bit matrix architecture, whose full design Google is now publishing.

Synaptics is also offering a new development board, Coralboard SR, intended for prototyping systems based on the SR200. In parallel, the Coralboard SL, equipped with an Astra SL2619 processor and an accelerator advertised at 1 TOPS, is now available commercially worldwide, notably through DigiKey.

Bo