Can a Raspberry Pi 5 run a real language model on its own, with no cloud API anywhere in the loop?
Google says yes, and the new LiteRT CLI turns that claim into a ten-minute test you can run tonight. LiteRT is the successor to TensorFlow Lite, retuned for Arm boards, and Google engineers published a step-by-step walkthrough on the Raspberry Pi blog that takes you from a blank SD card to a Gemma chat session over SSH. No monitor, no keyboard juggling, just an Ethernet cable between your laptop and the Pi.
What the setup actually looks like
Flash Raspberry Pi OS (64-bit) with Imager, tick the SSH box, set a hostname, and boot. Once uname -m prints aarch64 you are in business. The install itself is three commands: pull down the uv environment manager, spin up a Python 3.13 venv, then uv pip install litert-cli-nightly. Models come from Hugging Face, so you also need a read-scope access token exported as HUGGING_FACE_HUB_TOKEN before the first download.
The numbers that matter
Running litert lm run against gemma-4-E2B-it pulls a 2.59 GB model file, which clocked roughly 20.4 MB/s on the demo machine, so budget SD card space accordingly. Classic vision models are far lighter: litert download litert-community/efficientnet_b1 gives you a .tflite file that classifies an image in one command, and the sample run scored a tiger shark at 7.4043 against 4.7619 for the runner-up. Here is the gotcha worth knowing before you chase it: GPU acceleration exists behind V3D_WEBGPU_OVERRIDE=1 on the Pi 5‘s V3DV Vulkan driver, but it is experimental and currently slower than the CPU path. Skip the --gpu flag unless you are specifically testing Dawn WebGPU. Hailo AI HAT+ offload is on the roadmap, which is where the real speedup will land.
Try it on your capstone
A Pi 5, a camera module, and EfficientNet is a complete object-classification pipeline for a thesis demo, and it keeps working when campus WiFi dies. Start with the vision models rather than the LLM, since a 2.59 GB download is a painful first step on a shared connection. Full instructions and the exact commands are on the Raspberry Pi blog: Get started with LiteRT on Raspberry Pi. Grab a Pi 5 and an active cooler from Circuit.Rocks, because sustained inference will heat that chip fast.
Frequently Asked Questions
Do I need an AI HAT+ to run LiteRT on a Raspberry Pi 5?
No. The LiteRT CLI runs on the Pi 5‘s CPU out of the box, and right now the CPU path is actually faster than the experimental WebGPU one. Hailo AI HAT+ and AI HAT+ 2 offload is coming, which will speed things up without changing the commands you type.
How much storage does a Gemma model need on the SD card?
The gemma-4-E2B-it LiteRT build is a 2.59 GB download, so plan for a 32 GB card at minimum once you account for Raspberry Pi OS and the Python 3.13 venv. Vision models like EfficientNet B1 are a few megabytes and are a much gentler starting point.
What will I learn if I build this?
You will pick up headless Pi setup over SSH, Python virtual environments with uv, Hugging Face token auth, and how quantized models trade accuracy for speed on Arm hardware. Those skills transfer straight into any embedded vision or robotics capstone, and they look good on a thesis defense slide.
