Running AI Models on Raspberry Pi: A Practical Deployment Guide

Which AI Models Actually Run on Raspberry Pi

Computer Vision

YOLOv8n (nano) is the standard starting point for object detection on Pi. It is the smallest variant in the YOLOv8 family and the one Ultralytics explicitly positions for resource-constrained deployment — the variant name "n" for nano is the signal. On Pi 5 with an optimised runtime (NCNN or TFLite rather than PyTorch), it achieves frame rates suitable for many production applications. YOLOv8s, YOLOv8m, and larger variants are significantly slower — on Pi CPU alone they are not suitable for real-time use. If you are starting a new CV project on Pi, begin with YOLOv8n and only step up to a larger variant if accuracy profiling on your specific dataset demands it.

YOLOv5n remains in use on legacy deployments and is a valid choice if your tooling is already built around it. The same size constraint applies: nano variant only for real-time Pi inference.

MobileNet-SSD and other MobileNet-family classifiers are well-suited to Pi via TFLite. Designed for mobile and edge from the start, they are efficient on ARM and integrate naturally with the Coral USB Accelerator if you want hardware-assisted throughput on TFLite models.

OpenCV with the face_recognition library (dlib-based) is the practical choice for face recognition authentication tasks — not continuous high-FPS tracking, but event-driven recognition (door approach, button press) where you process one frame per trigger. On Pi 5, the dlib recognition pipeline is suitable for access-control response times. This is what we use in our FZLockers deployment.

Speech Recognition

Whisper tiny / base via whisper.cpp is the standard approach for speech-to-text on Pi. whisper.cpp is a C++ port of OpenAI's Whisper that runs significantly faster than the original Python implementation, making the tiny and base model variants practical on Pi 5 hardware. For lighter wake-word detection where Whisper's footprint is excessive, Vosk and Porcupine are alternatives with lower resource requirements and sub-second latency.

Classical ML and Sensor Inference

Scikit-learn models — random forests, gradient boosting, SVMs — exported to ONNX or Joblib are extremely fast on Pi: millisecond-class inference for tabular sensor data. Anomaly detection, predictive maintenance classification, and real-time sensor stream processing are all well within Pi's capability without any hardware acceleration. This category covers a large proportion of real production edge AI use cases and is often overlooked in favour of flashier computer vision work.

Large Language Models — Be Honest About the Limits

Small quantised LLMs — TinyLlama 1.1B or Phi-2 at Q4 quantisation via llama.cpp — will run on Pi 5 with 8 GB RAM. They are slow: expect single-digit tokens per second. This is batch-only territory, not interactive. A Pi running a local LLM can serve use cases where response latency is not critical — generating a short status report overnight, for example. For anything requiring conversational response times, Pi is not the right inference platform. Run the LLM server-side and use the Pi as the interface and orchestration layer.

The AI Deployment Pipeline on Raspberry Pi

Deploying an AI model to Pi is a pipeline, not a single step. Getting each stage right — from model selection through runtime choice to the capture pipeline — is what separates a prototype that works on a desk from a system that holds up in production. The diagram below shows the six stages every Pi AI deployment goes through.

You start with a model: either a pre-trained one (YOLOv8n weights, Whisper tiny, an OpenCV classifier) or one trained on your own data using cloud GPU resources. That model almost certainly is not ready to run on Pi in its native format — it needs to be optimised. Quantisation (FP32 to INT8) cuts memory footprint and speeds up inference with a small, usually acceptable accuracy trade-off. Format conversion — from PyTorch or TensorFlow to ONNX, then to TFLite or NCNN — is necessary to reach the most efficient Pi runtime. Once optimised, the model file lands on the Pi alongside the Python or C++ runtime that runs it. The input pipeline (camera, mic, sensors) feeds frames or audio into inference, and inference drives the application action: unlocking, logging, transcribing, dispensing.

Common Mistakes in Raspberry Pi AI Deployments

• ✕
  Benchmarking with unquantised PyTorch models

  Measuring Pi inference performance with a raw FP32 PyTorch model gives you a worst-case number, not a production-representative one. Convert and quantise first; then benchmark. Teams that skip this step incorrectly conclude Pi is too slow.

• ✕
  Demanding server-class frame rates

  Pi is not a $3,000 GPU workstation. Define the minimum acceptable frame rate for your application — not the maximum possible — and test against that. Many production use cases need 2–10 FPS, not 60.

• ✕
  Ignoring thermals

  A Pi 5 running sustained inference without active cooling will throttle and produce inconsistent latency. Thermals are a deployment engineering requirement, not an afterthought. Test under sustained load — not just for five seconds.

• ✕
  Not pinning Python and library versions

  TFLite, NCNN, OpenCV, and dlib bindings change across versions and can break each other. Pin your virtualenv or use a locked container image for production deployments. Uncontrolled package upgrades on a deployed Pi will break inference quietly.

• ✕
  Deploying without monitoring

  Edge AI systems require ongoing monitoring: inference latency, model accuracy over time as environmental conditions change, and hardware health (temperature, uptime). A model that was accurate at launch can degrade. Build structured logging and alerting in from day one.

Frequently Asked Questions

• Can Raspberry Pi run real AI models?+
  Yes — with the right model size and runtime format. Raspberry Pi 5 runs TensorFlow Lite, ONNX Runtime, PyTorch Mobile, and NCNN natively via Python or C++. Lightweight object detection (YOLOv8n), face recognition (OpenCV + dlib), speech-to-text (Whisper tiny/base via whisper.cpp), and scikit-learn sensor models all run well in production. The key constraint is model complexity: nano and tiny variants are the productive range for Pi CPU inference. With a Hailo HAT or Hailo HAT+ accelerator attached, the ceiling rises substantially for computer vision workloads.
• Which Raspberry Pi is best for AI — Pi 4 or Pi 5?+
  Pi 5, by a meaningful margin. The Pi 5 uses the BCM2712 SoC with quad-core ARM Cortex-A76 cores at 2.4 GHz and LPDDR4X memory. The Pi 4 uses BCM2711 with Cortex-A72 at 1.5 GHz. The Cortex-A76 is a significantly more capable core — a newer microarchitecture with better out-of-order execution and a higher IPC. For AI inference, this difference is substantial: workloads that ran near the edge of feasibility on Pi 4 often run comfortably on Pi 5. Pi 5 also has better support for active cooling and the M.2 slot for the Hailo HAT+, which Pi 4 lacks.
• Can a Raspberry Pi run YOLO in real time?+
  It depends on the variant, runtime format, and what 'real time' means for your application. YOLOv8n (nano) is the smallest YOLOv8 variant and the one Ultralytics explicitly positions for resource-constrained deployment. On Pi 5 with an optimised runtime (NCNN or TFLite, not PyTorch), it achieves frame rates suitable for many production CV applications. YOLOv8s and larger variants are significantly slower on Pi CPU alone — not practical for real-time use without a Hailo HAT. If you need multi-stream or high-FPS detection beyond what Pi CPU allows, the Hailo HAT+ (26 TOPS) changes the equation substantially.
• Can Raspberry Pi run Whisper speech-to-text?+
  Yes — whisper.cpp (a C++ port of OpenAI's Whisper) is the right tool for this on Pi. The tiny and base model variants are the practical size range. On Pi 5, whisper.cpp processes short utterances faster than real-time, making it suitable for command recognition and short-form transcription. The original Python Whisper is substantially slower and not the correct approach for Pi deployment; whisper.cpp is the standard for resource-constrained hardware and is what we use in our own Pi speech deployments.
• Do I need a Hailo HAT or Coral USB Accelerator?+
  Not necessarily — establish your CPU baseline first. For YOLOv8n-class computer vision at modest frame rates, or speech recognition with whisper.cpp, Pi 5 CPU alone may meet your application's requirements. Profile on your actual hardware and input pipeline before buying accelerators. If the CPU alone cannot hit your target frame rate, the Hailo HAT (Hailo-8L, 13 TOPS) and Hailo HAT+ (Hailo-8, 26 TOPS) are the first-party Pi acceleration options for CV workloads. The Coral USB Accelerator from Google is an alternative for TFLite-compatible models. Add hardware acceleration after profiling confirms it's needed.
• How much RAM do I need for AI on Raspberry Pi?+
  For computer vision workloads — YOLOv8n, face recognition, MobileNet-SSD — 4 GB is a comfortable baseline. It leaves headroom for the OS, camera pipeline, and any application logic running alongside inference. 2 GB is technically sufficient for single lightweight models but leaves little margin. For small quantised LLMs (TinyLlama 1.1B, Phi-2 Q4 via llama.cpp), 8 GB is recommended. Avoid the 1 GB Pi variants for any AI inference use case — they will hit memory pressure with even lightweight models when the OS and runtime are loaded.
• When should I NOT use a Raspberry Pi for AI?+
  When the application needs multi-stream HD inference at production frame rates, large transformer models (3B+ parameters) for interactive use, hard real-time control running simultaneously with heavy inference, or sustained throughput that Pi CPU cannot meet even with a Hailo HAT+. In those cases: NVIDIA Jetson Orin NX or Orin Nano for embedded edge AI, or server-side inference for non-latency-sensitive workloads. Pi is the right platform when it genuinely fits the requirement — and the wrong platform when it doesn't. Profile early, decide on evidence.