Five Docker Containers That Make a Home AI Lab Actually Work

Why a home AI lab is five containers, not one box

Think of the stack as five responsibilities rather than five products. Model serving answers inference requests. The interface turns those requests into something a human can use. Retrieval memory stores the vectors that let the lab answer questions from your own documents. Automation handles the glue jobs. Management lets you see what is running, what broke and what needs updating.

Keeping those jobs separate has a practical payoff: when a new model runtime appears, you replace one container. When a better interface ships, you replace another. The two non-AI containers, n8n and Portainer, are not decoration. They are the reason the other three can be maintained without memorizing a pile of shell commands. If you want to see the same separation-of-concerns logic applied to agents, Docker Agent: Run AI Agents Like Containers with a Single YAML File describes a design where an agent is a configuration file rather than a program, which is the same instinct in a different layer.

Ollama for model serving: the endpoint everything else talks to

Ollama pulls and runs local models and exposes an HTTP endpoint on port 11434. That port is the contract. Every other container in the lab talks to it, which is why it deserves to be documented somewhere other than your shell history.

The official Docker Hub image offers three routes. The CPU-only image runs anywhere. NVIDIA GPUs need the NVIDIA Container Toolkit and the --gpus=all flag. AMD GPUs use the rocm tag. Pick the route that matches your hardware and write it down.

The volume is the part people skip. Models must live in a named volume. A container recreated without one re-downloads every model from scratch, and worse, it can lose a model that was pulled from a tag which has since moved. If you have ever wondered why a model that worked last month now behaves differently after a restart, this is a likely culprit.

GPU pass-through is a host question, not an Ollama question

This is where expectations diverge from reality, so treat it as an aside rather than a sixth container. GPU pass-through depends on the host, not on the inference container. The documented Docker routes for GPU access are Linux and Windows with WSL2. Docker Desktop on macOS does not pass an NVIDIA GPU into containers at all.

That leaves Mac users with a choice: run Ollama natively for speed, or accept CPU inference inside Docker and keep the containerized workflow for consistency. Neither is wrong. What is wrong is spending an evening debugging a container when the constraint sits in the host layer. If you are on a Mac, decide up front which trade you are making and document it next to the compose file.

Open WebUI for the browser layer

Open WebUI adds chats, users, file uploads and connections to Ollama and other providers. Docker is the officially recommended install, and the quick start is a single command, which is exactly the kind of low-friction entry point that makes it easy to forget the volume.

The volume at /app/backend/data holds users, chats and uploaded files. Without it, every recreate starts from a blank login page. That is the single most common way a home lab loses a month of conversation history.

The second detail is WEBUI_SECRET_KEY. It signs sessions. Set it once, for example with openssl rand -hex 32, and keep it somewhere durable. Regenerating it logs everyone out, which is mildly annoying in a one-person lab and genuinely disruptive if other people on the same network have accounts.

There are :cuda and :ollama image variants for GPU acceleration and for bundling Ollama into the same container. Both are optional. A separate Ollama container is easier to upgrade and easier to share with other tools, which is the whole argument for the five-container layout.

Qdrant for retrieval memory

Qdrant is optional for casual chat and required the moment the lab has to answer questions from your own documents, notes or project files. It stores vectors and finds nearest neighbors. In plain terms, it is the memory the AI consults, and it is distinct from the AI itself.

The local quick start uses two ports: 6333 for the REST API and dashboard, and 6334 for gRPC. It also mounts a storage path so collections survive restarts. Miss that mount and you get the same story as the other containers, just with embeddings instead of chat logs.

The dashboard at localhost:6333/dashboard is the fastest way to confirm an ingestion pipeline is actually working. Before you debug a retrieval prompt, check that the collection has points in it. Most “the model is hallucinating” reports turn out to be “the ingestion job never ran.”

n8n for automation

n8n runs the glue jobs visually: embedding files that land in a folder, calling model endpoints on a schedule, pushing summaries to a destination a human will actually read. None of that is AI work, and all of it is what turns a demo into a lab.

Docker Compose is the sensible install route. n8n serves on localhost:5678 and defaults to SQLite, with Postgres as an option once workflows multiply. It holds credentials and workflow definitions, so its volume needs a backup. It also should not be exposed beyond your local network, because a workflow tool with stored credentials is a much more interesting target than a chat interface.

This is the container that quietly replaces the cron jobs living in your shell history. That migration is worth doing deliberately rather than eventually.

Portainer CE for management

Portainer CE is a browser UI for restarts, logs, volume checks, spotting image updates and cleaning up stopped containers. The official install runs on HTTPS port 9443 and keeps its state in a portainer_data volume.

The install also mounts the Docker socket at /var/run/docker.sock, and that is the part to treat with care. Access to the socket is effectively root-equivalent control over the host, so Portainer belongs on a trusted local network, behind a strong password, and nowhere else. Treat the socket mount as a deliberate decision rather than a line you copied without reading.

The failure modes that look like AI problems but are not

Four symptoms account for most of the frustration in home labs, and none of them are model problems.

Unrecorded ports. Something works, you move on, and six weeks later nothing can reach the endpoint because the port mapping only ever existed in a terminal scrollback. Every port in the stack, 11434, 6333, 6334, 5678 and 9443, should be written down next to the compose file.

Unindexed files. The documents are on disk, the model is fine, and the answers are wrong because ingestion never ran. The Qdrant dashboard settles this in seconds.

Cron jobs living in shell history. Scheduled work that exists only as a remembered command is not automation, it is a habit. Moving it into n8n makes it visible, editable and backed up.

Models lost on update. A recreated container without a named volume re-downloads everything, and a moved tag can change behavior silently. Named volumes plus pinned tags solve both halves.

Each of these traces back to a missing volume, a missing secret or an undocumented port.

What actually keeps a home lab alive

The pattern across all five containers is the same: the interesting part is never the image, it is the state around it. A named volume for models, a mounted path for vector collections, a fixed secret key for sessions, a backed-up volume for workflow credentials and a carefully scoped socket mount for management. Get those right and the lab survives updates, hardware changes and the occasional container that refuses to start.

Get them wrong and you will spend your evenings re-downloading models and re-creating accounts, convinced that local AI is fragile. It is not. The stack is five containers with clearly separated jobs, and the operational details are what make the separation real. Write down the ports, mount the volumes, set the secrets once, and back up the two containers that hold anything you would miss.

Ports, paths, image tags and flags verified against upstream documentation on 5 March 2026.