OGX, Formerly Llama Stack: Inside the Open-Source Agentic API Server That Swaps Models Without Rewriting Code

The GitHub page says it plainly: “Llama Stack is now OGX.” A rename is usually housekeeping. This one is a mission statement. Meta’s Llama Stack spent its first life as the serving layer for a single model family. OGX arrives positioned as something broader: an open-source agentic API server that is OpenAI-compatible, runs any model on any infrastructure, and ships under the MIT license.

The interesting claim is not the new name. It is the one underneath it: that the model and the infrastructure behind it are swappable without touching application code.

What OGX Actually Is

OGX describes itself as an open-source agentic API server for building AI applications. The pitch is a drop-in replacement for the OpenAI API that you run yourself, on a laptop, in a datacenter, or in the cloud, and then point any OpenAI-compatible client or agentic framework at.

The project reports more than 8,400 GitHub stars and 1,400 forks as of this writing, with regular community calls and active development. That matters for a practical reason: a large body of existing Llama Stack material, tutorials, issues and configuration notes, still applies directly to OGX under the new name. If you have Llama Stack code in a repository, the rename is not a rewrite.

What changed is the framing. Model-agnostic, multi-SDK, production-grade. The old project was the reference server for one family of weights. The new one wants to be the neutral layer between your application and whatever is serving tokens this quarter.

One API Surface, Many Backends

The architectural bet is a pluggable provider layer behind a single API surface. Develop locally against Ollama, deploy to production against vLLM, or connect to a managed cloud service, all without rewriting the application. The backend changes; the client code does not.

Routing happens through a naming convention. Model strings follow a provider/model-name format, so ollama/llama3.2:3b tells OGX which configured provider should handle the request. That single string is the seam between your code and the infrastructure. Swap it, and the same call lands somewhere else entirely.

For teams that have lived through a serving-stack migration, this is the part worth attention. The usual failure mode is not the model change itself but the glue: bespoke client wrappers, provider-specific request shapes, retrieval code that assumes one vendor’s embedding endpoint. Collapsing that into one OpenAI-compatible surface is the actual product.

The Responses API and Server-Side Orchestration

OGX’s Responses API handles server-side agentic orchestration, tool calling, MCP server integration, and built-in file search for retrieval-augmented generation in a single API call. The server does the loop, not your application.

That is a deliberate trade. Client-side orchestration gives you total control and total responsibility for retries, tool dispatch and context assembly. Server-side orchestration moves that machinery behind the API boundary. For teams still deciding where that boundary should sit, our breakdown of the four levels of production agent architecture, From Single Loops to Multi-Agent Teams: A Four-Level Roadmap for Building Production AI Agents, maps the ladder from a single loop up to coordinated teams. OGX sits at the lower rungs by design: it makes the single loop and its tools boring, which is a compliment.

OGX also exposes multiple client-compatible surfaces against the same instance, including the OpenAI, Anthropic and Google GenAI SDKs. That is an unusual claim, and it is worth testing in your own environment rather than taking on faith. If it holds, a team already standardized on one vendor’s client library can point it at a self-hosted server and keep the code, while a colleague using a different SDK talks to the same endpoint.

Standing Up a Local Server

Setup assumes uv, the Python package and project manager, and a local model backend. The tutorial uses Ollama, with the same code working unchanged against vLLM or a cloud provider. uv init scaffolds the project, then uv add ogx openai installs the server package and the standard OpenAI Python client.

With Ollama running and a model pulled, the server starts at https://localhost:8321. OGX listens on HTTPS by default, which is the right default for anything reachable and an annoying one locally. If you would rather skip the certificate dance, check the current CLI for a flag to disable TLS for local development; otherwise, testing against the self-signed certificate means passing -k to curl or constructing httpx.Client(verify=False) in Python. A healthy response lists registered models under /v1/models, confirming both that the server came up and that Ollama connected.

One detail trips people up: api_key="fake" is not a placeholder to fill in later. OGX does not check the key for a local server, so any string works. Engineers conditioned by hosted APIs to treat that field as a secret will spend ten minutes hunting for a value that does not exist.

On the response side, response.output_text is the convenience property that returns the final text answer. No manual walk through a choices array.

RAG Without the Wiring

The retrieval story is where OGX saves the most typing. client.files.create uploads the raw document, and purpose="assistants" marks it as retrievable content rather than, say, a fine-tuning dataset. Then client.vector_stores.create performs server-side chunking, embedding and indexing into a searchable store.

That is the work most RAG tutorials ask you to do by hand: pick a chunk size, choose an embedding model, stand up a vector database, write the retrieval script, keep all four in sync. Here it is one call. Retrieval is then activated per question through the tools list on the responses.create call, which keeps the capability scoped to the requests that need it.

The cost is the familiar one. Convenience costs visibility. You get fewer knobs over chunking strategy and embedding choice, and you inherit whatever the server decides. For prototypes and internal tools, that is a good deal. For retrieval-heavy production systems with tuned pipelines, it is a starting point, not a destination.

The DocBrain Example

The tutorial’s demo application, DocBrain, has two functions. The first is real RAG: upload documents, ask questions, get answers grounded in their content, with retrieval happening automatically rather than through a manually wired vector database and a separate retrieval script.

The second is genuinely agentic. While reviewing documents, the assistant can call a custom tool to flag and save a real action item, such as a lease renewal deadline or a follow-up task, to a file on disk. That is the difference between a chatbot that summarizes and an agent that changes state outside the conversation.

One caveat: the custom tool’s full implementation and the exact schema passed to the model are not covered in the material we reviewed, so treat those as things to test yourself rather than documented behavior.

The broader point stands regardless. Writing a file from inside a model-driven loop is where agentic systems stop being demos, and it is also where they start needing guardrails: path validation, idempotency, and a clear answer to what happens when the model calls the tool twice.

Where OGX Fits in the Agent Stack

The case for a self-hosted, OpenAI-compatible server is not ideological. It is operational. The pressure that matters most here is backend portability, and it is the one OGX is explicitly designed around. Develop on Ollama, ship on vLLM, burst to a managed service. The application does not know the difference. The same seam answers the other common objections as a side effect: when the application talks to a surface you control, switching providers is a model string rather than an engineering project, and inference happens wherever you choose to run the server.

It is worth noting how much of this pattern is now in the air. Training and orchestration tooling is converging on the same instinct, that the harness around a model should be decoupled from the model itself. Microsoft’s Agent Lightning v1.0: Microsoft’s 3,500-Line Framework Trains the Agent Harness You Already Deploy makes a parallel argument from the training side, and the research-to-product pipelines described in Microsoft Research Asia – Singapore at Year One: How a ‘Research-to-Impact Flywheel’ Is Reshaping Frontier AI show labs reorganizing around the same assumption: the serving layer is infrastructure, and infrastructure should be replaceable.

OGX is a young project with an active release cadence and a recent identity change, which means documentation will lag behavior for a while. The claim to watch is not the star count. It is whether the provider abstraction holds when a real application moves from a laptop to a cluster without a rewrite. That is the test the rename is promising, and the one engineers should run themselves.

Related: agentic rag naive linear.