A breakdown of AnythingLLM's desktop version: workspaces and RAG, agentic skills and MCP, what Desktop lacks compared to the server version, and what privacy actually depends on.

AnythingLLM is one of the few local AI apps that installs like a regular program and can answer questions about your documents right away. The developers describe the desktop version as a single-user app for Mac, Windows, and Linux with local models, RAG, and agent skills "with zero setup."

The "zero setup" claim is nearly honest. There's a ceiling to the format too, and it's not where people usually go looking for it.

How it's built

The central concept in AnythingLLM is a workspace. You create a workspace, drop documents into it, and the app chunks them, computes embeddings, and stores them in a vector database. From there, your question gets matched against that database, the retrieved chunks get fed to the model as context, and it answers with a source citation.

That's classic RAG, and it's the main reason to install AnythingLLM. How this mechanism works under the hood is covered in the article on local RAG.

The app doesn't lock you into one model: any provider connects, local or cloud. The embedder and vector database are built in, so nothing extra needs installing to get started.

What the desktop version includes

The feature set is wider than you'd expect from a single-user app. Documents stay on the device, any vector database and any LLM connect, the embedder is built in. On top: agentic skills — knowledge-base search, working with web pages, building charts, SQL queries, saving files. There's speech transcription through Whisper, third-party data connectors, and an API for your own integrations.

MCP is supported too. The config lives in an anythingllm_mcp_servers.json file inside the plugins folder within the app's storage, and gets created automatically on first use. There's a server management panel inside: start, stop, restart, view the error log. What MCP is and why it matters is covered in a separate breakdown.

What Desktop doesn't have

The developers list plainly what's server-version only:

multiple users;
user management and workspace-level permissions;
password protection;
embeddable chat widgets for a website.

The logic makes sense: Desktop is deliberately built single-user. For personal work, that's simplification, not a limitation. But if you were planning to install it on a work machine and give colleagues access, Desktop won't do it — you need the server version in Docker, and that's a different story involving administration.

About privacy: what it actually depends on

The "full privacy" claim in the description refers to the architecture: documents and the vector database live on your machine. That's true.

But the privacy of the whole setup isn't determined by that alone. AnythingLLM lets you connect any model — and if you pick a cloud provider, document content goes out with the request. The same applies to agentic skills: parsing a website and web search reach outward by definition, and third-party connectors talk to whatever services they're hooked up to.

◆

What makes an app local isn't its name — it's the sum of the choices you made: a local model, a local embedder, and skills that reach the network turned off. Worth checking the whole chain, not the "works locally" checkbox.

This isn't a knock on AnythingLLM specifically — every tool in this class works this way, including LM Studio and desktop agents. Just worth understanding exactly what you're trading for peace of mind.

Where the format hits its edge

AnythingLLM answers "what's written in my documents." It answers well: citations, sources, several workspaces for different projects, decent speed on a local model.

Past that starts a different class of task, and this is where the app's center of gravity works against you. Everything in AnythingLLM revolves around the knowledge base: for the agent to know something, it first has to be uploaded into a workspace and indexed. Tasks like "go through the project folder, find outdated files, and rewrite them," "run the tests and fix what broke," "pull together a summary from exports every Monday" fall outside this model. They're about the machine, not about a knowledge base.

Doka starts exactly where RAG stops. It's not a chat on top of an index — it's an agent that works with the filesystem and terminal directly: reads and changes files, runs commands, carries a task through several steps, and can run on a schedule. Documents don't need to be pre-uploaded anywhere — the agent opens them where they live.

What to pick

If the task sounds like "I have two hundred PDFs, I want to ask questions about them" — AnythingLLM Desktop is a good choice, and it installs in ten minutes.

If the task sounds like "I want to hand off work, not ask a question" — a RAG app needs scripts bolted onto it, and it's simpler to take a tool built from the start to be an executor. Doka works with documents and spreadsheets too, but doesn't require loading them into an index to get started. Download it for free and compare it on your own folder of files — more honest than any feature-list comparison.

AnythingLLM Desktop: What It Does and Where Chatting Over Documents Stops