Why agents should run on your own box

Cloud AI made agents easy to try. Local-first AI makes them practical to trust.

An agent that can help with real work needs access to the same workspace you use every day: source code, notes, tickets, logs, design drafts, and the small pieces of context that never fit cleanly into a prompt. That context is powerful. It is also sensitive. The closer an agent gets to the work, the more important it becomes to decide where the agent actually runs.

Running agents on your own machine changes the trust model. The agent can work inside the boundary you already use for development, research, writing, and operations. Files do not need to be copied into a hosted workspace just so a model can summarize them. Local tools can be called through the same permissions you apply to the rest of your environment. The default posture becomes: keep the work here, then selectively call out when there is a clear reason.

That does not mean every model must run locally. The useful split is simpler: the agent runtime belongs close to your files, while model calls can be routed according to the job. A small local model might classify notes or draft a command. A frontier hosted model might reason through a hard refactor. The important part is that the agent is not a remote black box with permanent access to your workspace.

type ModelRoute =
  | { kind: "local"; model: "small-coder" }
  | { kind: "hosted"; model: "frontier-reasoner"; purpose: string };

type AgentBoundary = {
  workspaceRoot: string;
  allowedTools: string[];
  routeForTask(task: string): ModelRoute;
};

Local agents also make memory more useful. A hosted chat can remember preferences in a general way, but a local agent can keep project-specific memory next to the project itself. It can see the difference between a personal writing vault, a production app, and an internal tool. It can apply different rules without turning every prompt into a policy document.

This matters most when the work is incremental. Agents are rarely valuable because they produce one perfect answer. They are valuable because they can inspect, edit, test, revise, and carry context across a chain of small decisions. That loop is much easier to govern when it happens on your box.

There is also a practical speed advantage. Local agents do not need to re-upload the same files over and over. They can index the workspace, search it quickly, and hand the model only the context required for the next step. That keeps prompts smaller, reduces latency, and makes the agent less dependent on a single provider’s context window.

The real question is not whether AI should be local or cloud. The question is where the authority lives. If the agent can run commands, edit files, and remember project state, that authority should live where you can inspect it, revoke it, and back it up.

Your own box is already the place where work becomes real. Agents should start there too.