Multi-Agent LLM Orchestration Hub: Roles, Budgets, and Action Control

Context

Multi-agent systems are one of the most talked-about topics in applied AI — and one of the most under-engineered. A demo where several LLM agents "negotiate" with each other can be assembled in an evening; a production loop you can trust with real operational workload cannot. The gap runs along three lines: governance (who decides which agents join a task, and how), economics (every model call costs money, and with a dozen agents running in parallel, spend grows non-linearly), and action safety (an agent that can only write text is harmless; an agent that edits files or places phone calls is not).

Provider heterogeneity adds another layer of difficulty. No single model is optimal for every role at once: reasoning tasks, fast classification, code generation, and result synthesis each gravitate toward different models with different token prices. A mature multi-agent system therefore has to run across several providers simultaneously — and their APIs differ in formats, rate limits, and failure modes. Abstracting that layer without losing cost control is an engineering problem in its own right.

The studio runs many parallel production tracks — web development, backends, bots, video pipelines, prepress. Each reduces to the same cycle: decompose the task, distribute it to executors, supervise, assemble the result. That made our own operational workload the ideal proving ground: building orchestration not on synthetic scenarios but on tasks backed by real deadlines and real money. The resulting system is both an internal tool and a reference for how we design multi-agent loops for clients.

The Task

The hypothesis was stated plainly: can the human be left with only two roles — task author and final arbiter — while the routine of planning and execution is handed to a team of LLM agents. The operator states one task in natural language; the system itself decides which specialists to involve, in what order, and how to merge their outputs into a single coherent result.

The second requirement was full observability. A run where you can see neither the plan, nor intermediate outputs, nor the price is unusable in production: an error cannot be localized, and spend cannot be justified. At every step, the orchestrator's plan, each agent's output, and the total dollar cost of the run had to be visible.

The third requirement was budget discipline from day one. The moment several paid providers join the workload, costs slip out of control easily. Money accounting had to be built into the LLM layer at the architecture level, not bolted on afterward: every call metered, cumulative daily spend capped by a hard limit, and any breach blocking intake of new tasks.

Approach

The orchestration core is a three-phase scheme: plan → dispatch → synthesize. In the plan phase, the Claude-based orchestrator reads the task and breaks it into role-bound subtasks. In dispatch, each subtask goes to its worker through the appropriate provider. In synthesize, the orchestrator assembles all agent outputs into a single final result. The scheme is deliberately simple: three phases are easy to observe, debug, and explain — unlike free-form "agent conversations" where accountability for the result dissolves.

The team roster is described declaratively in agents.json: a Claude Sonnet orchestrator plus workers in distinct roles — researcher, coder, analyst, critic, writer, fast, and others. Adding or reassigning a specialist is a one-file edit, not a code change. Each agent is bound to a specific provider and model, which lets expensive and cheap calls be split by role: reasoning roles get strong models, fast auxiliary roles get cheap ones.

A deliberate engineering constraint: the entire system is written on the Python standard library, with zero external dependencies. The unified client for four providers is built on urllib. This makes installation trivial, keeps the failure surface minimal, and makes behavior predictable: nothing breaks when third-party packages update, and all of the code can be read end to end without excavating someone else's abstractions.

Architecture

The model-access layer is the llm.py module — a single client for four providers: OpenRouter, Groq, Gemini, and Anthropic. It hides their API differences behind one interface, so the orchestrator does not care which channel a call travels through — routing is defined by the roster, not the code. Three providers are currently live — OpenRouter, Anthropic, and Groq; the direct Gemini key is invalid, but Gemini models remain reachable through OpenRouter, so model coverage is preserved. This illustrates a principle: the system must never depend on a single access channel to any model.

The orchestration layer is the orchestrator.py module with a run_task() function that writes execution progress to tasks/<id>.json files in real time. Run state lives on disk, not in process memory: the dashboard simply reads task files, orchestration survives server restarts, and run history stays available for audit. File-based state is a deliberate choice in favor of simplicity and recoverability: no message queue, no external database, not a single moving part that can fail independently of the system itself.

The observation layer is server.py — the Mission Control web dashboard on port 8772. It shows the roster with live statuses, a task submission form, and a run feed: the plan, each agent's outputs, the final result, and the cost. Submission goes through POST /api/task; execution is offloaded to a dedicated dispatcher thread, so the interface stays responsive during long runs. The style is clean white, matching the studio's internal tooling. A parallel control channel runs through Telegram: the /team <task> command triggers orchestrator.run_task and streams progress straight into the chat, so submission and supervision are available from a phone.

The secrets layer is kept separate: all provider keys are consolidated into a single protected file with chmod 600 permissions by a dedicated consolidator script. Secrets are not scattered across projects and per-service environment variables — they sit in one place with correct access permissions, which simplifies both key rotation and auditing which services have access to what.

Execution Loop and Budget Discipline

Beyond the reasoning workers, the system has an execution loop — the doer agent running through the claude_code provider. It launches the Claude Code CLI in bypass mode and performs real actions: editing files, working in the terminal and on the web. This has been verified on live operations — the agent created a file, opened a website in the browser on its own, and returned a report on the outcome. This is where the system crosses from text reasoning into actions with consequences, and precisely why the execution loop is surrounded by tighter safeguards than the rest.

Cost accounting is built into the LLM layer, not into reporting: llm.LAST records the cost of every call. For token-priced providers the price comes from the PRICES table; for claude_code it is the actual total_cost_usd reported by the tool itself. The dashboard surfaces this as a "Spent today" KPI and a per-task cost. The hard safeguard is the daily limit DAILY_LIMIT=$10: once exceeded, the POST request for a new task is rejected. Budget discipline is implemented as a property of the system, not an agreement with the operator.

The roster has grown to 19 agents — the orchestrator and 18 workers, of which 15 are ready and 3 are not yet connected. The ready flag filters the team at two levels: non-ready agents are excluded from planning, and run_worker rejects any call to a non-ready agent. This lets working roles and declared-but-unfinished integrations — db/Supabase, video/HeyGen, caller/Zvonok — coexist in the system, honestly shown in the interface as "not connected" and unable to break a run. A separate category is the specialist agents: artist, running through the kie provider, generates images and participates in work, while the real-call agent caller is deliberately non-autonomous — every individual call requires human confirmation.

The team's knowledge is systematized in a skills library: a dedicated generator produced 7 skills for the studio's domain pipelines — fastapi-supabase, telegram-bot, static-site-build, video-clip-pipeline, print-prepress, ai-profiling, realty-sourcing. Together with the base library, the system has accumulated about 87 skills — a shared toolkit from which agents take proven, ready procedures instead of deriving them from scratch on every run.

Outcome

Version v2 runs end-to-end. The operator submits a task from the dashboard or Telegram, the orchestrator breaks it into a plan, workers execute subtasks through the live providers, the execution loop performs real actions when needed, and the final result is synthesized into one coherent output. Every step is visible in the feed, every call is metered in dollars, every run is recoverable from the file-based journal.

All key control loops are operational: the daily spend limit blocks task intake once the budget is exceeded, the ready filter isolates unfinished integrations from live runs, and irreversible actions — phone calls — do not execute without human confirmation. This is what separates the system from demo multi-agent assemblies: the safeguards are built into the architecture, not declared in documentation.

For the studio, Agent OS solves two problems at once. As an internal tool, it removes the routine of decomposition and distribution from our own operational workload. As a reference architecture, it fixes the standard by which we design multi-agent loops for clients: multiple providers behind a single interface, observability of every step, the cost of every call, hard budget boundaries, and a human in the loop wherever actions are irreversible. It is exactly this set of properties that separates a production AI system from an impressive prototype.

What we built

• Unified LLM client

  The llm.py module on top of urllib hides the differences of four providers behind one interface; a User-Agent header was added for Groq, without which Cloudflare returns an error.

• Plan → dispatch → synthesize orchestrator

  run_task() in orchestrator.py decomposes the task, dispatches subtasks to workers, and synthesizes the final result, writing progress to tasks/<id>.json in real time — state survives server restarts.

• Mission Control dashboard

  server.py on port 8772: roster with live statuses, task submission form, run feed with the plan and each agent's outputs, a POST /api/task endpoint, and a dispatcher thread.

• Declarative roster

  agents.json describes 19 agents — the orchestrator and 18 workers in distinct roles; the ready flag controls who participates in planning, and run_worker rejects calls to non-ready agents.

• claude_code execution loop

  The doer agent launches the Claude Code CLI in bypass mode and genuinely edits files, works in the terminal and on the web; verified by creating a file and visiting a website with a report on the outcome.

• Cost accounting and daily limit

  llm.LAST records the cost of every call (the PRICES table; actual total_cost_usd for claude_code); the dashboard shows a spend KPI, and the DAILY_LIMIT=$10 cap rejects new tasks once exceeded.

• Telegram control channel

  The /team <task> command triggers the orchestrator and streams execution progress straight into the messenger — submission and supervision are available from a phone.

• Skills library

  A dedicated generator produced 7 domain skills for the studio's pipelines; together with the base library — about 87 skills from which agents take ready procedures.

Engineering challenges