Architecture

Text / voice (GPT-Live) → AppModel → CommandParser (OpenAI) → TaskOrchestrator → AgentAdapter (mock | ACP)
                             │                                                        │
                             └──────────────────── task events ←──────────────────────┘
                             │
                    BlobStateMachine → blobs (Metal shader)

1. Input. A typed command or a voice delegation from GPT-Live goes into AppModel.handleCommand — the same path for both.
2. Parsing. ModelCommandParser sends the rule text (commands.md) plus the scene as JSON to the model and gets back the intent, the project ids, the conversation and a clean task text, constrained by a JSON schema.
3. Orchestration. TaskOrchestrator (one for the app) keeps per-project conversations and queues, creates git worktrees, applies approval rules and restores tasks after a restart.
4. Agents. ACPAgentAdapter talks JSON-RPC over stdio to claude-agent-acp / codex-acp: one process per agent kind, one ACP session per conversation, with the folders and MCP servers of the project.
5. Feedback. Task events drive the blob states, the result panel and VoiceEvents the voice retells.

Folders

Path	What
blu/App	AppModel, the app entry, settings, AppIdentity
blu/Commands	command parsing, the decision model, the model catalog
blu/Prompts	built-in rule texts and PromptLibrary (user edits, three-way merge)
blu/Orchestration	TaskOrchestrator, GitWorktrees
blu/Agents, blu/Agents/ACP	the adapter protocol, ACP client, approvals mapping
blu/Models	Project, AgentTask, BlobColor, states
blu/Persistence	SQLiteData (GRDB), schema by CloudKit rules, sync
blu/Voice	GPTLiveVoiceSession, audio, voice prompt and events
blu/Floating, blu/UI, blu/Shaders	floating windows, panels, editors, the Metal blob shader

Principles

• Understanding text is the model’s job. No keyword lists, regexes or “ends with ?” checks over what people or agents say; the raw text goes to a model as data, and the rule goes into a prompt.
• No natural-language instructions in code. Only the rule texts reach the models; everything else is JSON.
• General rules only. Prompts never grow special cases for a test scenario.
• Decisions are recorded in docs/DECISIONS.md, plans in docs/ROADMAP.md.