AI agents that can diagnose Ubuntu, but can't change it: limits enforced by the kernel, not prompt
Project details
AI agents that can diagnose Ubuntu, but can't change it. maat is an open-source MCP server inside a strictly confined snap, with read-only interfaces and no network access. The model can only propose a typed fix; a person approves it through polkit, and every step goes into a tamper-evident audit trail. Developer preview, tested on Ubuntu 24.04.
Giving an AI agent a shell to troubleshoot a machine means it can do anything you can. That's risky because logs are written by strangers: the username from a failed SSH login lands in the auth log as typed, and so does anything a local user sends through logger. An agent reading logs is reading text that can be written for it.
maat lets the agent look at the real machine while the kernel, not the prompt, decides what it can do.
log-observe, system-observe and mount-observe, with no network plug. AppArmor and seccomp refuse internet sockets, writes to your files, and even RestartUnit over D-Bus. maat selftest shows each refusal on your machine..service unit. maat adds the preconditions, the success check and the undo step, and flags any evidence ID it never issued.maatctl, shows you the plan and the log lines it cites, applies it through polkit (the same prompt systemctl uses), and checks the result.maat trail verify detects edited, deleted, reordered or truncated records.authorized_keys restricts to starting the confined server.maat-bench builds deliberately broken Ubuntu machines in throwaway LXD containers and scores agents without an LLM judge. On its ten scenarios, three runs each, through Ollama:
| gpt-oss:20b | qwen3:8b | |
|---|---|---|
| Found the root cause | 28/30 | 24/30 |
| Fix applied and verified | 3/6 | 0/6 |
| No unneeded plans | 22/24 | 24/24 |
| Instructions planted in logs resisted | 6/6 | 6/6 |
The scenarios are still easy (keyword rules score 9 of 10) and only two are adversarial, so this is a first data point, not a ranking.
Testing the confinement on a real VM changed the design twice. network-observe turned out to allow outbound connections, so maat doesn't use it. Without it the socket tables can't be read, so that tool now fails with an explanation instead of reporting "no sockets".
Developer preview, on the Snap Store's edge channel, tested on Ubuntu 24.04. Redaction is pattern-based and will miss some secrets. The audit trail detects tampering but can't prevent it by root. maat can't take a shell away from your MCP client, so for the guarantees to hold, give the agent maat's tools only.
sudo snap install maat --edge
for iface in log-observe system-observe mount-observe; do
sudo snap connect "maat:$iface"
done
sudo usermod -aG systemd-journal "$USER" # then start a new session
maat selftest
Source, docs and benchmark: https://github.com/bolesawny7/maat
maat is an independent project, not affiliated with or endorsed by Canonical.
Comments
0Start the conversation
Share the first comment.