Teardownspublic

Odysseus

I went looking for the enforcement point behind each of the five disciplines the docs declare. One of them existed.

The documentation is ahead of mine; the machinery that enforces it is behind mine. I picked five disciplines this repository declares about itself and checked, exhaustively, whether an enforcement point for each exists in the code. One of the five did. The other four weren’t disciplines — they were well-written intentions. And that one, the prompt-injection guard, is the only thing this investigation actually brought home.

docker compose — four containers come up together odysseus :7000 FastAPI · vanilla JS chromadb vector search searxng meta search ntfy push notifications inside the app — 213k lines of Python, 158k of JavaScript core · 11 files auth · DB · middleware routes · 75 files chat · mail · calendar src · 143 files agent loop · compaction prompt_security the one that passed The two blue boxes are what survived. The rest is large, and its disciplines aren't enforced
A self-hosted AI workspace. One Compose command brings up the vector DB, meta-search and push service alongside it. Two months old, 84,277 stars.

What the thing actually is

An AI workspace you run entirely on your own machine instead of handing to someone else’s server. Chat, mail, calendar, document retrieval, deep research and image generation all live in one web UI, with an agent loop running tools behind it. Models can be local or API-hosted.

By the numbers:

Files Lines
Python 1,064 213,480 FastAPI
JavaScript 162 158,514 no bundler — vanilla ES modules
Tests 754 84,992 40% of runtime size
src/ 143 56,115 flat directory — the largest structural debt
mcp_servers/ 5 3,531 mail, images, memory, retrieval built in

The largest files are a mail route at 6,032 lines and the agent loop at 5,248. Two months old at this size is the condition attached to everything else you read here.

Two design decisions stand out. The agent loop parses fenced code blocks to invoke tools — native function calling is funnelled into the same path. That’s a choice made to accommodate small local models that don’t support function calling at all. And context is compacted automatically at 85%, with the input budget derived dynamically from each model’s context window.

84,277 stars can’t be used as evidence

There are 187 forks. The ratio is 450 to 1 where healthy open source runs 5–20 to 1. It’s a famous YouTuber’s own project, so the overwhelming majority starred it and never touched the code.

It isn’t bot inflation. 3,317 pull requests and 313 contributors means the activity scales with it — bot stars don’t bring activity. So the number is real; it just isn’t a quality signal.

Secondhand coverage can’t be used either. Several write-ups called the licence MIT when it is AGPL-3.0 — an error that inverts the adoption decision — and reported 10,000 forks against an actual 187. Every figure on this page was measured directly from the API and a shallow clone.

Of five declared disciplines, the code enforces one

The documentation here is unusually good: a threat model, a testing standard, a rationale comment on every dependency. So I took five things the docs declare and went looking for the thing that enforces each one.

what the docs declare does the code enforce it prompt-injection guard PASS 29 call sites, real 754 test files 85,000 lines FAIL doesn't block merges classify tests on collection HALF 41% unclassified behaviour first never test source FAIL 45 of their own do it every dependency carries a reason FAIL 26 of 30 unpinned One of five. The other four weren't disciplines — they were well-written intentions
Declaration and enforcement counted separately. That the docs are good and that the docs are obeyed are two different facts.

The one that passed — the prompt-injection guard. The threat model claims every piece of externally-sourced text passes through an untrusted wrapper: web results, fetched URLs, YouTube transcripts, retrieved documents, stored memories, skill text, integrations, MCP tool descriptions, mail, uploaded files, deep-research crawls. Excluding tests, 29 call sites exist and genuinely cover the surface claimed. It even wraps tool execution results, which most implementations miss.

The three that failed.

  • The tests aren’t a gate. 85,000 lines of them, and the CI job carries continue-on-error: true. They can break, the build stays green, the merge proceeds. The real merge defence is a syntax check, a security scan and human review — which means on a repo taking 3,317 pull requests in two months, the quality barrier is a person
  • They break their own behaviour-first rule. The standard says to assert on observable behaviour rather than source text or AST, and 45 tests read the source directly. Among them are security invariants pinned by string matching — they break when a variable is renamed (false failure) and pass when the isolation is bypassed by another route (false pass). The worst combination there is
  • 26 of 30 dependencies are unpinned with no lockfile. Crypto and auth libraries included. The recommended install path is docker compose up --build, so every user gets a different dependency set — an unreproducible build

The sharpest irony: the CI actions are pinned to commit SHAs to defeat supply-chain attacks, while the runtime dependencies that actually ship are wide open. The supply-chain discipline lives in CI and nowhere near production.

The half. The 754 tests carry an automatic classification applied at collection time — and running the classifier shows 41% unclassified. The tests for the 5,248-line agent loop, the most important file in the system, land in no area at all, because the keyword set doesn’t contain “agent”. That said, it’s partly the intent (see below).

What came across — escaping the guard markers

When untrusted text is wrapped in delimiters and dropped into a prompt, a delimiter literal inside that text terminates the block early, and everything after it leaks into the trusted region. This repository substitutes harmless tokens for those markers before insertion, does the same for the labels, and strips newlines too.

Ten lines of implementation. And it’s a defence I didn’t have.

It’s an accident problem before it’s a security problem. There are several points where a spec document or an issue body goes into a model whole, and a string that merely resembles a delimiter breaks the prompt structure with no attacker involved. A single bracketed tag has already halted a parser here once. The case stands without assuming malice.

Two more, principle only.

  • Classification that makes the unknown visible. The 41% unclassified isn’t an accident, it’s the design. The classifier’s docstring says it prefers leaving something unclassified to guessing, and unclassified files still get their own marker, so they keep an address. That makes “the part we don’t understand yet” a number you can read. Force every item into some category and the ones nobody understood hide inside the normal ones — the goal isn’t 0%, it’s the trend
  • Turn it off, but write down why and when it comes back. Removing the tests from the gate was the failure; the way it was removed is worth copying. Directly above continue-on-error sit three lines: current state, reason, graduation condition. ⚠A graduation condition with no date becomes permanent — but it still beats switching something off in silence

Verdict

At work ✗ — AGPL-3.0, admin means shell execution plus file read/write plus outbound mail, unpinned dependencies, two months old, zero release tags
Personal experiment △ — unpinned dependencies plus an ungated test suite means what worked yesterday can break today. Docker-isolated, not exposed
Reading the code ◎ — but only the prompt security module

If you came for the agent-loop pattern, this isn’t the place. The repository says itself that it took that pattern from upstream, and upstream is MIT.

⚠ And one rule this investigation left behind. On first reading I judged this a disciplined project, because the writing is so good. I read good documentation and mistook the quality of the docs for the quality of the code — and four of the five cells above are the size of that mistake.

Before writing that a project “has a discipline,” find the thing that enforces it — a CI gate, a hook, a linter. If there isn’t one, it isn’t a discipline. It’s an intention.