Mazda's memory self-editing works — I gave her a self-improvement task and she applied four precise, verified edits to her own instructions, unprompted on the mechanics. Separately, two accurate, code-grounded knowledge blocks appeared in her memory during the same session, referencing a file that did not exist minutes earlier. That is strong evidence of grounded self-documentation. What is not yet true is that the formal, gated agent_self_improvement control loop (run → judge → propose → A/B → gate → activate → rollback) is wired to the live agent. The framework exists in code; the live agent is not yet plugged into it. This document proves the first claim, is honest about the second, and lays out the path from one to the other.
“Self-improvement” is two different things in this project, and conflating them is the most common error. Keep them apart:
| Capability | What it is | Status |
|---|---|---|
| Memory self-editing | Mazda rewriting her own system/* instruction blocks with her memory_insert / memory_replace tools. | LIVE — proven §2 |
| Grounded self-documentation | Newly-authored memory that accurately reflects real, current code (not hallucinated). | OBSERVED §3 |
| The gated improvement loop | The agent_self_improvement framework: trace → judge → propose → experiment → gate → activate, with snapshot rollback. | BUILT, NOT WIRED §4 |
The thesis the framework is built on (from the Mazda manual): we improve the wrapper — system messages, prompts, tool descriptions, memory notes — never the model. Every claim below is about the wrapper.
Tier A — reproduced and verified this session.
I sent Mazda a single message: “update your own long-term memory to finish your training… make exactly these edits using your memory_insert / memory_replace tools.” She executed four tool calls, all returning status=success, then summarised what she changed — entirely on her own initiative for the mechanics.
message_type counts: {tool_call_message: 4, tool_return_message: 4, assistant_message: 1}
call 1: memory_insert label=system/verification_procedure -> success
call 2: memory_insert label=system/team_agents -> success
call 3: memory_replace label=system/persona -> success
call 4: memory_replace label=system/persona -> success
ASSISTANT: "Changed:
- Inserted the top-level document-processing & categorization workflow map at
the top of system/verification_procedure.
- Appended cost-discipline guidance and the document-processing specialists
table to system/team_agents.
- Replaced the system/persona primary task with the broader full-workflow version.
- Added the three new operating rules to system/persona's ." These are her real tools — note that memory_replace is a precise find-and-replace, not a blunt overwrite, which is why the edits are safe:
memory_replace(label, old_string, new_string)
"replace a specific string in a memory block with a new string …
Do NOT attempt to replace the entire contents of a memory block."
memory_insert(label, new_string, insert_line=-1)
"insert text at a specific location in a memory block."I did not take her word for it. I read the blocks back from the API and re-ran the authoritative memory check (the recompile projection count):
=== content markers present in live blocks ===
system/persona OK 'full ROL document-processing' OK 'cheapest, fastest reliable'
OK 'confidence is below 90%' OK 'memory_insert / memory_replace'
system/team_agents OK 'Cost discipline' OK 'document-parser-agent' OK 'RECOMMEND creating'
system/verification_procedure OK 'TOP-LEVEL WORKFLOW MAP' OK 'mazda_intake.py' OK 'confidence < 90%'
=== recompile projection (rendered system/ files) === 7 → 8 files, all render
The self-editing mechanism is real, controllable, and verifiable. An instruction to “improve yourself” produced correct, durable, rendered changes to the live wrapper. This is the foundational primitive every higher-order loop depends on.
Tier B — observed in the wild; I am deliberately not overclaiming the cause.
During the same session, two blocks I did not author appeared in Mazda's memory: system/intake_pipeline (7.3 KB) and skills/scan-to-report (7.7 KB). What makes them remarkable:
tools/mazda_intake.py as an established “team façade” — a file that did not exist until I created it ~30 minutes earlier in the same session.tools/mazda_intake.py, tools/self_improving_agent/mazda_run_team.py, tools/categorizer/find_category/, tools/categorizer/vendor_category.yaml, resolve_vendor_key_with_fallback.py, parse_receipt_cli.py.scan-to-report even adds a safety rule I never specified: “do not invent transactions, totals, or database matches.”PREFERRED ENTRYPOINT — use the team's facade, not the raw router
→ python3 tools/mazda_intake.py <path> --org-id=1 [--enable-parse]
recommended_action maps confidence to your next move (same 0.90 gate as Step 7):
"auto" (conf ≥0.90) → proceed
"review" (0.70–0.89) → INVOLVE THE USER before trusting it
"reject" (<0.70) → do not use; re-scan / ask the userI cannot name the exact actor/step that wrote these two blocks. The facts that constrain the explanation: enable_sleeptime on the live Mazda is None (so it was not an automated reflection agent); my four tool calls did not create them; yet they were absent at first recompile and present after. The most likely explanation is an autonomous or teammate-driven authoring step within the session window. The point that survives the uncertainty: the content is accurate and freshly grounded in code that had just changed — which is exactly the behaviour a working self-improvement system should produce, and exactly the behaviour a hallucinating one would not.
Tier C — built in code, GoF-clean, not yet connected to the running Mazda.
The agent_self_improvement package is a genuine, test-covered implementation of the improvement loop, built on textbook Gang-of-Four patterns. These are real classes, not aspirations:
| GoF pattern | Where it lives | Role in the loop |
|---|---|---|
| Command | improvement/commands.py — PromptPatchCommand, ToolDescriptionPatchCommand, MemoryNoteCommand, ContextRulePatchCommand (all _ApprovalGatedCommand) | A wrapper edit reified as an inspectable, reversible object — the unit the gates and rollback act on. |
| Strategy | evaluation/verdict.py — ILlmJudgeStrategy, DeferToHumanLlmJudge, used by FinanceVerdictJudge | Deterministic rules decide; the LLM judge is consulted only for genuine ambiguity. |
| Abstract Factory | contracts/factories.py — ten I*Factory families | Swap a whole backend family (persistence, evaluation, improvement…) without touching the kernel. |
| Chain of Responsibility | evaluation/gates.py — SafetyGate → CostGate → RegressionGate → UsefulnessGate via GateChain | A candidate edit must clear every gate before it is eligible for activation. |
| Memento | sqlite_persistence/snapshot_store.py, rollback_service.py | Snapshot the wrapper before activation; restore a known-good revision by id. |
Mazda's own verification_procedure block instructs her to call load_wrapper_revision, record_trace, and propose_improvement. Her actually-attached tools are only: web_search_exa, web_fetch_exa, send_message_to_agent_and_wait_for_reply, send_message_to_agent_async, executor_run, memory_replace, conversation_search, memory_insert. The three loop tools are not attached. So today the live agent can edit her wrapper but cannot trace, be judged, or propose a gated change through the framework. The control plane and the runtime are not yet joined.
🟩 live in code🟨 stub / not wired to Mazda
The difference between 5.1 and 5.2 is the whole project: today we have a verified self-edit; the target is a verified self-edit that was judged, gated, and is reversible.
Five phases, each independently valuable, ordered so every step is testable before the next.
record_trace, load_wrapper_revision, and propose_improvement as Letta tools on Mazda (her memory already references them). Back them with the existing SQLite evidence store via executor_run or a small MCP, mirroring the finance_verifiers_mcp pattern.TraceRecord row and can read back its active wrapper-revision ids.FinanceVerdictJudge (rules-first; ILlmJudgeStrategy only on ambiguity). Persist a VerdictRecord.FailureType.MemoryNoteCommand so they are snapshotted (Memento) and reversible, and must clear the GateChain first. This upgrades today's raw memory_insert into a gated, rollback-safe change.ExperimentRunner (currently a stub) to run baseline-vs-candidate wrappers over a fixed receipt fixture set and feed the RegressionGate.None, or a scheduled propose_improvement over recent FAIL traces) so the Tier-B behaviour from §3 becomes intentional and audited rather than incidental.Mazda runs a finance task, it is traced and judged, a failure pattern yields a gated, snapshotted wrapper proposal, the proposal is A/B-tested against regressions, and a human (E.G.) approves activation with one click — with rollback always one id away. At that point §2's primitive and §4's framework are the same system, and the Tier-B magic of §3 is something we can schedule, not just witness.
Is the self-improvement system working? The self-improvement primitive — a live agent reliably and verifiably rewriting her own wrapper, plus producing accurate, freshly-grounded self-documentation — is working today (§2, §3). The self-improvement system in its full, gated, reversible form is built and clean (§4) but not yet connected to the running agent. The gap is small, well-understood, and bridgeable in five phases (§6). We are not claiming a closed loop; we are claiming a working primitive and a clear, short path to closing the loop.