Server Rewrite

Progress

Operating rules

1. Program to the Interface. Domain services import ports, never concrete adapters.
2. Wire once. Concrete construction belongs in server.py or a dedicated application factory.
3. TDD is mandatory. No production Interface or Object is written before its failing test. Work proceeds Red → Green → Refactor.
4. Characterize before moving. Characterization tests pin public output, errors, caching, locking, and failure behavior before legacy code moves.
5. Types are executable contracts. Python boundaries use strict Pydantic models; new browser code uses strict TypeScript.
6. Keep pure code simple. Pure transforms remain functions; do not create an Interface without a boundary.
7. Side effects require ports. Network, process, filesystem, database, clock, logging, metrics, and notifications are injected.
8. One slice per change. Interface, Object, adapter, tests, wiring, and compatibility shim travel together.
9. No live dependencies in unit tests. Use small fakes that implement the same port contract.
10. Preserve HTTP contracts. Status, headers, JSON shapes, timeouts, and route names stay stable unless separately approved.
11. Measure every slice. Record server.py lines immediately before and after; explain every non-negative delta and distinguish concurrent edits from slice-attributable movement.
12. Delete completed detail. Once a row meets its exit test, remove it from this ledger. Git is the completion history.

Strict typed contracts and test-first development

Python contract policy

ABCs describe behavior; Pydantic models describe data. Commands, results, API requests/responses, repository records, configuration, events, and adapter payloads use named models derived from one strict base:

from pydantic import BaseModel, ConfigDict

class StrictModel(BaseModel):
    model_config = ConfigDict(strict=True, extra="forbid", frozen=True)

• No unvalidated dict[str, Any] may cross a port. Validate once at the outer adapter, then pass typed models inward.
• No implicit coercion at a boundary: "1" is not accepted where an integer is required.
• Unknown fields fail closed through extra="forbid"; immutable DTOs prevent hidden mutation.
• Discriminated unions model variant results such as up/down/starting or receipt/statement/unknown.
• Every public function and method in a rewritten module has complete parameter and return annotations.
• Any, unparameterized containers, and blanket type ignores require a written decision-log exception.

Static checking runs with pyright --project dashboard/pyrightconfig.json, where typeCheckingMode = "strict". Runtime Pydantic validation complements static checking; neither replaces the other.

TypeScript contract policy

• All new frontend code is TypeScript (.ts), with strict, noUncheckedIndexedAccess, exactOptionalPropertyTypes, and useUnknownInCatchVariables.
• Existing JavaScript may remain only as legacy. A touched module is migrated or placed under // @ts-check with complete JSDoc types until migration is practical.
• No blind type assertions (as Foo) for HTTP JSON. A type guard or runtime schema validates unknown input first.
• Discriminated unions replace stringly typed status objects and scattered property checks.
• bunx tsc --noEmit -p dashboard/tsconfig.json must pass before tests and deployment.
• The TypeScript build emits browser JavaScript deterministically; generated output is never hand-edited.

TDD order for every ledger row

1. RED: write the consumer behavior test and run it. Record the expected failure; import/contract absence counts as red for a new extraction.
2. Add Characterization tests around the legacy seam so the behavior being preserved is explicit.
3. Add Shared contract tests that every concrete adapter and fake must pass.
4. Add Negative validation tests proving malformed types, missing fields, extra fields, and invalid variants are rejected.
5. GREEN: add only enough typed Interface/Object code to make the focused tests pass.
6. REFACTOR: remove duplication and improve names while the tests and type checkers stay green.
7. Run the focused suite, complete Python suite, complete browser suite, static type checks, and coverage gate.

Required test layers

Target architecture

dashboard/
  server.py                 # construct adapters, start threads/server, compatibility exports
  http/                     # route registry, controllers, HTTP/WebSocket adapters
  agents/                   # Letta gateway, directory, messaging, agent health
  server_management/        # health probes, logs, SSH, restart orchestration
  model_usage/              # usage sources, history, provider health/failover
  intake/                   # scanners, classification, staging, dispatch, Trainer
  finance/                  # expense/receipt/report repositories and services
  voice/                    # existing transcription/cleanup pipeline
  tests/                    # mirrors the production domains above

The directory names are targets, not a mandate to move everything at once. A slice may begin as one flat module when that keeps the change reviewable.

Definition of done for the rewrite

• server.py is startup and dependency wiring, ideally below 500 lines.
• DashboardHandler adapts HTTP to registered controllers and contains no business rules.
• Every concrete external dependency can be replaced by a test fake through constructor injection.
• All Python boundary data is strict Pydantic; all new frontend modules are strict TypeScript.
• Pyright and TypeScript complete with zero errors and no unexplained suppressions.
• test_server.py contains only composition/compatibility tests; domain tests live beside their domains.
• The complete Python and JavaScript dashboard suites pass without contacting live infrastructure.

Active Interface/Object ledger

Each work row is one reviewable extraction. The checkbox is deliberately visual rather than browser-persistent: completion is shared by deleting the row in source control, not by storing private state in one browser.

Established examples — do not move backward

These are outside the active count because they already demonstrate the intended shape.

One-slice workflow

1. Choose exactly one active row; record its current callers and mutable globals.
2. Write the failing consumer test first, run it, and save the RED command plus failure.
3. Add characterization, shared-contract, and negative-validation tests before production code.
4. Define the smallest fully typed ABC and strict Pydantic models in the owning domain. The abstract module imports no concrete adapter.
5. Implement only enough typed behavior for GREEN, including a minimal fake that passes the same contract tests.
6. Inject the port into the service/controller; construct the concrete only at the composition root.
7. Leave a temporary forwarding function in server.py when compatibility requires it.
8. Move relevant tests from test_server.py into the matching domain test file.
9. Refactor with tests green, then run Pyright, TypeScript, focused tests, full suites, coverage, and the relevant manual smoke check.
10. Record current line counts, delete this work row, and commit the shrinking plan with the implementation.

Required evidence in each handoff or PR

Slice:
Observed RED:
RED command and failure:
Interface:
Pydantic model(s):
TypeScript type(s):
Concrete Object(s):
Callers migrated:
Compatibility shim:
Focused tests:
Contract/negative tests:
Pyright result:
TypeScript result:
Branch coverage:
Full Python result:
Full JavaScript result:
server.py lines before/after:
test_server.py lines before/after:
Ledger row removed:

Decision log

2026-07-29

Use an incremental ports-and-adapters rewrite; do not replace the stdlib HTTP server or change routes as part of extraction.

2026-07-29

Interfaces are required at meaningful boundaries and all cross-cutting side effects. Pure transforms remain functions.

2026-07-29

Completed work rows are deleted instead of archived here, so this page becomes smaller as the rewrite advances.

2026-07-29

Begin with the green test baseline, then model usage as the pilot production slice.

2026-07-29

Adopt strict Pydantic v2 models for every Python boundary and strict TypeScript for all new frontend work; static and runtime validation are both required.

2026-07-29

Adopt strict TDD: witness RED before production code, reach GREEN minimally, then refactor; new domain/application modules require 100% branch coverage.

2026-07-30

Completed the model-read sub-slice of ILettaGateway: RED was pytest -q tests/test_letta_gateway.py failing because the agents boundary did not exist; GREEN was 14 focused gateway/model-option tests. Added strict frozen Pydantic contracts, FakeLettaGateway, UrllibLettaGateway, composition-root wiring, and a temporary agent_model_payload forwarding shim. The ledger row remains because messages, thoughts, tool calls, and write paths are not migrated.

2026-07-31

Made server.py size a mandatory rewrite metric. Historical baseline: 10,486 lines. The pre-model-slice working-file backup was 10,871 lines; counting only the model-read extraction hunks gives 10,871 → 10,877 (+6, +0.06%) because composition-root wiring and compatibility shims were added before all legacy paths could be deleted. The current working file is 10,962 lines (+476, +4.54% versus baseline); unrelated concurrent dashboard work accounts for the remaining +85 lines in the observed +91 since the backup. Future slices must capture immediate before/after counts and report observed and slice-attributable deltas separately.

2026-08-26

Fixed the same-receipt rename boundary: RecentReportImageSynchronizer now preserves the vendor/date identity already stamped on the archived receipt, changes only the aggregate amount, and updates every associated expense's receipt_url/source_file. Last Scanner pages also reconstruct Archive Verification on mount instead of showing it only momentarily after Save All. The selected slice added 31 composition-root lines; concurrent rewrite work reduced other imports/wrappers while this fix was in progress, so the observed working file is 7,532 lines. Current count versus the 10,486 historical baseline is -2,954 lines (-28.17%). The non-negative attributable delta is the injected database-reference adapter; filename identity, aggregation, and rename behavior remain in finance/recent_report_image.py. Next deletion target: move that adapter behind the existing expense port and remove the server-local compatibility function.