Expense Itemization

Parent / line-item architecture: one charge, many real expenses. Written 2026-07-17 (end of shift).

Why this exists

On 2026-07-17 a Chase statement charge of $60.00 (Amazon order 112-5047119-7910611) was stored as expense 1504 with the generic category 143 "Amazon". The order's real contents — a Kirundo dress ($53.98), mascara ($7.45), and a Roku remote ($19.88) — belong in different categories (clothing, personal care, electronics). One row with one category_id cannot represent that honestly: today we categorize the payment channel, not the expense.

The same flaw applies to multi-item receipts. The $3,047 Children's Vision receipt (expense 1502) contains 9 separate donation line items forced into a single row — and it was initially miscategorized precisely because a single row invites a single lazy vendor-level guess.

Schema (Phase 0 — the only DDL)

ALTER TABLE expenses
  ADD COLUMN parent_expense_id BIGINT UNSIGNED NULL,
  ADD COLUMN expense_role ENUM('STANDALONE','PARENT','LINE_ITEM')
      NOT NULL DEFAULT 'STANDALONE',
  ADD CONSTRAINT fk_expenses_parent
      FOREIGN KEY (parent_expense_id) REFERENCES expenses(id),
  ADD INDEX idx_expenses_parent (parent_expense_id);

Backward compatible: every existing row (≈884 as of tonight, incl. 1504–1506) remains valid as STANDALONE, exactly like the expense_status migration on 2026-07-17. Integrity invariants (enforced in the repository layer, checked by tests):

• LINE_ITEM ⟺ parent_expense_id IS NOT NULL; the parent must have role PARENT.
• Children of a parent must sum exactly to the parent's amount.
• Reports and views sum only expense_role != 'PARENT'.
• A PARENT has category_id NULL (it is not an expense; it is a payment event).
• id_light for children: <parent_id_light>-item-<n> (unique key preserved).

Views needing the expense_role != 'PARENT' predicate (audit each; these are the known summers): vw_monthly_expense_summary, vw_report_financials, vw_large_transactions, vw_balance_by_org, and any vw_*transaction* view that joins expenses.

Architecture — program to the interface

All new code lives behind ABC ports with concretes injected at the composition root (the CLI main() / dashboard boot — never inside the domain logic), matching the established pattern in rol_finances and the dashboard JS. Four ports, one Abstract Factory:

class LineItemSource(ABC):
    """Yields candidate line items for a charge/receipt. Pure read."""
    @abstractmethod
    def line_items(self, charge: ChargeContext) -> list[LineItem]: ...

class ItemizationPolicy(ABC):
    """Decides whether itemization is HONEST for this charge. Fail closed."""
    @abstractmethod
    def evaluate(self, charge: ChargeContext,
                 items: list[LineItem]) -> ItemizationDecision: ...

class ItemCategorizer(ABC):
    """Assigns a category to ONE line item (not to the whole charge)."""
    @abstractmethod
    def categorize(self, item: LineItem,
                   charge: ChargeContext) -> CategoryAssignment: ...

class ItemizedExpenseRepository(ABC):
    """Persists parent + children ATOMICALLY (one transaction), or
       re-parents an existing STANDALONE row into a PARENT + children."""
    @abstractmethod
    def store_itemized(self, parent: ExpenseDraft,
                       children: list[ExpenseDraft]) -> ItemizedStoreResult: ...
    @abstractmethod
    def itemize_existing(self, expense_id: int,
                         children: list[ExpenseDraft]) -> ItemizedStoreResult: ...

class ItemizationFactory(ABC):
    """GoF Abstract Factory: one factory per document family guarantees the
       source/policy/categorizer/repository quartet is mutually consistent.
       Client code (Mazda tools, store scripts) NEVER constructs concretes."""
    @abstractmethod
    def line_item_source(self) -> LineItemSource: ...
    @abstractmethod
    def itemization_policy(self) -> ItemizationPolicy: ...
    @abstractmethod
    def item_categorizer(self) -> ItemCategorizer: ...
    @abstractmethod
    def repository(self) -> ItemizedExpenseRepository: ...

class ReceiptItemizationFactory(ItemizationFactory):
    """Receipts: items come from the parsed receipt itself; policy is
       ExactReconciliationPolicy (receipt lines ALWAYS sum to the total —
       if they don't, the parse is wrong, refuse)."""

class AmazonStatementItemizationFactory(ItemizationFactory):
    """Amazon charges: items from AmazonOrderSpreadsheetSource (wraps
       lookup_amazon_order.py's lookup); policy additionally handles the
       split-shipment case → NOT_ITEMIZABLE unless items reconcile to
       the charge amount exactly."""

# Registry keyed by document family; the ONLY place concretes are wired.
def factory_for(doc_family: str) -> ItemizationFactory: ...

Concrete implementations (first wave)

Phases (do them in order)

Blast radius — audit each before P1 ships

• e_two_e_processing/duplicate_checker.py — statement lines must match against PARENT/STANDALONE only, never LINE_ITEMs (a $19.88 child must not "duplicate" a real $19.88 charge elsewhere).
• store_statement_transactions.py / parse_and_categorize.py --save — route through the repository port; keep the fail-closed behavior added 2026-07-16.
• create_id_light / generate_id_light — child suffix scheme.
• All vw_* views listed above; grep for any raw SUM( over expenses in dashboard/server.py too.
• Baker / receipt-URL baking — children share the parent's receipt_url/source_file.
• New Records categorize flow (searches report.html by date+amount) — same id-based fix as the picker.

Standing constraints (project rules — do not relearn these the hard way)

• Program to the interface; concretes wired only at the composition root. ABC ports + DI in Python, injected controllers in dashboard JS.
• Fail closed — refusing to itemize is success; wrong itemization is failure. (Same rule the Trainer enforces on Mazda's evidence.)
• Cheap model for simple tasks — spreadsheet reading, item-name lookups, text extraction: GPT-5.4. Reserve big models for judgment calls.
• rol_finances is live and multi-agent-dirty — git status + diff before editing, stage by filename, never -A, never stash the whole tree.
• The live box runs two yolo-mode letta.js processes writing concurrently — check the tree before assuming it's clean.
• Vendor yaml edits require a mazda-tools-mcp restart (cached map).

Open follow-ups inherited by this work (context, not blockers)

• Raw-vs-normalized alias audit: resolve_vendor_key() matches alias regexes against the RAW description, but many of the ~261 aliases in vendor_category.yaml are written in normalized (underscore) form and may never match anything. Tonight's 3 *_general aliases were deliberately written raw-form and verified. A systematic audit belongs to whoever touches VendorCategoryStoreCategorizer.
• check_vendor_key fuzzy matcher too permissive — resolved Amazon rows to Apple/Audible with recognized:true (Trainer report 2026-07-17, wrap-v024 covers receipts but not statement rows). The item-level categorizer must not inherit this: plausibility-gate all fuzzy matches.
• Mazda proposals 84–86 + her memory note from 2026-07-17 document the vendor-resolution saga; read them before changing her wrapper.