# Changelog

Shipped work, by area. Forward-looking plans live in [docs/roadmap/](docs/roadmap/index.md); completed items land here so the roadmaps stay forward-only.

### Breaking

- **`SourceFile::num_lines` is no longer `const fn`** — the line-start table it counts is built on first use now, behind a `OnceLock`, so the method reads a lazily-initialized value and cannot be `const`. Callers using it in a const context need to move the call; every other use is unaffected. `rustledger_booking::interpolate_in_place` and `Rollback` are new public API alongside it, for callers that want to interpolate a transaction they own rather than a copy (PR #2105).
- **`Posting::cost` and `Posting::price` are boxed** — both are now `Option<Box<CostSpec>>` / `Option<Box<PriceAnnotation>>`. `Posting` was 280 bytes and a plain two-posting expense carried roughly 190 bytes of `None`; the two largest and least often present fields are now behind a pointer, taking it to 160. This is not only a memory question: rkyv mirrors the in-memory layout, so every byte of that absent-cost padding was written to the parse cache and then validated and materialized on every cache hit, which is why the archive ran 5.2x the size of the ledger it came from. The cache file drops 36%, peak RSS 11-28% depending on shape, and warm `check` 1.8%; ledgers that *do* carry costs, where the boxes are really allocated, come out flat (-0.1% / +0.03%). Reading through the field takes `as_deref()` — `Deref` carries field accesses, so most call sites are unaffected; construction takes `Box::new` or `.map(Box::new)`. Both parse-cache versions move (loader v29, WASM v15): the archived *values* are unchanged and only the *shape* moved, which is the more dangerous kind to get wrong, because nothing downstream looks wrong while the bytes are read at the wrong offsets (PR #2104).
- **`rustledger::report::SourceCache` is removed** — it exposed `new` and `add` and no reader: the `sources` field is private and there is no getter or iterator, so nothing inside or outside the crate could read what went into it. `check` filled one on every run — a read plus a lossy UTF-8 decode per source file — and dropped it; the diagnostic paths re-read their own source at the point of use, which is why the gap went unnoticed. `Utf8Chunks::next` was the hottest single function in a `check` profile at 9.8% of all instructions, entirely for this. Embedders constructing one can delete the call (PR #2104).

- **`BookingEngine::apply_posting` is now `replay_posting`, and the realize path is private** — one public entry point served two different questions. *Realizing the booked ledger* (`apply`) treats booking's decision as authoritative, and a `{*}` meeting a pool other than the one booking recorded is a defect it refuses (#2068, #2070). *Replaying a filtered subset* — what the query executor does with whatever a `WHERE` clause left — legitimately meets a different pool, and answering it with the whole ledger's pool is itself the bug (#1985). The distinction was maintained only by which level a caller happened to reach for, and by a comment explaining why the `{*}` precondition sits on one and not the other; nothing stopped a filtered replay being handed booking's answer. Callers replaying a filtered stream want `replay_posting`; callers realizing a ledger want `apply`, which applies a whole transaction and runs the precondition first. **Both now refuse an unbooked posting** rather than silently skipping it: a posting whose units are still elided returns the new `BookingError::NotBooked`, where it previously returned success and left the caller with empty balances and no complaint. Booking fills those units in, so a booked ledger cannot reach it — nothing in the workspace suite and none of the 735 compatibility corpus files do (issue #2082, step 1).
- **A file included from more than one place is now reported** — reaching the same file twice in the include graph without forming a cycle (a shared prices or accounts file included from two monthly journals, say) loaded it once and said nothing. Python beancount reports `Duplicate filename parsed` for exactly this and `bean-check` exits 1; rustledger only emitted that diagnostic for a true *cycle*. The file's directives still load exactly once — the report is in addition to loading correctly, not instead of it — but `rledger check` now exits non-zero on a ledger that previously passed. Remove the redundant `include` to fix. The MCP server's `validate_file` reports it too, and no longer inlines such a file twice, which had been silently doubling its amounts (issue #2084 follow-up).
- **STRICT no longer resolves an ambiguous lot match by falling back to FIFO** — when a reduction's cost spec matched several lots that shared a cost number and cost currency, booking drained them oldest-first instead of reporting the ambiguity. The comment justifying that said the user "could not have observed a different outcome" and that beancount does the same; neither was true. Beancount's `booking_method_STRICT` has no fallback, and the outcome is observable because the surviving lot carries its own acquisition date — which drives the short/long split in `report capgains` and per-lot IRR eligibility. Lots that are identical in *every* cost field (number, currency, date and label) are still drained in date order, because beancount's inventory merges those into one position and would never see them as ambiguous. Name the lot (`{74.09 USD, 2022-05-10}`, or a label) or book the account FIFO/HIFO if the choice genuinely does not matter. **`STRICT_WITH_SIZE` is fixed in the same place**: when several lots matched the reduction's size it took whichever was stored first, which is insertion order and only coincidentally date order — a lot with an explicit cost date breaks the coincidence. It now takes the oldest, as beancount does, which changes both the basis realized and the holding period of the lot left standing (issue #2097).
- **Same-date directives now book in file order** — booking previously floated cost augmentations ahead of the reductions matching against them within a `(date, priority)` group. Python has no such rule (`entry_sortkey` is `(date, type_priority, lineno)`), and the reordering changed which lots a match could see: a sell with an empty cost spec written before a same-date buy became an ambiguous match on a ledger Python accepts (issue #2093). Ledgers that relied on the reorder — a sell written before the same-date transaction that supplies its lot — now report the same error Python does. Writing the two in the order they happen fixes them. `rustledger_core::booking_sort_key` returns `(NaiveDate, DirectivePriority)` instead of `(NaiveDate, DirectivePriority, bool)`, and `Directive::has_cost_reduction` — the source of that third element — is removed with the tiebreak it existed for. Downstream code that destructures the key or calls the predicate needs updating; code that only passes the key to a sort does not (PR #2095).
- **`rustledger-lsp` URI conversion is now fallible and typed** — `uri_to_path` and `path_to_uri` return `Result<AbsPathBuf, PathUriError>` / `Result<Uri, PathUriError>` instead of `Option`. `uri_to_path` also refuses a non-`file:` scheme and any URI whose path segment percent-decodes into path syntax, both of which previously resolved to real filesystem paths. Embedders calling these directly must handle the error; the variants name which step failed (PR #1869).

### Performance

- **Multi-currency interpolation is deterministic** — residuals were accumulated in a `std::collections::HashMap`, whose `RandomState` is seeded per process, and the multi-currency split fills the original posting from whichever non-zero residual came first. The same ledger therefore emitted its postings in a different currency order from one run to the next: `report balances` on a two-currency absorb produced two distinct outputs across 30 runs before this change and one after. Amounts always agreed, so nothing was wrong with the numbers — but a diff churned, a snapshot could flake, and which posting index carried which currency moved. The order is now the order the transaction introduces each currency. One use site already sorted its keys to dodge this, under a comment saying unsorted order "would produce non-reproducible test output"; that covered one of the three iterations (PR #2105).
- **Interpolation writes in place** — it used to fill a clone of the transaction and hand it back for the caller to assign over the original, so every transaction paid a full clone and the original's drop, 7.7% of a warm `check`, so that a rare failure could be undone. Interpolation touches one or two postings, so those are saved instead. A transaction that fails to book is not discarded — it is set aside and merged back into the ledger — so it must still read as the author wrote it, which is now maintained explicitly and asserted (PR #2105).
- **The source-map line index is built on first use** — every consumer is a diagnostic (`line_col`, `line`, `line_start`, `num_lines`), so a ledger that reports nothing built a `usize`-per-line table and never read it: 6.8M instructions and 320 KB on a 40,000-line ledger. `line_col` also stops locating a line with a linear scan over a sorted array in favor of a binary search, which only one diagnostic never noticed and a file reporting thousands paid for each time (PR #2105).
- **`simple_index` folded into the units cache** — two maps keyed on the same interned currency and always read together, so `add` hashed it three or four times per posting. Beyond the hashing, a stale slot was harmless but a missing one was a wrong answer, so keeping the pair in agreement was a standing obligation at every mutation site (PR #2105).
- **`rledger check` cache-hit path** — the path the nightly benchmark actually measures, since `bench.yml` runs hyperfine with the parse cache enabled and every timed run is therefore a hit that never parses. It had regressed 22% since v0.21.0 (215.7M vs 176.8M instructions on the 10,000-transaction workload) and is now back to parity at 177.3M, and ~20% better in wall clock because most of what was recovered is memory the instruction count does not see. Four changes: the dead `SourceCache`; `CacheEntry::into_load_result` decoding cached source with `from_utf8_lossy(..).into_owned()`, which validates a byte at a time and copies even when every byte was valid, where `DiskFileSystem::read` already used `String::from_utf8` with a lossy fallback (22.8M instructions, 12.9% of the run); a tautological `sum <= Decimal::MAX` after a `checked_add` that returns `None` on overflow by definition, run twice per posting; and interning during rkyv deserialization under a scoped `InternScope` instead of deduplicating with a second whole-directive walk afterwards (PR #2104).
- **Cold `check` is 31% faster than v0.21.0** (797.3M -> 552.4M instructions), mostly the quadratic work in #2056-#2067 plus right-sizing the per-transaction `postings` vector — `Vec`'s first allocation is 4 elements, so a two-posting transaction reserved four slots' worth to hold two, 10,000 separate overshoots on a 10k ledger (PR #2104).
- **PR benchmarks build the allocator the release binaries ship** — `bench-pr.yml` built both sides without `--features mimalloc` while `release-build.yml` ships it, so the job compared two builds nobody installs. Not cosmetic: a string-interning change measured -14% against the system allocator and 0% against mimalloc, because what it recovered was glibc's per-allocation overhead. It was dropped rather than shipped; the job as written would have endorsed it (PR #2104).

- **Source double-allocation removed** — `Arc<str>` for source code instead of clone (Phase 0.1, ~16% faster, 50% less source memory).
- **Profile-Guided Optimization (PGO)** — release binaries built with benchmark profile data (Phase 0.2, ~13% faster).
- **LTO release profiles** — `thin` LTO on `release` (avoids Apple linker crash), `fat` LTO on dedicated `release-linux` and `wasm-release` profiles, with `codegen-units = 1` + `strip` (Phase 0.3).
- **Zero-copy string parsing** — parser returns borrowed `&str`/`Cow<str>` and interns later instead of eager `.to_string()` (Phase 1, ~7% faster).
- **FxHashMap / rustc-hash on hot paths** — non-cryptographic hashing for interned-string/currency keys across core, validate, booking, query, loader; inventory hot path is fxhash-only (Phase 2.4, issue #1237).
- **imbl persistent collections** — `Inventory.positions` uses `imbl::Vector` for O(1) clone via structural sharing, removing quadratic snapshot blow-up (Phase 2.5, issue #1086).
- **Full string interning** — `InternedStr` extended to payee/narration/tags/links on `Transaction` and `Document`; cache re-interning deduplicates 150+ strings/ledger (Phase 3, ~6% faster).
- **Rayon validation parallelization** — parallel transaction interpolation and independent checks, sorting kept sequential (Phase 4, ~5% faster).
- **Parallel query execution** — simple per-posting query evaluation runs via `par_iter()` above a threshold, `DISTINCT` dedup stays sequential (Phase 4.3).
- **parking_lot locks** — faster, smaller, non-poisoning `RwLock`/`Mutex` in query executor and LSP state (Phase 4.4).
- **Binary cache (rkyv)** — zero-copy deserialization cache keyed on BLAKE3 of path+mtime+size, with invalidation and `--no-cache`/`-C` CLI flags; 2.3x faster on cache hit (30ms → 13ms) (Phase 5, issue #939).
- **Logos lexer + structured rowan CST parser** — SIMD-accelerated tokenization and hand-written lossless CST, replacing the Chumsky combinator parser (Phase 6.1).
- **Parser fast paths** — empty-collection avoidance, SIMD escape check, in-parser string interning, hand-rolled Decimal/date parsing, `Cow` strings; 1K txns 1,204μs → 700μs, -42% (Phase 6.3, PR #812).
- **Validation fast paths** — precomputed tolerances, BigDecimal bypass when Decimal residual is zero, alloc-free account-name validation; 1K txns 210μs → 90μs, -57% (Phase 6.4, PR #814).
- **Parallel file loading** — sibling includes read + parsed in parallel via rayon with sequential order-preserving merge (DiskFileSystem only); multi-file ledgers load 2-4x faster (Phase 6.5, PR #813).

**Overall:** ~20x faster validation and ~34x faster balance reports vs Python beancount on 10K-transaction ledgers; ~13ms on cache hit.

### Importing & Ingestion

- Import trait system (`Importer` trait, `ImportResult`, registry)
- CSV importer with column mapping, date formats, and debit/credit split
- OFX/QFX importer
- CSV auto-inference: delimiter, date format, and column-role detection (`--auto`)
- `rustledger-ops` crate: pure operations for dedup, categorize, fingerprint, reconcile, merchants, and transfer detection
- Rules engine with substring, regex, and exact-match rules and priority ordering
- Built-in merchant dictionary (~230 patterns across groceries, dining, transport, subscriptions, etc.)
- Transaction fingerprinting via blake3 structural hashing for stable dedup
- Enriched import results (`EnrichedImportResult`) with confidence scores and categorization method
- Naive Bayes ML categorization trained on the user's ledger, wired into `rledger extract --suggest-categories`
- WASM import plugins: third-party importers as sandboxed `.wasm` modules
- Balance-directive generation via `rledger extract --balance`/`--balance-date` (amount user-supplied; automatic statement extraction still future)

### Testing & Engineering Quality

- **Phase 1 — Quick Wins:** Miri weekly UB detection (`miri.yml`), `cargo-nextest` for fast parallel CI tests, Codecov coverage reporting (`quality.yml`), and Criterion micro-benchmarks in CI (`bench-pr.yml`).
- **Phase 2 — Fuzzing infrastructure:** CI fuzzing (`fuzz.yml`) with `fuzz_parse`, `fuzz_parse_line`, `fuzz_query_parse` (query engine), and `fuzz_booking` targets, running on parser PRs and nightly.
- **Phase 3 — Compatibility enhancements:** Expanded BQL compat corpus to ~17 curated queries (`bql-queries.toml`) covering aggregates, date/string functions, and edge cases; file sampling via `compat-bql-test.py` (`MAX_FILES = 30`, plugin fixtures prioritized).
- **Phase 4 — Formal verification bridge:** Kani proof harnesses (`crates/rustledger-core/src/kani_proofs.rs`, `kani.yml`) verifying core invariants, plus TLA+ trace-to-test automation wired into `tla.yml` (19 specs).
- **Phase 5 — Mutation testing:** Per-package `cargo-mutants` matrix (`mutation.yml`, #1238) — each package mutated in its own fail-isolated job with its own time budget and per-package artifact upload.
- **Phase 6 — WASM testing:** `wasm-pack` Node.js + browser tests for `rustledger-wasm` (`wasm.yml`, `tests/node.rs`).
- **Pipeline-boundary property tests (#1235):** `pipeline_invariants.rs` across parser/booking/validate/query/plugin, plus `booking_phase_invariants.rs`, `tla_proptest.rs`, and `plugin_determinism.rs` — pinning parse/format roundtrip + idempotence, booking idempotence, validation determinism, plugin wire-format roundtrip, and query-result determinism.
- **Grep ratchets (#1237):** `check-sync-primitives.sh` (forbids `std::sync::Mutex`/`RwLock` in library code) and `check-hot-path-collections.sh` (forbids SipHash `HashMap`/`HashSet` in `fxhash-only` modules), wired into `ci.yml` alongside the existing `check-unsafe-invariant.sh` gate.
- **SemVer / API-stability gate (#1233):** `semver-plugin-types` CI job runs `cargo-semver-checks` against `rustledger-plugin-types` (Tier-1 plugin DTOs) on every PR.
- **FFI synth-pass parity (#1404):** `rustledger-ffi-wasi`'s `load_file`/`load_source` route loading through the canonical `process::load` pipeline instead of a duplicated partial loader, so the pre-booking synth pass (`auto_accounts`, `document_discovery`) runs through the JSON-RPC and WIT/component surfaces — previously it ran only natively. Added `load_synth_plugins.rs` (ffi-wasi) and component parity tests pinning generated-`Open` output, closing the coverage gap that let the divergence ship.
- **FFI-Component release artifact (#1410):** the release pipeline now builds and attaches the wasip2 Component-Model wasm (`rustledger-ffi-component-<version>.wasm` + `.sha256`) alongside the existing `rustledger-ffi-wasi` artifact, so in-process Component-Model embedders have a downloadable component (the crate stays `publish = false` — it is a wasm artifact, not a crates.io library).
- **`format` keeps open currency lists comma-separated (#1405):** the CST formatter no longer rewrites an account's currency constraint list space-separated (`open Assets:Wallet USD,EUR` → `USD EUR`, invalid beancount); it now emits `USD,EUR` and is idempotent.
- **LSP completion-resolve detail (#1408):** `completionItem/resolve` now also surfaces the ledger-wide summary (account balance + transaction count, or currency usage + latest price) in the item's `detail`, not only its documentation popup, so clients that render `detail` inline show it.

### Formal Verification

- Built 18 TLA+ specifications, 16 of which are model-checked on every CI run (`.github/workflows/tla.yml`).
- Conservation/inventory invariants: `Conservation.tla` (ConservationInvariant, NonNegativeInventory), `MultiCurrency.tla` (multi-currency conservation).
- Double-entry and account invariants: `DoubleEntry.tla` (TransactionsBalance, NoSelfTransfer), `AccountStateMachine.tla` (ClosedHaveZeroBalance, TypeOK), `Interpolation.tla` (AtMostOneNull, CompleteImpliesBalanced).
- Booking-method specs: `FIFOCorrect.tla`, `LIFOCorrect.tla`, `HIFOCorrect.tla`, `STRICTCorrect.tla`, `AVERAGECorrect.tla`, `NONECorrect.tla` — each verifying correct lot selection / booking-method semantics.
- System specs: `PriceDB.tla` (price database consistency), `ValidationCorrect.tla` (balance-assertion validation), `PluginCorrect.tla` (plugin execution ordering), `ConcurrentAccess.tla` (concurrent read/write safety), `QueryExecution.tla` (BQL query execution invariants).
- Illustrative demo specs (not in CI): `SimpleInventory.tla` (basic add/reduce) and `BuggyInventory.tla` (demonstrates TLC catching bugs); `FIFOCheck.tla` retained for historical reference.
- Found and fixed a real bug: `FIFOCheck.tla` revealed `inventory.rs` was selecting lots by insertion order instead of acquisition date; the TLC counterexample was converted to `crates/rustledger-core/tests/tla_fifo_bug_test.rs` and the bug fixed via date-based sorting in `reduce_ordered()`.
- Established CI runner tooling (`tools/tla2tools.jar`) with per-spec `.cfg` files and multi-core model-checking support.
