Главная Обзор Помощь
Вход
Cesar Rodas cesar
Репозитории Активность

cesar запушил(а) sql-hex-text-auditability в cesar/ledger-rs

  • 2015894107 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • 007e73f2c5 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • ee2e0e2b12 Introduce Kuatia, an append-only, auditable multi-asset ledger Kuatia tracks value as signed postings rather than mutable balance fields, so every state change is an immutable record and the full history is auditable. A transfer atomically consumes and creates postings and must conserve value per asset (the sum of consumed equals the sum of created), which is the double-entry safety invariant enforced on every commit. The public surface is intent-based. Callers describe movements (pay/deposit/withdraw) through a builder; the core resolves them into a concrete envelope of postings to consume and create, selecting inputs greedily and computing change or an overdraft posting as the account policy allows. Overdraft behavior is a per-account policy: NoOverdraft forbids negative postings, capped and uncapped variants permit them down to a floor or without bound, and system/external accounts model the ledger boundary. Commits run through a two-step saga, reserve then finalize, with validation as the last thing before the writes, automatic retry, and LIFO compensation. Transfers are content-addressed (a double SHA-256 of their canonical bytes), which gives idempotency and tamper evidence. Storage is deliberately dumb: each write primitive applies one conditional update and returns the number of rows it changed, and the saga owns all interpretation, idempotency, and compensation. Crash safety comes from a phase-tracked write-ahead record plus a recover() that rolls a half-applied commit forward rather than unwinding it. The code separates a pure, sans-IO core from the async layer. The core is deterministic and unit-testable; the async layer adds the Store trait and the saga. Storage backends (in-memory and SQLite/PostgreSQL) share one conformance suite, and concurrency tests pin the guarantees that matter: double-spend prevention is exact, while the overdraft floor re-check is best-effort under concurrency (documented, with conservation preserved). Identifiers are snowflake-style i64 values generated in Rust, never by the database, and are unique across threads. The monetary type hides its backing integer, which is swappable from i64 to i128 at compile time. The repository ships architecture and API docs plus a set of ADRs recording the design decisions.
  • 39a4d6a10b Introduce Kuatia, an append-only, auditable multi-asset ledger Kuatia tracks value as signed postings rather than mutable balance fields, so every state change is an immutable record and the full history is auditable. A transfer atomically consumes and creates postings and must conserve value per asset (the sum of consumed equals the sum of created), which is the double-entry safety invariant enforced on every commit. The public surface is intent-based. Callers describe movements (pay/deposit/withdraw) through a builder; the core resolves them into a concrete envelope of postings to consume and create, selecting inputs greedily and computing change or an overdraft posting as the account policy allows. Overdraft behavior is a per-account policy: NoOverdraft forbids negative postings, capped and uncapped variants permit them down to a floor or without bound, and system/external accounts model the ledger boundary. Commits run through a two-step saga, reserve then finalize, with validation as the last thing before the writes, automatic retry, and LIFO compensation. Transfers are content-addressed (a double SHA-256 of their canonical bytes), which gives idempotency and tamper evidence. Storage is deliberately dumb: each write primitive applies one conditional update and returns the number of rows it changed, and the saga owns all interpretation, idempotency, and compensation. Crash safety comes from a phase-tracked write-ahead record plus a recover() that rolls a half-applied commit forward rather than unwinding it. The code separates a pure, sans-IO core from the async layer. The core is deterministic and unit-testable; the async layer adds the Store trait and the saga. Storage backends (in-memory and SQLite/PostgreSQL) share one conformance suite, and concurrency tests pin the guarantees that matter: double-spend prevention is exact, while the overdraft floor re-check is best-effort under concurrency (documented, with conservation preserved). Identifiers are snowflake-style i64 values generated in Rust, never by the database, and are unique across threads. The monetary type hides its backing integer, which is swappable from i64 to i128 at compile time. The repository ships architecture and API docs plus a set of ADRs recording the design decisions.
  • Просмотр сравнение для этих 4 коммитов »

1 день назад

cesar запушил(а) refactor/dumb-storage-saga-recovery в cesar/ledger-rs

  • 4f8f528196 Make crash recovery validating, phase-tracked, and non-double-spending A review of the first recovery cut found it could commit envelopes that never validated and could double-spend: recover() force-completed blindly, skipping validation and ignoring the affected-row counts from reserve/deactivate. It also take()-d the plan (breaking finalize retry) and deleted the pending record even after a mid-finalize failure (losing the roll-forward record). Rework recovery around a persisted phase and a single verified finalize path. The write-ahead PendingSaga now carries a phase: Reserving (before reserve) is bumped to Finalizing at the point of no return, after validation passes and just before the consumed postings start turning Inactive. recover() branches on it: a Reserving saga is re-run through the real saga, which re-reserves and re-validates against current state (aborting cleanly if a posting was taken or an account frozen); a Finalizing saga is rolled forward through finalize_envelope. - Add Ledger::finalize_envelope: one idempotent, end-state-verified commit used by both the saga's finalize step and recovery. It re-validates while the consumed postings are still pre-deactivation (the last-step floor/freeze-close guard), then never creates or stores unless ALL consumed postings are confirmed Inactive — the double-spend guard. No plan take(), so finalize is retry-safe. - commit_envelope keeps the pending record on a mid-finalize failure (roll forward) and deletes only on commit or a clean pre-finalize abort. - Collapse the pipeline: validation moves into finalize as its last-step check, so the saga is reserve -> finalize; remove ValidateTransferStep and the unused ResolveStep. - Tests cover each crash phase: re-drive Reserving, roll forward a partial finalize, abort+release when an account is frozen, and refuse to double-spend a taken posting. - Floor/freeze guards are now tightest-best-effort (re-checked just before the writes, on the recovery path too) but not strictly atomic; documented as such. - Sync all docs, READMEs, module docs, and the ADR to the phase-tracked model.
  • 752185dbc1 Make crash recovery validating, phase-tracked, and non-double-spending A review of the first recovery cut found it could commit envelopes that never validated and could double-spend: recover() force-completed blindly, skipping validation and ignoring the affected-row counts from reserve/deactivate. It also take()-d the plan (breaking finalize retry) and deleted the pending record even after a mid-finalize failure (losing the roll-forward record). Rework recovery around a persisted phase and a single verified finalize path. The write-ahead PendingSaga now carries a phase: Reserving (before reserve) is bumped to Finalizing at the point of no return, after validation passes and just before the consumed postings start turning Inactive. recover() branches on it: a Reserving saga is re-run through the real saga, which re-reserves and re-validates against current state (aborting cleanly if a posting was taken or an account frozen); a Finalizing saga is rolled forward through finalize_envelope. - Add Ledger::finalize_envelope: one idempotent, end-state-verified commit used by both the saga's finalize step and recovery. It re-validates while the consumed postings are still pre-deactivation (the last-step floor/freeze-close guard), then never creates or stores unless ALL consumed postings are confirmed Inactive — the double-spend guard. No plan take(), so finalize is retry-safe. - commit_envelope keeps the pending record on a mid-finalize failure (roll forward) and deletes only on commit or a clean pre-finalize abort. - Collapse the pipeline: validation moves into finalize as its last-step check, so the saga is reserve -> finalize; remove ValidateTransferStep and the unused ResolveStep. - Tests cover each crash phase: re-drive Reserving, roll forward a partial finalize, abort+release when an account is frozen, and refuse to double-spend a taken posting. - Floor/freeze guards are now tightest-best-effort (re-checked just before the writes, on the recovery path too) but not strictly atomic; documented as such. - Sync all docs, READMEs, module docs, and the ADR to the phase-tracked model.
  • 45b57f2319 Make storage a dumb instruction-follower; move commit + recovery into the saga The commit path bundled everything into one monolithic store transaction (commit_transfer) and exposed two confusing entry points (commit / commit_atomic). That put a lot of domain assumptions in storage — it interpreted state, enforced guards, decided idempotency and error semantics — and the saga's durable crash recovery was designed (SagaStore, legend pause/resume) but never wired, so that single transaction was the only thing protecting against a half-applied commit. Invert it. Storage becomes a dumb instruction-follower: every write applies one update and returns the number of affected rows (or an I/O error), never deciding state, idempotency, or compensation. The saga reads each count and owns the logic (full continue / partial error+compensate / zero read-and-check-same-envelope). Crash-safety moves to a write-ahead PendingSaga record plus idempotent roll-forward in Ledger::recover(), so a crash at any point converges without a global transaction and leaves no orphaned reservations. - Storage primitives now return counts: reserve/release/deactivate_postings, insert_postings, store_transfer(record, involved), idempotent append_event (dedup_key in 001_init.sql). Drop the semantic write-outcome StoreError variants. - Unify on commit(transfer) = resolve + commit_envelope (reserve -> validate -> finalize on the primitives); reverse() uses it; remove commit_atomic and the CommitStore/CommitRequest/commit_transfer atomic boundary and Plan guard fields. - Add Ledger::recover() with write-ahead persistence; prove it with recovery and mid-finalize crash-injection tests. - Tradeoff: the CappedOverdraft floor and freeze/close guards are now validate-time and best-effort under concurrency; double-spend safety still holds via the reservation protocol. Recorded in doc/adr/0001-dumb-storage-saga-recovery.md. - Sweep docs, READMEs, module docs, and examples to the new model.
  • a19dfcd050 Make storage a dumb instruction-follower; move commit + recovery into the saga The commit path bundled everything into one monolithic store transaction (commit_transfer) and exposed two confusing entry points (commit / commit_atomic). That put a lot of domain assumptions in storage — it interpreted state, enforced guards, decided idempotency and error semantics — and the saga's durable crash recovery was designed (SagaStore, legend pause/resume) but never wired, so that single transaction was the only thing protecting against a half-applied commit. Invert it. Storage becomes a dumb instruction-follower: every write applies one update and returns the number of affected rows (or an I/O error), never deciding state, idempotency, or compensation. The saga reads each count and owns the logic (full continue / partial error+compensate / zero read-and-check-same-envelope). Crash-safety moves to a write-ahead PendingSaga record plus idempotent roll-forward in Ledger::recover(), so a crash at any point converges without a global transaction and leaves no orphaned reservations. - Storage primitives now return counts: reserve/release/deactivate_postings, insert_postings, store_transfer(record, involved), idempotent append_event (dedup_key in 001_init.sql). Drop the semantic write-outcome StoreError variants. - Unify on commit(transfer) = resolve + commit_envelope (reserve -> validate -> finalize on the primitives); reverse() uses it; remove commit_atomic and the CommitStore/CommitRequest/commit_transfer atomic boundary and Plan guard fields. - Add Ledger::recover() with write-ahead persistence; prove it with recovery and mid-finalize crash-injection tests. - Tradeoff: the CappedOverdraft floor and freeze/close guards are now validate-time and best-effort under concurrency; double-spend safety still holds via the reservation protocol. Recorded in doc/adr/0001-dumb-storage-saga-recovery.md. - Sweep docs, READMEs, module docs, and examples to the new model.
  • 93e35fed20 Close double-spend races and atomicity gaps in the storage layer A concurrent saga or raw transfer could double-spend a posting: the SQL store checked posting status with a SELECT, then mutated with an UPDATE keyed only on the primary key, so under PostgreSQL READ COMMITTED two transactions could both pass the read and both apply the write. Reservation, finalize, and lifecycle paths had matching gaps, and the account snapshot pinned at validation was never re-checked at commit. Make the status/reservation precondition the authorization boundary via conditional UPDATE ... WHERE + rows_affected checks, matching the InMemory reference. Remove the split finalize_postings/store_transfer write APIs so all posting mutation flows through the atomic commit_transfer (which also fixes the created-only transfer indexing). Propagate lifecycle event-append errors, reject close on any non-Inactive posting, and re-check account versions atomically at commit via new account_guards on the plan and commit request.
  • Просмотр сравнение для этих 22 коммитов »

1 день назад

cesar запушил(а) rc/v0.2.0 в cesar/ledger-rs

  • b3fcc0152e Prepare 0.2.0 release Bump the workspace and all crate versions to 0.2.0 and document the release. Since 0.1.0 the ledger gained an HTTP dashboard for observing accounts, postings, transfers, and events, and the SQL backend switched to hex/JSON text columns so a ledger can be audited directly with SQL tooling. Update the changelog with the 0.2.0 section and regenerate Cargo.lock.
  • cfed6747f3 Prepare 0.2.0 release Bump the workspace and all crate versions to 0.2.0 and document the release. Since 0.1.0 the ledger gained an HTTP dashboard for observing accounts, postings, transfers, and events, and the SQL backend switched to hex/JSON text columns so a ledger can be audited directly with SQL tooling. Update the changelog with the 0.2.0 section and regenerate Cargo.lock.
  • 2e53654b6c Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • c9eeff1890 Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • 2015894107 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • Просмотр сравнение для этих 8 коммитов »

1 день назад

cesar запушил(а) rc/v0.1.1 в cesar/ledger-rs

  • 2e53654b6c Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • c9eeff1890 Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • 2015894107 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • 007e73f2c5 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • ee2e0e2b12 Introduce Kuatia, an append-only, auditable multi-asset ledger Kuatia tracks value as signed postings rather than mutable balance fields, so every state change is an immutable record and the full history is auditable. A transfer atomically consumes and creates postings and must conserve value per asset (the sum of consumed equals the sum of created), which is the double-entry safety invariant enforced on every commit. The public surface is intent-based. Callers describe movements (pay/deposit/withdraw) through a builder; the core resolves them into a concrete envelope of postings to consume and create, selecting inputs greedily and computing change or an overdraft posting as the account policy allows. Overdraft behavior is a per-account policy: NoOverdraft forbids negative postings, capped and uncapped variants permit them down to a floor or without bound, and system/external accounts model the ledger boundary. Commits run through a two-step saga, reserve then finalize, with validation as the last thing before the writes, automatic retry, and LIFO compensation. Transfers are content-addressed (a double SHA-256 of their canonical bytes), which gives idempotency and tamper evidence. Storage is deliberately dumb: each write primitive applies one conditional update and returns the number of rows it changed, and the saga owns all interpretation, idempotency, and compensation. Crash safety comes from a phase-tracked write-ahead record plus a recover() that rolls a half-applied commit forward rather than unwinding it. The code separates a pure, sans-IO core from the async layer. The core is deterministic and unit-testable; the async layer adds the Store trait and the saga. Storage backends (in-memory and SQLite/PostgreSQL) share one conformance suite, and concurrency tests pin the guarantees that matter: double-spend prevention is exact, while the overdraft floor re-check is best-effort under concurrency (documented, with conservation preserved). Identifiers are snowflake-style i64 values generated in Rust, never by the database, and are unique across threads. The monetary type hides its backing integer, which is swappable from i64 to i128 at compile time. The repository ships architecture and API docs plus a set of ADRs recording the design decisions.
  • Просмотр сравнение для этих 6 коммитов »

1 день назад

cesar запушил(а) lifecycle-transition-recovery в cesar/ledger-rs

  • 32f50f7ac6 Make account lifecycle transitions crash-safe freeze, unfreeze, and close each append a new account version and then append the matching lifecycle event as two separate store writes with no shared transaction. A crash between them left a durable version bump with no event, and nothing repaired it. This is the same window recover() already closes for store_transfer followed by append_event on the commit path, left open for account transitions. Route the transitions through that existing write-ahead / repair path. The three methods, previously near-identical copies of the same five-step shape, collapse into one transition primitive parameterized by the flag mutation and the event. It persists a PendingTransition write-ahead record before either write; recover() rolls it forward, appending the version only when it is not yet present and re-appending the event. Rolling the event forward requires it to be idempotent, which lifecycle events were not: event_dedup_key returned None for them, so a second append duplicated the row. Give the three transition events a version field and key event_dedup_key on (account, version). The key type generalizes to a string; the transfer form stays the lowercase hex the dedup_key column already holds, so no migration is needed, and the field is serde-default so existing event rows still load. The write-ahead records now share one tagged PendingRecord enum so recover() dispatches an envelope commit saga and an account transition to their own completion paths. See ADR-0019.
  • dc818a6246 Squashed commit of the following: commit 631a6e9e4241bd4995f6152f5881741e76b8b2d5 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Sun Jul 19 13:29:27 2026 -0300 Coordinate balance-projection appends through storage, not memory Address a review of ADR-0019. The background cache-point append had no dedup, so a hot account read at high QPS spawned one full fold per read. An in-process guard cannot coordinate the multiple independent instances the projection is built for, so move the guard into the shared balance_projection rows: append_cache_point returns before the fold when a cache point already sits within a debounce window below the target watermark. Redundant spawns then cost one indexed read, and the dedup holds across processes and restarts with no lease, lock, or CAS, matching the coordination-free design. Log a warning when a background append fails so a projection that silently stops advancing is visible. Add a deterministic test that folds a non-empty tail onto a real snapshot, the watermark-boundary case the existing tests never reached (they either created no snapshot or left the tail empty). Correct the property-test comment that claimed cache points were appended during the run, and the ADR header that still named the rejected lease and projector service. commit b169e1f084375cd6a9658375cfb046ab7019a3f9 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Fri Jul 17 09:20:35 2026 -0300 Add cached balance projection (ADR-0019) Balance was summed from every live posting on each read, which is O(N live postings) and unbounded under UTXO fragmentation. Add a rebuildable balance projection: a read returns the closest stored snapshot plus a Rust-summed tail of committed transfer deltas past the snapshot's commit-time watermark. Summing every one of an account's transfer created(+)/consumed(-) deltas for an asset equals its live-posting sum, so the projection-aware read always agrees with the authoritative balance, and it is faster once a snapshot keeps the tail short. Make the projection append-only rather than a mutable row guarded by a lock or a CAS. Each cache point is one snapshot of a (account, subaccount, asset) balance plus the watermark it covers, tagged with a Rust-minted monotonic id, and rows are only ever inserted. A read selects the cache point closest to (at or before) the target time (largest watermark not exceeding it, tie-broken by highest id), so a stale or duplicate append is harmless and concurrent appends need no coordination. Add a BalanceProjectionStore sub-trait (append + closest-at-or-before get) with an in-memory and SQL implementation (migration 007, balance stored as TEXT like every other monetary column) and conformance tests. Trigger snapshots lazily off the read path, not on commit. A read whose folded tail has accrued at least snapshot-interval new credits/debits since the closest cache point spawns a best-effort background append; a read that finds no cache point returns the authoritative live sum directly and bootstraps one in the background. The append folds only up to now minus a grace window, so a snapshot never captures a commit still racing to become visible. Both bounds are tunable (with_snapshot_interval, with_projection_grace_ms) with sane defaults. Commit itself never touches the projection, keeping the write path unchanged; correctness never depends on any of this. Validation keeps reading the authoritative live sum (compute_balance, now public), so the UTXO reservation protocol remains the exact concurrency control for no-overdraft accounts and the projection stays a pure read accelerator.
  • 770e0ea7a5 Make account lifecycle transitions crash-safe freeze, unfreeze, and close each append a new account version and then append the matching lifecycle event as two separate store writes with no shared transaction. A crash between them left a durable version bump with no event, and nothing repaired it. This is the same window recover() already closes for store_transfer followed by append_event on the commit path, left open for account transitions. Route the transitions through that existing write-ahead / repair path. The three methods, previously near-identical copies of the same five-step shape, collapse into one transition primitive parameterized by the flag mutation and the event. It persists a PendingTransition write-ahead record before either write; recover() rolls it forward, appending the version only when it is not yet present and re-appending the event. Rolling the event forward requires it to be idempotent, which lifecycle events were not: event_dedup_key returned None for them, so a second append duplicated the row. Give the three transition events a version field and key event_dedup_key on (account, version). The key type generalizes to a string; the transfer form stays the lowercase hex the dedup_key column already holds, so no migration is needed, and the field is serde-default so existing event rows still load. The write-ahead records now share one tagged PendingRecord enum so recover() dispatches an envelope commit saga and an account transition to their own completion paths. See ADR-0019.
  • Просмотр сравнение для этих 3 коммитов »

1 день назад

cesar запушил(а) fix/sql-double-spend-races в cesar/ledger-rs

  • 93e35fed20 Close double-spend races and atomicity gaps in the storage layer A concurrent saga or raw transfer could double-spend a posting: the SQL store checked posting status with a SELECT, then mutated with an UPDATE keyed only on the primary key, so under PostgreSQL READ COMMITTED two transactions could both pass the read and both apply the write. Reservation, finalize, and lifecycle paths had matching gaps, and the account snapshot pinned at validation was never re-checked at commit. Make the status/reservation precondition the authorization boundary via conditional UPDATE ... WHERE + rows_affected checks, matching the InMemory reference. Remove the split finalize_postings/store_transfer write APIs so all posting mutation flows through the atomic commit_transfer (which also fixes the created-only transfer indexing). Propagate lifecycle event-append errors, reject close on any non-Inactive posting, and re-check account versions atomically at commit via new account_guards on the plan and commit request.
  • 519edc4f62 Close double-spend races and atomicity gaps in the storage layer A concurrent saga or raw transfer could double-spend a posting: the SQL store checked posting status with a SELECT, then mutated with an UPDATE keyed only on the primary key, so under PostgreSQL READ COMMITTED two transactions could both pass the read and both apply the write. Reservation, finalize, and lifecycle paths had matching gaps, and the account snapshot pinned at validation was never re-checked at commit. Make the status/reservation precondition the authorization boundary via conditional UPDATE ... WHERE + rows_affected checks, matching the InMemory reference. Remove the split finalize_postings/store_transfer write APIs so all posting mutation flows through the atomic commit_transfer (which also fixes the created-only transfer indexing). Propagate lifecycle event-append errors, reject close on any non-Inactive posting, and re-check account versions atomically at commit via new account_guards on the plan and commit request.
  • dad6c656b9 Add runnable examples for connecting to and using a ledger New users had to reverse-engineer usage from integration tests. Add self-contained example programs that connect to a real SQLite-backed ledger via sqlx and walk through the core operations. - create_accounts: open a ledger and create user/system/external accounts. - fund_and_trade: deposit two assets into two accounts, then swap them in one atomic two-movement transfer. - withdraw: fund an account, then move value out to the external boundary. Also add an Account::new(id, policy) convenience constructor so callers avoid a seven-field struct literal for the common case, and wire kuatia-storage-sql + sqlx as dev-dependencies of kuatia for the examples.
  • 6f1c5f1a5b Add runnable examples for connecting to and using a ledger New users had to reverse-engineer usage from integration tests. Add self-contained example programs that connect to a real SQLite-backed ledger via sqlx and walk through the core operations. - create_accounts: open a ledger and create user/system/external accounts. - fund_and_trade: deposit two assets into two accounts, then swap them in one atomic two-movement transfer. - withdraw: fund an account, then move value out to the external boundary. Also add an Account::new(id, policy) convenience constructor so callers avoid a seven-field struct literal for the common case, and wire kuatia-storage-sql + sqlx as dev-dependencies of kuatia for the examples.
  • c715bd3924 Make the Store the atomic invariant boundary for commits A static review found the pure validator was strong but the commit path was split across separate storage calls, so a crash could change balances without recording the transfer, and declared protections (CAS guards, book policy, overdraft) were computed but never enforced. This reworks the Store trait to be the single place ledger state changes. - Add CommitStore::commit_transfer: postings, transfer record, the both-sided account index, and events apply in one transaction. Events are no longer best-effort. CAS guards are enforced (retryable Conflict) and consumed postings are authorized against a ReservationId so only the reserving saga can finalize or release them. - Model overdraft as a negative posting covering a shortfall, down to the floor for CappedOverdraft and unbounded for UncappedOverdraft; resolve() mints it and validation permits negative postings on every policy except NoOverdraft. - Enforce BookPolicy (allowed assets/accounts/flags) in validation. - Make the SQL backend portable: per-backend migrations (BLOB vs BYTEA), a migrations ledger for idempotency, and portable upserts. - Stop content-addressed ids depending on randomness: deterministic DEFAULT_BOOK, and move the AutoId timestamp base to a fixed recent epoch. - Sync all documentation and project context to the new behavior.
  • Просмотр сравнение для этих 18 коммитов »

1 день назад

cesar запушил(а) feature/subaccounts в cesar/ledger-rs

  • 0d52129545 Add a subaccount dimension to account identity Some workloads need several distinct balances under one owner (sub-ledgers, earmarks, reservations) that are individually addressable and closed independently, without minting unrelated top-level accounts. This implements ADR-0012: fold a subaccount into account identity so each partition is a full account record with its own policy and lifecycle, while existing behaviour is unchanged when subaccounts are not used. AccountId becomes { id: i64, sub: i64 } where sub = 0 is the main account. Conservation, double-spend, and floor checks are preserved because they already key on the whole AccountId, which now distinguishes subaccounts, so they become per-subaccount with no logic change. Balances are always reported per subaccount and never summed across them; the ledger gains balances() and list_subaccounts(). Aggregate reads take a base id plus an optional subaccount filter, while exact entity operations take the full AccountId. Book membership is scoped by base account. The SQL backend adds a migration that widens the accounts and transfer_accounts primary keys and adds a subaccount column to postings, with existing rows defaulting to the main account. The dashboard surfaces subaccounts in the account list and detail views. For presentation and routing, AccountId gains an IBAN-style string form: two mod-97 check digits followed by a base-36 body, with no country code. Parsing validates the checksum, so a mistyped id is rejected before it reaches the store. To avoid leaking raw ids, the (id, sub) pair is run through a keyed 128-bit Feistel permutation before encoding and inverted on parse, so codes look random and a base account and its subaccount are not visibly related. The permutation key is a global seed with a default, configurable via set_id_seed (the dashboard exposes --id-seed / KUATIA_ID_SEED). This is obfuscation, not security. The string is an edge form only; storage, the Store trait, content hashing, and serde keep the two i64 legs. Finally, make the SQL entity writes safe under concurrency with pessimistic locking. create_account, append_account_version, and create_book each ran a separate existence or version check followed by an insert, a check-then-act TOCTOU where two concurrent callers could both pass the check and then both write. Each now runs in a transaction that locks the relevant rows with SELECT ... FOR UPDATE so a competing writer for the same entity blocks until commit. FOR UPDATE is Postgres-only, so the backend is detected once and the clause is omitted on SQLite, which serializes writers itself; an ON CONFLICT DO NOTHING insert remains as the portable backstop and covers the append phantom-insert a row lock does not. Claude-Session: https://claude.ai/code/session_01SJFJen8Ethv9Q6Ysb1xmz4
  • e7b2cfd7e9 Add a subaccount dimension to account identity Some workloads need several distinct balances under one owner (sub-ledgers, earmarks, reservations) that are individually addressable and closed independently, without minting unrelated top-level accounts. This implements ADR-0012: fold a subaccount into account identity so each partition is a full account record with its own policy and lifecycle, while existing behaviour is unchanged when subaccounts are not used. AccountId becomes { id: i64, sub: i64 } where sub = 0 is the main account. Conservation, double-spend, and floor checks are preserved because they already key on the whole AccountId, which now distinguishes subaccounts, so they become per-subaccount with no logic change. Balances are always reported per subaccount and never summed across them; the ledger gains balances() and list_subaccounts(). Aggregate reads take a base id plus an optional subaccount filter, while exact entity operations take the full AccountId. Book membership is scoped by base account. The SQL backend adds a migration that widens the accounts and transfer_accounts primary keys and adds a subaccount column to postings, with existing rows defaulting to the main account. The dashboard surfaces subaccounts in the account list and detail views. For presentation and routing, AccountId gains an IBAN-style string form: two mod-97 check digits followed by a base-36 body, with no country code. Parsing validates the checksum, so a mistyped id is rejected before it reaches the store. To avoid leaking raw ids, the (id, sub) pair is run through a keyed 128-bit Feistel permutation before encoding and inverted on parse, so codes look random and a base account and its subaccount are not visibly related. The permutation key is a global seed with a default, configurable via set_id_seed (the dashboard exposes --id-seed / KUATIA_ID_SEED). This is obfuscation, not security. The string is an edge form only; storage, the Store trait, content hashing, and serde keep the two i64 legs. Finally, make the SQL entity writes safe under concurrency with pessimistic locking. create_account, append_account_version, and create_book each ran a separate existence or version check followed by an insert, a check-then-act TOCTOU where two concurrent callers could both pass the check and then both write. Each now runs in a transaction that locks the relevant rows with SELECT ... FOR UPDATE so a competing writer for the same entity blocks until commit. FOR UPDATE is Postgres-only, so the backend is detected once and the clause is omitted on SQLite, which serializes writers itself; an ON CONFLICT DO NOTHING insert remains as the portable backstop and covers the append phantom-insert a row lock does not.
  • b3fcc0152e Prepare 0.2.0 release Bump the workspace and all crate versions to 0.2.0 and document the release. Since 0.1.0 the ledger gained an HTTP dashboard for observing accounts, postings, transfers, and events, and the SQL backend switched to hex/JSON text columns so a ledger can be audited directly with SQL tooling. Update the changelog with the 0.2.0 section and regenerate Cargo.lock.
  • cfed6747f3 Prepare 0.2.0 release Bump the workspace and all crate versions to 0.2.0 and document the release. Since 0.1.0 the ledger gained an HTTP dashboard for observing accounts, postings, transfers, and events, and the SQL backend switched to hex/JSON text columns so a ledger can be audited directly with SQL tooling. Update the changelog with the 0.2.0 section and regenerate Cargo.lock.
  • 2e53654b6c Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • Просмотр сравнение для этих 10 коммитов »

1 день назад

cesar запушил(а) feature/subaccount-api в cesar/ledger-rs

  • 25d490cb45 Add subaccount-scoped account and movement API and docs Split the reusable subaccount surface out of the inflight work so it can land on its own. `AccountId` already carries a subaccount dimension (ADR-0012); this exposes the ergonomics and documentation that go with it. `Account::new_ref` builds an account for a specific subaccount reference, with `Account::new` delegating to it for the main subaccount. On the builder, `movement_ref`/`pay_ref` are now the primitives that target a specific subaccount and `movement`/`pay` delegate to them, inverting the previous arrangement. Doc comments on the posting, movement, and account types are clarified to note the subaccount dimension. The dashboard's `account_label` now keys on the base account so a subaccount inherits its base account's label. Docs gain a Subaccounts section in accounts.md (the `{id, sub}` model, per-subaccount balances, `list_subaccounts`) and a corrected IBAN example plus a subaccount note in the glossary.
  • c32e45eca1 Add subaccount-scoped account and movement API and docs Split the reusable subaccount surface out of the inflight work so it can land on its own. `AccountId` already carries a subaccount dimension (ADR-0012); this exposes the ergonomics and documentation that go with it. `Account::new_ref` builds an account for a specific subaccount reference, with `Account::new` delegating to it for the main subaccount. On the builder, `movement_ref`/`pay_ref` are now the primitives that target a specific subaccount and `movement`/`pay` delegate to them, inverting the previous arrangement. Doc comments on the posting, movement, and account types are clarified to note the subaccount dimension. The dashboard's `account_label` now keys on the base account so a subaccount inherits its base account's label. Docs gain a Subaccounts section in accounts.md (the `{id, sub}` model, per-subaccount balances, `list_subaccounts`) and a corrected IBAN example plus a subaccount note in the glossary.
  • feee785d6c Document journaling support and record the framing decision Users asked whether the ledger supports journaling, in particular compound entries that touch more than two accounts. The mechanics already existed (a Transfer is a list of Movements committed atomically, and the transfer log is append-only), but there was no affirmative doc saying so and no ADR capturing the naming/framing choice. This makes the accounting model discoverable and pins down what is and is not the journal. Add doc/journaling.md (single, compound, and multi-asset entries; the transfer log as the journal; auditability by replay) and ADR-0013, which decides to model journaling with existing types rather than adding Journal/JournalEntry. Update the README overview, ADR index, CHANGELOG, CLAUDE.md doc listing, and cross-links from transfers.md and accounting-mapping.md.
  • 72d622e627 Document journaling support and record the framing decision Users asked whether the ledger supports journaling, in particular compound entries that touch more than two accounts. The mechanics already existed (a Transfer is a list of Movements committed atomically, and the transfer log is append-only), but there was no affirmative doc saying so and no ADR capturing the naming/framing choice. This makes the accounting model discoverable and pins down what is and is not the journal. Add doc/journaling.md (single, compound, and multi-asset entries; the transfer log as the journal; auditability by replay) and ADR-0013, which decides to model journaling with existing types rather than adding Journal/JournalEntry. Update the README overview, ADR index, CHANGELOG, CLAUDE.md doc listing, and cross-links from transfers.md and accounting-mapping.md.
  • 0d52129545 Add a subaccount dimension to account identity Some workloads need several distinct balances under one owner (sub-ledgers, earmarks, reservations) that are individually addressable and closed independently, without minting unrelated top-level accounts. This implements ADR-0012: fold a subaccount into account identity so each partition is a full account record with its own policy and lifecycle, while existing behaviour is unchanged when subaccounts are not used. AccountId becomes { id: i64, sub: i64 } where sub = 0 is the main account. Conservation, double-spend, and floor checks are preserved because they already key on the whole AccountId, which now distinguishes subaccounts, so they become per-subaccount with no logic change. Balances are always reported per subaccount and never summed across them; the ledger gains balances() and list_subaccounts(). Aggregate reads take a base id plus an optional subaccount filter, while exact entity operations take the full AccountId. Book membership is scoped by base account. The SQL backend adds a migration that widens the accounts and transfer_accounts primary keys and adds a subaccount column to postings, with existing rows defaulting to the main account. The dashboard surfaces subaccounts in the account list and detail views. For presentation and routing, AccountId gains an IBAN-style string form: two mod-97 check digits followed by a base-36 body, with no country code. Parsing validates the checksum, so a mistyped id is rejected before it reaches the store. To avoid leaking raw ids, the (id, sub) pair is run through a keyed 128-bit Feistel permutation before encoding and inverted on parse, so codes look random and a base account and its subaccount are not visibly related. The permutation key is a global seed with a default, configurable via set_id_seed (the dashboard exposes --id-seed / KUATIA_ID_SEED). This is obfuscation, not security. The string is an edge form only; storage, the Store trait, content hashing, and serde keep the two i64 legs. Finally, make the SQL entity writes safe under concurrency with pessimistic locking. create_account, append_account_version, and create_book each ran a separate existence or version check followed by an insert, a check-then-act TOCTOU where two concurrent callers could both pass the check and then both write. Each now runs in a transaction that locks the relevant rows with SELECT ... FOR UPDATE so a competing writer for the same entity blocks until commit. FOR UPDATE is Postgres-only, so the backend is detected once and the clause is omitted on SQLite, which serializes writers itself; an ON CONFLICT DO NOTHING insert remains as the portable backstop and covers the append phantom-insert a row lock does not. Claude-Session: https://claude.ai/code/session_01SJFJen8Ethv9Q6Ysb1xmz4
  • Просмотр сравнение для этих 14 коммитов »

1 день назад

cesar запушил(а) feature/inflight в cesar/ledger-rs

  • 7f5351a0ac Add inflight holds via per-destination holding subaccounts Callers need to reserve funds for a multi-leg trade without settling it, then confirm it fully or in parts or void it, and to run several such holds against one account at the same time. The ledger is append-only with derived balances, so a hold has to be real committed state, and holds must be attributable to the account they belong to. Model an inflight transaction as the ordinary trade with every destination rewritten to a per-destination holding subaccount (NoOverdraft), committing that rewritten transfer to park the funds. Confirm and void are ordinary commits from the holds to their destinations or back to the funders recorded in the authorize transfer's metadata. Over-confirmation is blocked by the NoOverdraft hold, and concurrent confirmations serialize on the shared holding posting. The inflight facts live in a single CBOR-encoded metadata entry, and confirm accepts a batch of legs built with the existing TransferBuilder.pay interface. A hold reuses the existing account subaccount dimension: it is a subaccount of its destination, keyed by a value derived from a hash of the submitted trade, so different trades derive different subaccounts and a destination hosts many concurrent inflights, while the identical trade collides on its existing hold. Balances are read per subaccount and never summed, so a hold's remaining amount is just its balance. Everything rides the existing commit and recover path, so idempotency, conservation, and crash recovery are inherited, with no new store, sub-trait, or migration. Recorded in ADR-0013 (inflight holds via holding accounts), building on the subaccount dimension from ADR-0012.
  • 8d273d1de1 Add inflight holds via per-destination holding subaccounts Callers need to reserve funds for a multi-leg trade without settling it, then confirm it fully or in parts or void it, and to run several such holds against one account at the same time. The ledger is append-only with derived balances, so a hold has to be real committed state, and holds must be attributable to the account they belong to. Model an inflight transaction as the ordinary trade with every destination rewritten to a per-destination holding subaccount (NoOverdraft), committing that rewritten transfer to park the funds. Confirm and void are ordinary commits from the holds to their destinations or back to the funders recorded in the authorize transfer's metadata. Over-confirmation is blocked by the NoOverdraft hold, and concurrent confirmations serialize on the shared holding posting. The inflight facts live in a single CBOR-encoded metadata entry, and confirm accepts a batch of legs built with the existing TransferBuilder.pay interface. A hold reuses the existing account subaccount dimension: it is a subaccount of its destination, keyed by a value derived from a hash of the submitted trade, so different trades derive different subaccounts and a destination hosts many concurrent inflights, while the identical trade collides on its existing hold. Balances are read per subaccount and never summed, so a hold's remaining amount is just its balance. Everything rides the existing commit and recover path, so idempotency, conservation, and crash recovery are inherited, with no new store, sub-trait, or migration. Recorded in ADR-0013 (inflight holds via holding accounts), building on the subaccount dimension from ADR-0012.
  • ee20cc1f2b Remove the vestigial UserData type and the orphaned withdraw saga step UserData held three fixed-width correlation slots on Account, Transfer, and Envelope, but no caller ever set them: every construction site used UserData::default(). The slots still cost a schema column, a JSON round-trip on every account write, and bytes in the content-addressed hash preimage, so they were pure overhead. Drop the type, its fields, the builder methods, and the accessor. Because the fields were part of the canonical ToBytes serialization for Envelope and Account, removing them changes the hash preimage. Bump CANONICAL_VERSION so the format change is explicit rather than silent, and add migration 003 to drop the accounts.user_data column. The historical 001/002 migrations keep the column so an existing database still upgrades in order. Also delete WithdrawInput and WithdrawMovementStep. They were defined and implemented alongside the pay and deposit saga steps but never referenced, because withdrawals go through TransferBuilder::withdraw() instead. Both slipped past dead-code detection only because they are pub in a library crate.
  • 58b7f23fea Remove the vestigial UserData type and the orphaned withdraw saga step UserData held three fixed-width correlation slots on Account, Transfer, and Envelope, but no caller ever set them: every construction site used UserData::default(). The slots still cost a schema column, a JSON round-trip on every account write, and bytes in the content-addressed hash preimage, so they were pure overhead. Drop the type, its fields, the builder methods, and the accessor. Because the fields were part of the canonical ToBytes serialization for Envelope and Account, removing them changes the hash preimage. Bump CANONICAL_VERSION so the format change is explicit rather than silent, and add migration 003 to drop the accounts.user_data column. The historical 001/002 migrations keep the column so an existing database still upgrades in order. Also delete WithdrawInput and WithdrawMovementStep. They were defined and implemented alongside the pay and deposit saga steps but never referenced, because withdrawals go through TransferBuilder::withdraw() instead. Both slipped past dead-code detection only because they are pub in a library crate.
  • 25d490cb45 Add subaccount-scoped account and movement API and docs Split the reusable subaccount surface out of the inflight work so it can land on its own. `AccountId` already carries a subaccount dimension (ADR-0012); this exposes the ergonomics and documentation that go with it. `Account::new_ref` builds an account for a specific subaccount reference, with `Account::new` delegating to it for the main subaccount. On the builder, `movement_ref`/`pay_ref` are now the primitives that target a specific subaccount and `movement`/`pay` delegate to them, inverting the previous arrangement. Doc comments on the posting, movement, and account types are clarified to note the subaccount dimension. The dashboard's `account_label` now keys on the base account so a subaccount inherits its base account's label. Docs gain a Subaccounts section in accounts.md (the `{id, sub}` model, per-subaccount balances, `list_subaccounts`) and a corrected IBAN example plus a subaccount note in the glossary.
  • Просмотр сравнение для этих 18 коммитов »

1 день назад

cesar запушил(а) feature/dashboard в cesar/ledger-rs

  • 2e53654b6c Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • c9eeff1890 Add a dashboard to observe a Kuatia ledger over HTTP The ledger had no way to inspect its state without writing Rust against the Store trait. This adds kuatia-dashboard, a read-only observer that serves a server-rendered HTML UI (Tera templates with htmx for live refresh) and a JSON REST API under /api built from the same data layer, so the same state can drive both the built-in pages and any richer client someone wants to build. It connects to any supported backend (in-memory or file SQLite, or PostgreSQL) and binds a configurable host and port through CLI flags that fall back to environment variables. Seeding demo data is opt-in via --seed and is a no-op against a database that already holds accounts, so pointing the dashboard at a real, persistent ledger just visualizes it.
  • 2015894107 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • 007e73f2c5 Store SQL columns as hex/JSON text for auditability The SQL backend kept content-addressed ids, receipts, payloads and saga state as raw BLOB/BYTEA. Opening the database showed opaque binary, so a row could not be read or cross-checked against logs without decoding it by hand, which undercuts the ledger's audit story. Move every column to a text type. Content-addressed ids (posting and transfer ids, event dedup keys) and the opaque saga blob are stored as lower-case hex, matching the form already used in Debug output and logs. Structured payloads (account user_data/metadata, transfer and receipt, event and book data) are stored as their JSON serialization. The JSON is never queried into, so text is enough and no binary or indexed column is needed. With the BLOB/BYTEA split gone the DDL is identical for both backends, so the per-backend migration directories collapse into one portable schema and the backend probe in migrate() is dropped. A test asserts the raw columns hold hex ids and JSON text.
  • ee2e0e2b12 Introduce Kuatia, an append-only, auditable multi-asset ledger Kuatia tracks value as signed postings rather than mutable balance fields, so every state change is an immutable record and the full history is auditable. A transfer atomically consumes and creates postings and must conserve value per asset (the sum of consumed equals the sum of created), which is the double-entry safety invariant enforced on every commit. The public surface is intent-based. Callers describe movements (pay/deposit/withdraw) through a builder; the core resolves them into a concrete envelope of postings to consume and create, selecting inputs greedily and computing change or an overdraft posting as the account policy allows. Overdraft behavior is a per-account policy: NoOverdraft forbids negative postings, capped and uncapped variants permit them down to a floor or without bound, and system/external accounts model the ledger boundary. Commits run through a two-step saga, reserve then finalize, with validation as the last thing before the writes, automatic retry, and LIFO compensation. Transfers are content-addressed (a double SHA-256 of their canonical bytes), which gives idempotency and tamper evidence. Storage is deliberately dumb: each write primitive applies one conditional update and returns the number of rows it changed, and the saga owns all interpretation, idempotency, and compensation. Crash safety comes from a phase-tracked write-ahead record plus a recover() that rolls a half-applied commit forward rather than unwinding it. The code separates a pure, sans-IO core from the async layer. The core is deterministic and unit-testable; the async layer adds the Store trait and the saga. Storage backends (in-memory and SQLite/PostgreSQL) share one conformance suite, and concurrency tests pin the guarantees that matter: double-spend prevention is exact, while the overdraft floor re-check is best-effort under concurrency (documented, with conservation preserved). Identifiers are snowflake-style i64 values generated in Rust, never by the database, and are unique across threads. The monetary type hides its backing integer, which is swappable from i64 to i128 at compile time. The repository ships architecture and API docs plus a set of ADRs recording the design decisions.
  • Просмотр сравнение для этих 6 коммитов »

1 день назад

cesar запушил(а) feature/compact-accounts в cesar/ledger-rs

  • ad12b43d5b Extract prelude and envelope_saga into file modules The workspace hygiene rules forbid inline module bodies (mod NAME { ... }) in non-test code so that every namespace is visible in the directory tree. Two modules still used the inline form: the crate prelude and the legend! saga wrapper. Move both into their own files (prelude.rs and ledger/envelope_saga.rs) and switch the declarations to file modules. The prelude doc comment moves to inner form, dropping now-redundant explicit intra-doc link targets that resolve on their own once the re-exports are in scope.
  • 92108a8c4c Extract prelude and envelope_saga into file modules The workspace hygiene rules forbid inline module bodies (mod NAME { ... }) in non-test code so that every namespace is visible in the directory tree. Two modules still used the inline form: the crate prelude and the legend! saga wrapper. Move both into their own files (prelude.rs and ledger/envelope_saga.rs) and switch the declarations to file modules. The prelude doc comment moves to inner form, dropping now-redundant explicit intra-doc link targets that resolve on their own once the re-exports are in scope.
  • 22c5b840cd Shorten the account code to a fixed 20-character form The IBAN-style account code was 28 characters: two leading check digits and a 26-character base-36 body that encoded both i64 legs at full width. That is long to read or speak and does not group evenly, so the code read worse than an IBAN or a card number. Pack the base id (63 bits) and the subaccount (30 bits) into one 93-bit value, run it through a keyed format-preserving permutation (a 94-bit Feistel with cycle-walking, replacing the 128-bit one), and base-36 encode it in 18 characters, then append the two mod-97 check digits. The result is a fixed 20 characters, five groups of four, with the checksum at the tail. The change is presentation-only: ToBytes, serde, the SQL schema, and every content hash keep the two full i64 legs, so there is no migration. The cost is a 30-bit subaccount range, which caps the hash-derived inflight hold subaccounts, so they are now masked to that width. See ADR-0015.
  • e4514148a5 Shorten the account code to a fixed 20-character form The IBAN-style account code was 28 characters: two leading check digits and a 26-character base-36 body that encoded both i64 legs at full width. That is long to read or speak and does not group evenly, so the code read worse than an IBAN or a card number. Pack the base id (63 bits) and the subaccount (30 bits) into one 93-bit value, run it through a keyed format-preserving permutation (a 94-bit Feistel with cycle-walking, replacing the 128-bit one), and base-36 encode it in 18 characters, then append the two mod-97 check digits. The result is a fixed 20 characters, five groups of four, with the checksum at the tail. The change is presentation-only: ToBytes, serde, the SQL schema, and every content hash keep the two full i64 legs, so there is no migration. The cost is a 30-bit subaccount range, which caps the hash-derived inflight hold subaccounts, so they are now masked to that width. See ADR-0015.
  • 7f5351a0ac Add inflight holds via per-destination holding subaccounts Callers need to reserve funds for a multi-leg trade without settling it, then confirm it fully or in parts or void it, and to run several such holds against one account at the same time. The ledger is append-only with derived balances, so a hold has to be real committed state, and holds must be attributable to the account they belong to. Model an inflight transaction as the ordinary trade with every destination rewritten to a per-destination holding subaccount (NoOverdraft), committing that rewritten transfer to park the funds. Confirm and void are ordinary commits from the holds to their destinations or back to the funders recorded in the authorize transfer's metadata. Over-confirmation is blocked by the NoOverdraft hold, and concurrent confirmations serialize on the shared holding posting. The inflight facts live in a single CBOR-encoded metadata entry, and confirm accepts a batch of legs built with the existing TransferBuilder.pay interface. A hold reuses the existing account subaccount dimension: it is a subaccount of its destination, keyed by a value derived from a hash of the submitted trade, so different trades derive different subaccounts and a destination hosts many concurrent inflights, while the identical trade collides on its existing hold. Balances are read per subaccount and never summed, so a hold's remaining amount is just its balance. Everything rides the existing commit and recover path, so idempotency, conservation, and crash recovery are inherited, with no new store, sub-trait, or migration. Recorded in ADR-0013 (inflight holds via holding accounts), building on the subaccount dimension from ADR-0012.
  • Просмотр сравнение для этих 22 коммитов »

1 день назад

cesar запушил(а) refactor/insufficient-funds-error в cesar/ledger-rs

  • 111880f294 Relocate the insufficient-funds error beside the selection code posting_selection.rs was a shallow, mis-named module: a file whose only contents were a one-variant error enum and its trait impls, with no selection logic at all. The actual posting selection (greedy largest-first) lives in posting_resolution.rs, so the module name misdirected a reader looking for where selection happens. The error is also not resolve-internal plumbing. InsufficientFunds is a shared domain leaf with two independent producers: resolve_envelope raises it during selection, and the inflight over-confirm guard constructs it directly without any resolution. LedgerError flattens it into a public Selection variant. Folding it into ResolveError would bury a shared error under a type one of its producers never creates. Collapse the one-variant enum to a struct (selection has exactly one failure mode) and move it next to the code that raises it, in posting_resolution.rs. Drop the two From impls that were never exercised. The Selection variant names and the public LedgerError surface are unchanged apart from the payload type.
  • e8859de701 Squashed commit of the following: commit 32f50f7ac6d96c1591e7b307199cd5d63a76b808 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Sat Jul 18 21:21:23 2026 -0300 Make account lifecycle transitions crash-safe freeze, unfreeze, and close each append a new account version and then append the matching lifecycle event as two separate store writes with no shared transaction. A crash between them left a durable version bump with no event, and nothing repaired it. This is the same window recover() already closes for store_transfer followed by append_event on the commit path, left open for account transitions. Route the transitions through that existing write-ahead / repair path. The three methods, previously near-identical copies of the same five-step shape, collapse into one transition primitive parameterized by the flag mutation and the event. It persists a PendingTransition write-ahead record before either write; recover() rolls it forward, appending the version only when it is not yet present and re-appending the event. Rolling the event forward requires it to be idempotent, which lifecycle events were not: event_dedup_key returned None for them, so a second append duplicated the row. Give the three transition events a version field and key event_dedup_key on (account, version). The key type generalizes to a string; the transfer form stays the lowercase hex the dedup_key column already holds, so no migration is needed, and the field is serde-default so existing event rows still load. The write-ahead records now share one tagged PendingRecord enum so recover() dispatches an envelope commit saga and an account transition to their own completion paths. See ADR-0019.
  • dc818a6246 Squashed commit of the following: commit 631a6e9e4241bd4995f6152f5881741e76b8b2d5 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Sun Jul 19 13:29:27 2026 -0300 Coordinate balance-projection appends through storage, not memory Address a review of ADR-0019. The background cache-point append had no dedup, so a hot account read at high QPS spawned one full fold per read. An in-process guard cannot coordinate the multiple independent instances the projection is built for, so move the guard into the shared balance_projection rows: append_cache_point returns before the fold when a cache point already sits within a debounce window below the target watermark. Redundant spawns then cost one indexed read, and the dedup holds across processes and restarts with no lease, lock, or CAS, matching the coordination-free design. Log a warning when a background append fails so a projection that silently stops advancing is visible. Add a deterministic test that folds a non-empty tail onto a real snapshot, the watermark-boundary case the existing tests never reached (they either created no snapshot or left the tail empty). Correct the property-test comment that claimed cache points were appended during the run, and the ADR header that still named the rejected lease and projector service. commit b169e1f084375cd6a9658375cfb046ab7019a3f9 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Fri Jul 17 09:20:35 2026 -0300 Add cached balance projection (ADR-0019) Balance was summed from every live posting on each read, which is O(N live postings) and unbounded under UTXO fragmentation. Add a rebuildable balance projection: a read returns the closest stored snapshot plus a Rust-summed tail of committed transfer deltas past the snapshot's commit-time watermark. Summing every one of an account's transfer created(+)/consumed(-) deltas for an asset equals its live-posting sum, so the projection-aware read always agrees with the authoritative balance, and it is faster once a snapshot keeps the tail short. Make the projection append-only rather than a mutable row guarded by a lock or a CAS. Each cache point is one snapshot of a (account, subaccount, asset) balance plus the watermark it covers, tagged with a Rust-minted monotonic id, and rows are only ever inserted. A read selects the cache point closest to (at or before) the target time (largest watermark not exceeding it, tie-broken by highest id), so a stale or duplicate append is harmless and concurrent appends need no coordination. Add a BalanceProjectionStore sub-trait (append + closest-at-or-before get) with an in-memory and SQL implementation (migration 007, balance stored as TEXT like every other monetary column) and conformance tests. Trigger snapshots lazily off the read path, not on commit. A read whose folded tail has accrued at least snapshot-interval new credits/debits since the closest cache point spawns a best-effort background append; a read that finds no cache point returns the authoritative live sum directly and bootstraps one in the background. The append folds only up to now minus a grace window, so a snapshot never captures a commit still racing to become visible. Both bounds are tunable (with_snapshot_interval, with_projection_grace_ms) with sane defaults. Commit itself never touches the projection, keeping the write path unchanged; correctness never depends on any of this. Validation keeps reading the authoritative live sum (compute_balance, now public), so the UTXO reservation protocol remains the exact concurrency control for no-overdraft accounts and the projection stays a pure read accelerator.
  • 651a7dd5d6 Squashed commit of the following: commit 2d7deddcb63e91946f64c746484dd19916de0007 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Thu Jul 16 12:56:10 2026 -0300 Address review: assert flags round-trip, bump canonical version Add a store-conformance assertion that an account's flags survive a create/get round-trip, guarding the SQL flags-column mapping now that the balance constraint lives there. Bump CANONICAL_VERSION 4 -> 5: removing the policy field from the Account preimage changes account snapshot hashes, following the same convention used when UserData was removed. Record the version bump and the former-system-account over-debit behavior change in ADR-0018. commit fd37985682fe8b785c3c8796b78203d00ddbb471 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Thu Jul 16 09:05:52 2026 -0300 Collapse account policy into a single overdraft flag The five-variant AccountPolicy enum only ever enforced one real distinction: may an account's balance go negative or not? The capped floor, the System/External labels, and the Uncapped variant all resolved to the same runtime behavior (overdraft allowed, no floor) while carrying a serialized enum, a SQL column, resolve/validate match arms, and a dashboard DTO. Replace it with one AccountFlags bit, DEBIT_MUST_NOT_EXCEED_CREDIT. Overdraft is allowed by default: a shortfall becomes a negative offset posting and the transfer records as long as it conserves value per asset. Setting the flag forbids a negative balance and negative postings. The capped credit-line floor is dropped; a specific limit is now an application concern. Drop the policy column via migration 006. Validation and resolution branch on Account::forbids_overdraft() instead of a policy match; resolve takes the set of overdraft-permitting accounts. Rewrite the floor tests and turn the ignored write-skew test into a real conservation-under- concurrency test. ADR-0018 records the decision and supersedes ADR-0004.
  • 5a8bfe0035 Lift the transfer filter and page cut above the Store seam query_transfers and query_postings had copied the same filter closure and the same total/skip/take pagination tail on both sides of the Store trait. Worse, the two query_transfers adapters answered an account-less query differently behind one signature: the in-memory default returned an error while SQL ran a store-wide scan, a divergence no conformance test held them to. The dashboard already relied on the scan (it queries with a default, account-less filter), so the in-memory backend would have failed there. Introduce a query module in kuatia-storage that states the contract once: filter_transfers for the time-window and book predicates, and paginate for the total plus offset/limit cut. Each backend now only loads its candidate records and hands them to the shared code; SQL keeps its genuine LIMIT push-down for postings. query_transfers loses its trait default so both backends must implement the load, and both now scan store-wide when no account is given. A new conformance test pins that shared answer.

1 день назад

cesar создал новую ветку refactor/insufficient-funds-error в cesar/ledger-rs

1 день назад

cesar запушил(а) master в cesar/ledger-rs

  • e8859de701 Squashed commit of the following: commit 32f50f7ac6d96c1591e7b307199cd5d63a76b808 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Sat Jul 18 21:21:23 2026 -0300 Make account lifecycle transitions crash-safe freeze, unfreeze, and close each append a new account version and then append the matching lifecycle event as two separate store writes with no shared transaction. A crash between them left a durable version bump with no event, and nothing repaired it. This is the same window recover() already closes for store_transfer followed by append_event on the commit path, left open for account transitions. Route the transitions through that existing write-ahead / repair path. The three methods, previously near-identical copies of the same five-step shape, collapse into one transition primitive parameterized by the flag mutation and the event. It persists a PendingTransition write-ahead record before either write; recover() rolls it forward, appending the version only when it is not yet present and re-appending the event. Rolling the event forward requires it to be idempotent, which lifecycle events were not: event_dedup_key returned None for them, so a second append duplicated the row. Give the three transition events a version field and key event_dedup_key on (account, version). The key type generalizes to a string; the transfer form stays the lowercase hex the dedup_key column already holds, so no migration is needed, and the field is serde-default so existing event rows still load. The write-ahead records now share one tagged PendingRecord enum so recover() dispatches an envelope commit saga and an account transition to their own completion paths. See ADR-0019.

5 дней назад

cesar запушил(а) master в cesar/ledger-rs

  • dc818a6246 Squashed commit of the following: commit 631a6e9e4241bd4995f6152f5881741e76b8b2d5 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Sun Jul 19 13:29:27 2026 -0300 Coordinate balance-projection appends through storage, not memory Address a review of ADR-0019. The background cache-point append had no dedup, so a hot account read at high QPS spawned one full fold per read. An in-process guard cannot coordinate the multiple independent instances the projection is built for, so move the guard into the shared balance_projection rows: append_cache_point returns before the fold when a cache point already sits within a debounce window below the target watermark. Redundant spawns then cost one indexed read, and the dedup holds across processes and restarts with no lease, lock, or CAS, matching the coordination-free design. Log a warning when a background append fails so a projection that silently stops advancing is visible. Add a deterministic test that folds a non-empty tail onto a real snapshot, the watermark-boundary case the existing tests never reached (they either created no snapshot or left the tail empty). Correct the property-test comment that claimed cache points were appended during the run, and the ADR header that still named the rejected lease and projector service. commit b169e1f084375cd6a9658375cfb046ab7019a3f9 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Fri Jul 17 09:20:35 2026 -0300 Add cached balance projection (ADR-0019) Balance was summed from every live posting on each read, which is O(N live postings) and unbounded under UTXO fragmentation. Add a rebuildable balance projection: a read returns the closest stored snapshot plus a Rust-summed tail of committed transfer deltas past the snapshot's commit-time watermark. Summing every one of an account's transfer created(+)/consumed(-) deltas for an asset equals its live-posting sum, so the projection-aware read always agrees with the authoritative balance, and it is faster once a snapshot keeps the tail short. Make the projection append-only rather than a mutable row guarded by a lock or a CAS. Each cache point is one snapshot of a (account, subaccount, asset) balance plus the watermark it covers, tagged with a Rust-minted monotonic id, and rows are only ever inserted. A read selects the cache point closest to (at or before) the target time (largest watermark not exceeding it, tie-broken by highest id), so a stale or duplicate append is harmless and concurrent appends need no coordination. Add a BalanceProjectionStore sub-trait (append + closest-at-or-before get) with an in-memory and SQL implementation (migration 007, balance stored as TEXT like every other monetary column) and conformance tests. Trigger snapshots lazily off the read path, not on commit. A read whose folded tail has accrued at least snapshot-interval new credits/debits since the closest cache point spawns a best-effort background append; a read that finds no cache point returns the authoritative live sum directly and bootstraps one in the background. The append folds only up to now minus a grace window, so a snapshot never captures a commit still racing to become visible. Both bounds are tunable (with_snapshot_interval, with_projection_grace_ms) with sane defaults. Commit itself never touches the projection, keeping the write path unchanged; correctness never depends on any of this. Validation keeps reading the authoritative live sum (compute_balance, now public), so the UTXO reservation protocol remains the exact concurrency control for no-overdraft accounts and the projection stays a pure read accelerator.

5 дней назад

cesar запушил(а) unify-store-transfer-involved в cesar/ledger-rs

  • 382812cdf1 Make both stores honor the store_transfer involved set verbatim The store_transfer(record, involved) contract diverged between backends. SqlStore trusted the caller's involved set and wrote one transfer_accounts row per account; InMemoryStore ignored the parameter and re-derived participation from stored postings at read time. They agreed only when the passed set matched what the postings implied, and the conformance helper passed only created owners, so the consumed-owner branch was never checked differentially. A saga that passed a wrong involved set would look correct in the fast in-memory suite yet be wrong on SQL, silently. Dropping the parameter is not viable: consumed-posting owners live nowhere in EnvelopeRecord (the envelope carries consumed PostingIds only, and the account snapshots cover created owners), and SQL keeps those ids inside an opaque JSON blob it cannot join on. So make both backends follow the same instruction instead. InMemoryStore now records the involved set in an explicit transfer_accounts index and resolves get_transfers_for_account from it, mirroring the SQL table; it derives nothing from postings. The trait doc states the contract, and a new conformance test indexes a consumed-posting owner distinct from every created owner (its posting never seeded) so the transfer is retrievable only if the backend trusts involved, forcing both adapters to agree.
  • 50d603c883 Name every AccountFlags bit so the full flag space round-trips Only 11 of AccountFlags' 32 bits had named constants, so the reserved system bits (3-7) and the upper half of the user space (16-31) were unreachable: BookPolicy::allowed_flags could scope at most eight user flags, and the SQL read path's from_bits_truncate silently dropped any set bit without a constant, meaning those bits could never survive a storage round-trip. Give every bit a name. Bits 3-7 become RESERVED_3..RESERVED_7 (held for future system flags, not user assignment) and the user range extends with USER_8..USER_23 to cover bits 16-31. The backing type stays u32, so the canonical hashing bytes and account hashes are unchanged and no SQL migration is needed; the existing bits() as i32 / from_bits_truncate casts round-trip bit 31 correctly in Rust, which a new test pins.
  • cb49d9c434 Co-locate the canonical-bytes contract so the hash preimage is auditable The content-addressing contract (what bytes get hashed, in what order, and under which version byte) is the definition EnvelopeId and the account snapshot hashes depend on. It was split across three regions of the types crate: the ToBytes trait, CANONICAL_VERSION, and the write helpers near the top; the Cent impl in the middle; and the other fifteen impls at the bottom. Auditing "is every field folded in, in the right order" meant holding three windows open, which is exactly the bug class the CANONICAL_VERSION comments track. Move the whole contract into a canonical module: the trait, the version byte, the big-endian write helpers, and every impl ToBytes, in that order, with a module doc that states the encoding rules. The preimage surface is now visible at once. The public API is unchanged; lib.rs re-exports ToBytes, CANONICAL_VERSION, and the write helpers, so downstream paths still resolve.
  • be55820f6d Split the Ledger god-object into concern-named submodules The async Ledger had grown to a single ~900-line file carrying about ten responsibilities behind one Arc<dyn Store> field. The write-ahead saga/commit engine, the invariant code that a ledger fundamentally is, read as one method among thirty, physically wedged between account lifecycle, balance queries, and thin store pass-throughs. Group the methods into sibling file-modules by concern so the deep engine stands on its own: commit (resolve/commit/reverse/recover/finalize plus the write-ahead saga types), lifecycle (create/freeze/unfreeze/close), balance (per-subaccount balances), and query (the read-only Store delegations). The struct, its public API, and every method signature are unchanged; this only moves code and re-exports the two public types the submodules now own. Keep the query pass-throughs rather than deleting them: they have live callers across the dashboard, examples, and tests that depend on the LedgerError return type, so dropping them would break those callers and downgrade them to StoreError for no structural gain.
  • f142d6c622 Give the ledger a live-postings primitive so inflight stops reaching into storage ADR-0014 promises the inflight layer is a thin veneer over Ledger methods (commit, create_account, get_transfer, balance, close). In practice close_if_drained punched through that seam to store().get_postings_by_account(.., Live), re-implementing byte-for-byte the emptiness probe Ledger::close already runs, because no Ledger method answered "does this account have live postings?". Surface that missing primitive as Ledger::has_live_postings and route both close and close_if_drained through it. The inflight layer no longer names PostingFilter or touches the store, so the seam the ADR describes is real again. Separately, confirm_all, void, and inflight_status each copied the same per-hold/per-asset traversal (holds_of + destination_of + assets_of). Factor that into one pure group_holds helper returning a HoldGroup per hold, so a change to how holds are walked touches a single site and the void-distribution arithmetic is reachable from one place. Line ordering is preserved: the grouping yields sorted (hold, asset) pairs, matching the prior BTreeMap order.

5 дней назад

cesar создал новую ветку unify-store-transfer-involved в cesar/ledger-rs

5 дней назад

cesar запушил(а) review/append-only-hot-indexes-followups в cesar/ledger-rs

  • 1d2cd7fc82 Chunk id-batch posting primitives to a safe statement size The id-batch primitives (get_postings, get_posting_states, reserve, release, deactivate) matched every id in one statement via an OR of equality pairs. That OR chain grows with the batch, and SQLite rejects it once the expression tree passes its depth limit (default 1000), well before the bind-parameter limits are reached; PostgreSQL has its own ceilings too. A caller passing a large id set would hit a hard database error. Split every id batch into fixed-size chunks, keeping the two write primitives' chunks inside their existing single transaction so the claim or release stays atomic, and summing the affected-row counts. Reads accumulate across chunks. The batch size the primitives accept now has no practical ceiling. Also encode hex without a per-byte allocation, and add a test that drives a batch larger than one chunk through reserve, read, and deactivate.
  • 41ce95cb4d Make posting reads deterministic and migrations atomic Reviewing the append-only value-table / hot-index split surfaced three follow-ups worth fixing. Migrations ran statement-by-statement outside any transaction, recording a migration as applied only after all of its statements succeeded. The new migration that drops and rebuilds the postings table could therefore crash half-applied, leaving the schema in a state the migration could not be re-run against. Wrap each migration's statements and its bookkeeping insert in one transaction so a crash rolls back cleanly and the migration is retried whole. Both SQLite and PostgreSQL support transactional DDL. Posting reads returned rows in an unspecified order, so LIMIT/OFFSET pagination could skip or repeat rows across pages, worst for the live set whose source is a UNION of two tables. Order get_postings_by_account by the posting id in both backends so the primitive that balance, selection, close, and pagination all build on returns a stable sequence. Bring the reference docs back in line with the derived-state model: postings are immutable and carry no lifecycle column; state is Active, Reserved, Spent, or Missing derived from index membership.
  • 288392181a Separate append-only value tables from disposable hot indexes The primary goal is correctness. Value and audit data lives only in append-only tables that are inserted into and never updated or deleted, so no code path or credential can corrupt or lose history. `postings` is the immutable record of every posting; `accounts` is the immutable log of every account version. Fast access is served by separate disposable tables that behave like indexes over that truth and can be dropped and rebuilt from it: `active_postings` and `reserved_postings` hold the spendable and in-flight set, and `account_head` points at each account's current version. A posting's lifecycle state and an account's current version are derived from membership in these tables, not from a mutable column. Every write is an INSERT or a DELETE; nothing issues an UPDATE. That shrinks the margin for error and is enforceable with database grants: the ledger role needs INSERT on the value tables, INSERT and DELETE on the hot tables, and UPDATE on nothing. The hot tables carry full row copies of the live set, so reads hit them directly without joining back to the value tables, and the reserve, release, and consume primitives move the whole input set with bounded set-based statements instead of per-row loops. See ADR-0016 and ADR-0017.
  • e56b34bc04 Squashed commit of the following: commit 92108a8c4ca6f53970c4a47719993a4ee162ace8 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Fri Jul 10 19:04:35 2026 -0300 Extract prelude and envelope_saga into file modules The workspace hygiene rules forbid inline module bodies (mod NAME { ... }) in non-test code so that every namespace is visible in the directory tree. Two modules still used the inline form: the crate prelude and the legend! saga wrapper. Move both into their own files (prelude.rs and ledger/envelope_saga.rs) and switch the declarations to file modules. The prelude doc comment moves to inner form, dropping now-redundant explicit intra-doc link targets that resolve on their own once the re-exports are in scope. commit e4514148a5b0bb1d476c4a85390b202ed11793e8 Author: Cesar Rodas <cesar@rodasm.com.py> Date: Thu Jul 9 18:18:00 2026 -0300 Shorten the account code to a fixed 20-character form The IBAN-style account code was 28 characters: two leading check digits and a 26-character base-36 body that encoded both i64 legs at full width. That is long to read or speak and does not group evenly, so the code read worse than an IBAN or a card number. Pack the base id (63 bits) and the subaccount (30 bits) into one 93-bit value, run it through a keyed format-preserving permutation (a 94-bit Feistel with cycle-walking, replacing the 128-bit one), and base-36 encode it in 18 characters, then append the two mod-97 check digits. The result is a fixed 20 characters, five groups of four, with the checksum at the tail. The change is presentation-only: ToBytes, serde, the SQL schema, and every content hash keep the two full i64 legs, so there is no migration. The cost is a 30-bit subaccount range, which caps the hash-derived inflight hold subaccounts, so they are now masked to that width. See ADR-0015.
  • 3d245083cd Add inflight holds via per-destination holding subaccounts Callers need to reserve funds for a multi-leg trade without settling it, then confirm it fully or in parts or void it, and to run several such holds against one account at the same time. The ledger is append-only with derived balances, so a hold has to be real committed state, and holds must be attributable to the account they belong to. Model an inflight transaction as the ordinary trade with every destination rewritten to a per-destination holding subaccount (NoOverdraft), committing that rewritten transfer to park the funds. Confirm and void are ordinary commits from the holds to their destinations or back to the funders recorded in the authorize transfer's metadata. Over-confirmation is blocked by the NoOverdraft hold, and concurrent confirmations serialize on the shared holding posting. The inflight facts live in a single CBOR-encoded metadata entry, and confirm accepts a batch of legs built with the existing TransferBuilder.pay interface. A hold reuses the existing account subaccount dimension: it is a subaccount of its destination, keyed by a value derived from a hash of the submitted trade, so different trades derive different subaccounts and a destination hosts many concurrent inflights, while the identical trade collides on its existing hold. Balances are read per subaccount and never summed, so a hold's remaining amount is just its balance. Everything rides the existing commit and recover path, so idempotency, conservation, and crash recovery are inherited, with no new store, sub-trait, or migration. Recorded in ADR-0013 (inflight holds via holding accounts), building on the subaccount dimension from ADR-0012.

5 дней назад

cesar создал новую ветку review/append-only-hot-indexes-followups в cesar/ledger-rs

5 дней назад

cesar запушил(а) refactor/split-ledger-modules в cesar/ledger-rs

  • be55820f6d Split the Ledger god-object into concern-named submodules The async Ledger had grown to a single ~900-line file carrying about ten responsibilities behind one Arc<dyn Store> field. The write-ahead saga/commit engine, the invariant code that a ledger fundamentally is, read as one method among thirty, physically wedged between account lifecycle, balance queries, and thin store pass-throughs. Group the methods into sibling file-modules by concern so the deep engine stands on its own: commit (resolve/commit/reverse/recover/finalize plus the write-ahead saga types), lifecycle (create/freeze/unfreeze/close), balance (per-subaccount balances), and query (the read-only Store delegations). The struct, its public API, and every method signature are unchanged; this only moves code and re-exports the two public types the submodules now own. Keep the query pass-throughs rather than deleting them: they have live callers across the dashboard, examples, and tests that depend on the LedgerError return type, so dropping them would break those callers and downgrade them to StoreError for no structural gain.
  • f142d6c622 Give the ledger a live-postings primitive so inflight stops reaching into storage ADR-0014 promises the inflight layer is a thin veneer over Ledger methods (commit, create_account, get_transfer, balance, close). In practice close_if_drained punched through that seam to store().get_postings_by_account(.., Live), re-implementing byte-for-byte the emptiness probe Ledger::close already runs, because no Ledger method answered "does this account have live postings?". Surface that missing primitive as Ledger::has_live_postings and route both close and close_if_drained through it. The inflight layer no longer names PostingFilter or touches the store, so the seam the ADR describes is real again. Separately, confirm_all, void, and inflight_status each copied the same per-hold/per-asset traversal (holds_of + destination_of + assets_of). Factor that into one pure group_holds helper returning a HoldGroup per hold, so a change to how holds are walked touches a single site and the void-distribution arithmetic is reachable from one place. Line ordering is preserved: the grouping yields sorted (hold, asset) pairs, matching the prior BTreeMap order.
  • 51e7b56331 Make intent resolution pure and preserve typed errors across the saga Two changes to leverage the pure-core/async split the codebase already uses for validation. Extract the intent resolve algorithm into kuatia-core as a sibling of validate_and_plan. resolve held the real intent-layer logic (net-debit aggregation, greedy selection with change, and the overdraft-shortfall offset-posting branch) welded to store reads, so its most interesting paths were only reachable by standing up a full async Ledger and committing real deposits. Split it into two sans-IO passes: draft_movements aggregates movements into output postings and per-account net debits, telling the async layer exactly what to load; resolve_envelope selects postings, computes change, and covers an overdraft shortfall. Ledger::resolve now just loads per-debit state between the two. The change-making and shortfall branches gain direct unit tests. Carry the typed LedgerError across the legend saga seam instead of a stringified round-trip. The seam converted LedgerError to a String-only SagaError and back to Store(Internal), so an OverdraftExceeded, AccountFrozen, or InsufficientFunds detected during commit reached the caller as an internal storage fault, and the typed variants callers branch on were unreachable through the commit path. The macro path used here only requires the step error be Send + Sync + Clone (the ledger never serializes the legend Execution; it has its own PendingSaga WAL), so LedgerError becomes the step error type directly and StoreError and LedgerError gain Clone. Genuine plumbing faults map to Store(Internal) via a small helper.
  • 6ab091b493 Extract the AccountId IBAN codec into its own file-module The types crate mixed the domain vocabulary (Book, Posting, Envelope, Account) with the machinery of the account-code string form: bit-packing, a base-36 body, ISO 7064 mod-97 check digits, and a keyed Feistel permutation with cycle-walking. About a third of the file was cryptographic helpers, so the types a reader comes for were buried under them. Move the whole codec cluster to account_code.rs behind its real interface: Display, FromStr, and to_grouped on AccountId, plus ParseAccountIdError. The internals (pack/unpack, base36, mod97, the Feistel functions, obfuscate/deobfuscate, the seed static) become module-private; only the bit-width constants and the seed controls stay public, re-exported from the crate root so the existing API and the kuatia_core re-export chain are unchanged. The codec tests move with the module, which puts the suite on the Display/FromStr seam. No behavior change: the fixed golden vector still round-trips.
  • edbcc9d619 Squashed commit of the following: commit 1d2cd7fc82e1de741cf51c50b2d35d337c476aed Author: Cesar Rodas <cesar@rodasm.com.py> Date: Tue Jul 14 12:20:13 2026 -0300 Chunk id-batch posting primitives to a safe statement size The id-batch primitives (get_postings, get_posting_states, reserve, release, deactivate) matched every id in one statement via an OR of equality pairs. That OR chain grows with the batch, and SQLite rejects it once the expression tree passes its depth limit (default 1000), well before the bind-parameter limits are reached; PostgreSQL has its own ceilings too. A caller passing a large id set would hit a hard database error. Split every id batch into fixed-size chunks, keeping the two write primitives' chunks inside their existing single transaction so the claim or release stays atomic, and summing the affected-row counts. Reads accumulate across chunks. The batch size the primitives accept now has no practical ceiling. Also encode hex without a per-byte allocation, and add a test that drives a batch larger than one chunk through reserve, read, and deactivate. commit 41ce95cb4de2b02e23fd1494b8199f46b46e0c7a Author: Cesar Rodas <cesar@rodasm.com.py> Date: Tue Jul 14 12:13:04 2026 -0300 Make posting reads deterministic and migrations atomic Reviewing the append-only value-table / hot-index split surfaced three follow-ups worth fixing. Migrations ran statement-by-statement outside any transaction, recording a migration as applied only after all of its statements succeeded. The new migration that drops and rebuilds the postings table could therefore crash half-applied, leaving the schema in a state the migration could not be re-run against. Wrap each migration's statements and its bookkeeping insert in one transaction so a crash rolls back cleanly and the migration is retried whole. Both SQLite and PostgreSQL support transactional DDL. Posting reads returned rows in an unspecified order, so LIMIT/OFFSET pagination could skip or repeat rows across pages, worst for the live set whose source is a UNION of two tables. Order get_postings_by_account by the posting id in both backends so the primitive that balance, selection, close, and pagination all build on returns a stable sequence. Bring the reference docs back in line with the derived-state model: postings are immutable and carry no lifecycle column; state is Active, Reserved, Spent, or Missing derived from index membership.

5 дней назад