Skip to content
Open
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
20b20e7
fix(platform-wallet-storage): rehydrate core_derived_addresses from p…
lklimek Jun 9, 2026
4f432c9
fix(platform-wallet-storage): repair partial-state derived-address re…
lklimek Jun 9, 2026
67d0eba
fix(platform-wallet-storage): harden core_derived_addresses with BIP3…
lklimek Jun 10, 2026
ff1208a
test(platform-wallet-storage): close core_derived_addresses coverage …
lklimek Jun 10, 2026
d8d2239
fix(platform-wallet-storage): fail loud when a pool-declared address …
lklimek Jun 11, 2026
cccd217
test(platform-wallet-storage): assert resolved account_index of the s…
lklimek Jun 11, 2026
63fea7a
fix(platform-wallet): emit in-band pool snapshot on derivation so acc…
lklimek Jun 11, 2026
f7b6136
test(platform-wallet-storage): prove in-band pool-snapshot resolution…
lklimek Jun 11, 2026
ebb4b30
refactor(platform-wallet-storage): replace pool mirror/reconcile with…
lklimek Jun 11, 2026
925dfcb
Merge branch 'merge/wallet-rehydration' into merge/wallet-core-derived
lklimek Jun 15, 2026
156bd98
fix(platform-wallet-storage): add account_index to core_derived_addre…
lklimek Jun 16, 2026
925b109
fix(platform-wallet-storage): repair label-split fallout in pool_type…
lklimek Jun 16, 2026
b450649
fix(platform-wallet): add background_generation guard to PlatformAddr…
lklimek Jun 18, 2026
1f3ea29
fix(platform-wallet): close shielded_sync generation-guard TOCTOU (lo…
lklimek Jun 18, 2026
fa4584d
docs(platform-wallet-storage): update stale doc comment on ACCOUNT_IN…
lklimek Jun 22, 2026
aed5652
docs(platform-wallet): correct CHECK-column count and port generation…
lklimek Jun 22, 2026
ca68690
test(platform-wallet): cover generation-guard restart and contacts.st…
lklimek Jun 22, 2026
8d8724e
refactor(platform-wallet)!: hardcode core UTXO account_index=0; retir…
lklimek Jun 22, 2026
e8308ed
fix(platform-wallet): persist non-default-account UTXOs under index 0…
lklimek Jun 22, 2026
ea0082e
docs(platform-wallet-storage): drop deleted-table refs from accounts.…
lklimek Jun 22, 2026
4e4b38c
docs(platform-wallet-storage): sync SCHEMA.md and stale code comments…
lklimek Jun 22, 2026
e553015
fix(platform-wallet): extend AddressPool depth on rehydration to cove…
lklimek Jun 22, 2026
bef9bda
fix(platform-wallet): bound rehydration pool-extension per-chain and …
lklimek Jun 22, 2026
4f83a07
docs(platform-wallet): document apply_persisted_core_state manifest p…
lklimek Jun 22, 2026
bde7060
Merge branch 'mb/cascade-work' into mb/cascade-3828
lklimek Jun 23, 2026
eb59f7c
fix(rs-platform-wallet): harden core-state rehydration + tidy storage…
lklimek Jun 23, 2026
e10f91d
fix(platform-wallet): report unspent UTXO count in RehydrationTopolog…
lklimek Jun 24, 2026
5367471
feat(platform-wallet)!: layered opt-in secret protection (Tier-2 pass…
Claudius-Maginificent Jun 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 32 additions & 8 deletions packages/rs-platform-wallet-storage/SCHEMA.md
Original file line number Diff line number Diff line change
Expand Up @@ -105,9 +105,10 @@ erDiagram
CORE_DERIVED_ADDRESSES {
BLOB wallet_id PK
TEXT account_type PK
TEXT address PK "bech32 / Base58 address string"
INTEGER account_index
TEXT derivation_path "pool_type/derivation_index"
TEXT pool_type PK "external | internal | absent | absent_hardened"
INTEGER derivation_index PK
INTEGER account_index "account-level context; the value the read returns"
TEXT address UK "bech32 / Base58 address string"
INTEGER used "0 | 1"
}

Expand Down Expand Up @@ -403,12 +404,34 @@ finalized. Rows are removed when the transaction becomes confirmed.

### `core_derived_addresses`

Address-to-account-index map. Written before UTXOs in the same
transaction so the UTXO writer can resolve `account_index` by address.

- PK: `(wallet_id, account_type, address)`.
Address-to-account-index map the UTXO writer joins to resolve a UTXO's
`account_index` by address. Populated from three sources, all routed
through the shared `core_state::upsert_derived_address_row` helper so the
rows are identical regardless of origin:

1. **Live `addresses_derived` events** — written before UTXOs in the same
transaction so the writer sees fresh rows.
2. **`apply_pools` registration mirror** — every pool-snapshot address is
mirrored here at registration, so a UTXO landing on a registered
address resolves even before its live derive event arrives
(genesis-rescan).
3. **Load-time reconcile** — on load, pool snapshots fill any address the
table is missing, purely additively (`INSERT OR IGNORE`), so an
authoritative live/mirrored row is never overwritten and a would-be
`UNIQUE(address)` collision is skipped rather than aborting the load.

An unspent UTXO whose address is absent from this table cannot resolve an
account; the writer **skips** it (with a `warn`) rather than erroring, so
one unresolvable row never aborts a whole flush. Its balance re-warms once
the address is later derived.

- PK: `(wallet_id, account_type, pool_type, derivation_index)` — the BIP32
leaf identity (one row per derived address).
- `UNIQUE(wallet_id, address)` — the read-index invariant (one
account_index per address); its index also backs the address lookup, so
no separate index is needed. `address` is a derived attribute, never a
key, so every collision surfaces loud.
- FK: `wallet_id → wallets(wallet_id) ON DELETE CASCADE`.
- Index: `idx_core_derived_addresses_addr(wallet_id, address)`.

### `core_sync_state`

Expand Down Expand Up @@ -604,6 +627,7 @@ fifth (`contacts.state`) is a synthetic lifecycle label naming which
| `account_address_pools` | `account_type` | `sqlite::schema::accounts::ACCOUNT_TYPE_LABELS` |
| `account_address_pools` | `pool_type` | `sqlite::schema::accounts::POOL_TYPE_LABELS` |
| `core_derived_addresses` | `account_type` | `sqlite::schema::accounts::ACCOUNT_TYPE_LABELS` |
| `core_derived_addresses` | `pool_type` | `sqlite::schema::accounts::POOL_TYPE_LABELS` |
| `asset_locks` | `status` | `sqlite::schema::asset_locks::ASSET_LOCK_STATUS_LABELS` |
| `contacts` | `state` | `sqlite::schema::contacts::CONTACT_STATE_LABELS` |

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -147,15 +147,20 @@ CREATE TABLE core_derived_addresses (
wallet_id BLOB NOT NULL,
account_type TEXT NOT NULL CHECK (account_type IN {account_type_check}),
account_index INTEGER NOT NULL,
pool_type TEXT NOT NULL CHECK (pool_type IN {pool_type_check}),
derivation_index INTEGER NOT NULL,
address TEXT NOT NULL,
derivation_path TEXT NOT NULL,
used INTEGER NOT NULL,
PRIMARY KEY (wallet_id, account_type, address),
-- PK is the BIP32 leaf identity. `address` is a derived attribute, not
-- a key, so every collision (within- or cross-pool) trips
-- UNIQUE(address) loud. `account_index` is account-level context (the
-- value the read returns), not a uniqueness discriminator. The UNIQUE
-- index also backs ACCOUNT_INDEX_BY_ADDRESS_SQL.
PRIMARY KEY (wallet_id, account_type, pool_type, derivation_index),
UNIQUE (wallet_id, address),
FOREIGN KEY (wallet_id) REFERENCES wallets(wallet_id) ON DELETE CASCADE
);
Comment thread
Claudius-Maginificent marked this conversation as resolved.
Outdated
Comment thread
Claudius-Maginificent marked this conversation as resolved.
Outdated

CREATE INDEX idx_core_derived_addresses_addr ON core_derived_addresses(wallet_id, address);

CREATE TABLE core_sync_state (
wallet_id BLOB NOT NULL PRIMARY KEY,
last_processed_height INTEGER,
Expand Down
8 changes: 5 additions & 3 deletions packages/rs-platform-wallet-storage/src/sqlite/error.rs
Original file line number Diff line number Diff line change
Expand Up @@ -219,9 +219,11 @@ pub enum WalletStorageError {
},

/// An unspent UTXO named an address absent from
/// `core_derived_addresses`, so its account index can't be resolved;
/// persisting it would mis-file live funds, so the write is refused.
/// Spent-only placeholder rows tolerate a missing mapping.
/// `core_derived_addresses`, so its account index can't be resolved.
/// Retained as a fatal-classified typed marker; the apply path no
/// longer raises it — it skips such a UTXO (logged) so one
/// unresolvable row never aborts a whole flush, and the balance
/// re-warms when the address later derives.
#[error("unspent utxo address {address} is not in core_derived_addresses")]
UtxoAddressNotDerived { address: String },

Expand Down
19 changes: 18 additions & 1 deletion packages/rs-platform-wallet-storage/src/sqlite/persister.rs
Original file line number Diff line number Diff line change
Expand Up @@ -861,7 +861,7 @@ impl PlatformWalletPersistence for SqlitePersister {
/// # }
/// ```
fn load(&self) -> Result<ClientStartState, PersistenceError> {
let conn = self.conn().map_err(PersistenceError::from)?;
let mut conn = self.conn().map_err(PersistenceError::from)?;
let mut state = ClientStartState::default();

let addrs_all = schema::platform_addrs::load_all(&conn).map_err(PersistenceError::from)?;
Expand Down Expand Up @@ -901,6 +901,23 @@ impl PlatformWalletPersistence for SqlitePersister {
))
})?;

// Reconcile `core_derived_addresses` against the pool snapshots:
// fill any pool address the derived table is missing (DBs predating
// the mirror, or partial state) so the next sync's UTXO writer can
// resolve pool-address accounts. Additive — never clobbers a live
// row; rows already covering the pools cost only no-op inserts.
{
let tx = conn
.transaction()
.map_err(WalletStorageError::from)
.map_err(PersistenceError::from)?;
schema::core_state::rehydrate_derived_addresses_from_pools(&tx, &wallet_id)
.map_err(PersistenceError::from)?;
tx.commit()
.map_err(WalletStorageError::from)
.map_err(PersistenceError::from)?;
}

let account_manifest =
schema::accounts::load_state(&conn, &wallet_id).map_err(PersistenceError::from)?;
let core_state = schema::core_state::load_state(&conn, &wallet_id, network)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -152,6 +152,23 @@ pub fn apply_pools(
pool_type,
payload,
])?;
// Mirror every snapshot address into `core_derived_addresses` (same
// tx) so a UTXO landing on a pool address resolves its account even
// when no live derive event preceded it (genesis-rescan). The shared
// helper keeps these rows identical to the live derive path.
for info in &entry.addresses {
let address = info.address.to_string();
crate::sqlite::schema::core_state::upsert_derived_address_row(
tx,
wallet_id,
account_type,
i64::from(account_index),
pool_type,
info.index,
&address,
info.used,
)?;
}
}
Ok(())
}
Expand Down
208 changes: 178 additions & 30 deletions packages/rs-platform-wallet-storage/src/sqlite/schema/core_state.rs
Original file line number Diff line number Diff line change
Expand Up @@ -56,31 +56,23 @@ pub fn apply(
}
// Derived addresses are written before UTXOs (same tx) so the UTXO
// writer's address→account_index lookup sees the fresh rows.
if !cs.addresses_derived.is_empty() {
let mut stmt = tx.prepare_cached(
"INSERT INTO core_derived_addresses \
(wallet_id, account_type, account_index, address, derivation_path, used) \
VALUES (?1, ?2, ?3, ?4, ?5, ?6) \
ON CONFLICT(wallet_id, account_type, address) DO UPDATE SET \
account_index = excluded.account_index, \
derivation_path = excluded.derivation_path",
for da in &cs.addresses_derived {
let account_type = crate::sqlite::schema::accounts::account_type_db_label(&da.account_type);
let account_index = crate::sqlite::schema::accounts::account_index(&da.account_type);
let pool_type = crate::sqlite::schema::accounts::pool_type_db_label(&da.pool_type);
let address = da.address.to_string();
// Live derive events carry no `used` flag — default false; a pool
// snapshot (the other caller of this helper) carries the real one.
upsert_derived_address_row(
tx,
wallet_id,
account_type,
i64::from(account_index),
pool_type,
da.derivation_index,
&address,
false,
)?;
for da in &cs.addresses_derived {
let account_type =
crate::sqlite::schema::accounts::account_type_db_label(&da.account_type);
let account_index = crate::sqlite::schema::accounts::account_index(&da.account_type);
let pool_type = crate::sqlite::schema::accounts::pool_type_db_label(&da.pool_type);
let address = da.address.to_string();
let path = format!("{}/{}", pool_type, da.derivation_index);
stmt.execute(params![
wallet_id.as_slice(),
account_type,
i64::from(account_index),
address,
path,
false
])?;
}
}
if !cs.new_utxos.is_empty() {
let mut stmt = tx.prepare_cached(UPSERT_UTXO_SQL)?;
Expand Down Expand Up @@ -167,13 +159,21 @@ fn execute_upsert_utxo(
.optional()?;
let account_index: i64 = match looked_up {
Some(idx) => idx,
// Refuse an unspent UTXO whose address we never derived — it would
// mis-bucket live money under account 0 and never re-derive. The
// spent-only path below tolerates the fallback (a wrong index is inert).
// Skip an unspent UTXO whose address we never derived: bucketing it
// under account 0 would mis-file live money, and erroring would abort
// the whole flush (genesis-rescan can match a UTXO before its derive
// event lands). Funds-safe: the balance re-warms only once the
// address is later derived (gap-limit dependent), not unconditionally.
// The spent-only arm keeps the inert fallback.
None if !spent => {
return Err(WalletStorageError::UtxoAddressNotDerived {
address: address.clone(),
});
tracing::warn!(
wallet_id = %hex::encode(wallet_id),
address = %address,
txid = %utxo.outpoint.txid,
vout = utxo.outpoint.vout,
"skipping unspent UTXO at an address absent from core_derived_addresses; balance re-warms only once the address is later derived"
);
return Ok(());
}
None => {
tracing::debug!(
Expand All @@ -196,6 +196,111 @@ fn execute_upsert_utxo(
Ok(())
}

// Conflict target = the BIP32-leaf PK. A same-leaf re-derive is
// deterministic — `address` is a pure function of the slot and `used` is
// write-once — so there is nothing legitimate to update; DO NOTHING. A
// different leaf yielding the same `address` is a UNIQUE(address)
// violation, not a PK hit, so it surfaces loud.
const UPSERT_DERIVED_ADDRESS_SQL: &str = "INSERT INTO core_derived_addresses \
(wallet_id, account_type, account_index, pool_type, derivation_index, address, used) \
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7) \
ON CONFLICT(wallet_id, account_type, pool_type, derivation_index) DO NOTHING";
Comment thread
Claudius-Maginificent marked this conversation as resolved.
Outdated

// Additive reconcile: fill gaps only, never touch an existing row. `OR
// IGNORE` skips ALL constraint violations (PK and UNIQUE(address)) so a
// would-be address collision can't abort the load — safe because an
// authoritative row already owns any colliding address.
const INSERT_DERIVED_ADDRESS_IF_ABSENT_SQL: &str = "INSERT OR IGNORE INTO core_derived_addresses \
(wallet_id, account_type, account_index, pool_type, derivation_index, address, used) \
VALUES (?1, ?2, ?3, ?4, ?5, ?6, ?7)";

/// Upsert one `core_derived_addresses` row. Single writer for both the
/// live `addresses_derived` event path and the `apply_pools` snapshot
/// path, so the address→account_index map the UTXO writer joins is
/// identical regardless of which source populated it. `used` is set on
/// insert only — the conflict clause leaves an existing `used` untouched
/// so a later live re-derive (which carries no flag) cannot clear a
/// snapshot's real value.
// Args map 1:1 onto the row's NOT-NULL columns; a wrapper struct would add
// a single-use type for the two call sites without improving clarity.
#[allow(clippy::too_many_arguments)]
pub(crate) fn upsert_derived_address_row(
tx: &Transaction<'_>,
wallet_id: &WalletId,
account_type: &str,
account_index: i64,
pool_type: &str,
derivation_index: u32,
address: &str,
used: bool,
) -> Result<(), WalletStorageError> {
let mut stmt = tx.prepare_cached(UPSERT_DERIVED_ADDRESS_SQL)?;
stmt.execute(params![
wallet_id.as_slice(),
account_type,
account_index,
pool_type,
i64::from(derivation_index),
address,
used,
])?;
Ok(())
}

/// Reconcile `core_derived_addresses` for `wallet_id` against its
/// `account_address_pools` snapshots, filling any address the snapshots
/// declare but the derived table is missing (already-persisted DBs that
/// predate the pool→derived mirror, or partial state where some addresses
/// derived live but others never did).
///
/// Purely additive: every insert is `INSERT OR IGNORE`, so an existing
/// authoritative row (live or mirrored) keeps its account_index,
/// pool_type, derivation_index, and used flag untouched, and a would-be
/// UNIQUE(address) collision is skipped rather than aborting the load. A
/// wallet whose derived rows already cover its pools incurs only no-op
/// inserts. Decoding a snapshot blob is fail-hard (corruption is never
/// skipped).
pub(crate) fn rehydrate_derived_addresses_from_pools(
tx: &Transaction<'_>,
wallet_id: &WalletId,
) -> Result<(), WalletStorageError> {
let snapshots: Vec<Vec<u8>> = {
let mut stmt = tx.prepare_cached(
"SELECT snapshot_blob FROM account_address_pools WHERE wallet_id = ?1",
)?;
let rows = stmt.query_map(params![wallet_id.as_slice()], |row| {
row.get::<_, Vec<u8>>(0)
})?;
let mut out = Vec::new();
for r in rows {
out.push(r?);
}
out
};

let mut insert_stmt = tx.prepare_cached(INSERT_DERIVED_ADDRESS_IF_ABSENT_SQL)?;
for payload in snapshots {
let entry: platform_wallet::changeset::AccountAddressPoolEntry = blob::decode(&payload)?;
let account_type =
crate::sqlite::schema::accounts::account_type_db_label(&entry.account_type);
let account_index = crate::sqlite::schema::accounts::account_index(&entry.account_type);
let pool_type = crate::sqlite::schema::accounts::pool_type_db_label(&entry.pool_type);
for info in &entry.addresses {
let address = info.address.to_string();
insert_stmt.execute(params![
wallet_id.as_slice(),
account_type,
i64::from(account_index),
pool_type,
i64::from(info.index),
address,
info.used,
])?;
}
}
Ok(())
}

fn upsert_sync_state(
tx: &Transaction<'_>,
wallet_id: &WalletId,
Expand Down Expand Up @@ -443,3 +548,46 @@ pub fn list_unspent_utxos(
}
Ok(by_account)
}

/// One `core_derived_addresses` row. Used by tests that assert the
/// address→account map written by `apply_pools` matches the live derive
/// path (row-shape parity) and that load-time rehydration repopulates it.
Comment thread
Claudius-Maginificent marked this conversation as resolved.
Outdated
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated
#[cfg(any(test, feature = "__test-helpers"))]
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct DerivedAddressRow {
pub account_type: String,
pub account_index: i64,
pub pool_type: String,
pub derivation_index: i64,
pub address: String,
pub used: bool,
}

/// Every `core_derived_addresses` row for one wallet, ordered for
/// determinism. Retained for this crate's integration tests.
#[cfg(any(test, feature = "__test-helpers"))]
pub fn list_derived_addresses_for_test(
conn: &Connection,
wallet_id: &WalletId,
) -> Result<Vec<DerivedAddressRow>, WalletStorageError> {
let mut stmt = conn.prepare(
"SELECT account_type, account_index, pool_type, derivation_index, address, used \
FROM core_derived_addresses WHERE wallet_id = ?1 \
ORDER BY account_type, pool_type, derivation_index",
)?;
let rows = stmt.query_map(params![wallet_id.as_slice()], |row| {
Ok(DerivedAddressRow {
account_type: row.get(0)?,
account_index: row.get(1)?,
pool_type: row.get(2)?,
derivation_index: row.get(3)?,
address: row.get(4)?,
used: row.get(5)?,
})
})?;
let mut out = Vec::new();
for r in rows {
out.push(r?);
}
Ok(out)
}
Loading
Loading