Why event-driven indexing misses state changes on Blend and Soroban

The intuitive shape of a blockchain indexer is a loop: watch for events, decode each one, fold it into the balance. That is how we built our first pass at indexing Blend, and it is what most people reach for — it is a reasonable assumption, and it is how comparable systems are built on other chains. The assumption is that events are a complete description of state change.

On Stellar they are not, and we found out from two directions. Blend expresses its most consequential actions under names our event classifier could not match, so those actions never reached the activity feed. And underneath the protocol, Soroban’s state archival evicts a wallet’s position from the ledger entirely, with no user action at all — so our fold kept valuing a position the chain no longer held.

We caught both by building a second, independent implementation and replaying fifteen months of mainnet through both engines. This is what it found, and what we got wrong along the way — including once when our own fix over-corrected.

Quirk	The assumption it breaks	What actually happens	What it cost us
Interest via rate drift	balances change only on a transaction	the amount is shares × rate, and the rate lives on the reserve, not on you	a balance that silently never grows
Liquidation as auction fill	actions are named (liquidate)	emitted as fill_auction with a numeric auction_type	the highest-frequency action missing from the activity feed
State eviction (TTL lapse)	an entry exists until a user deletes it	the protocol evicts unmaintained entries	valuations published for a position that was off-ledger

The action we could not name

Our activity classifier decoded event names by keyword. To catch liquidations it looked for the substring liquid. On real Blend v2 mainnet data, that branch never fires — because Blend does not emit a liquidate event. Liquidations, bad-debt fills, and interest fills all run through the same auction machinery and surface as fill_auction, carrying a numeric auction_type topic (0 = user liquidation, 1 = bad debt, 2 = interest). A substring match never inspects a numeric field.

The action vocabulary underneath is the pool’s RequestType enum, and it is numbers all the way down:

#[repr(u32)]
pub enum RequestType {
    Supply = 0,                     Withdraw = 1,
    SupplyCollateral = 2,           WithdrawCollateral = 3,
    Borrow = 4,                     Repay = 5,
    FillUserLiquidationAuction = 6, FillBadDebtAuction = 7,
    FillInterestAuction = 8,        DeleteLiquidationAuction = 9,
}

We went and counted. Reading the Blend v2 contracts against fifteen months of observed mainnet events across six position-bearing pools produced a catalogue of 43 distinct actions. Our classifier decoded 12 event names and silently quarantined 9 others that actually occur on mainnet. The largest by volume was fill_auction — 3,835 occurrences on one pool alone, none of them producing an activity row.

What makes it high-stakes is who calls these:

Request	Who initiates it	Whose balance moves
FillUserLiquidationAuction (6)	a third-party liquidator	the liquidated user’s — who signed nothing
FillBadDebtAuction (7)	backstop / liquidator	the defaulted user’s, via socialization
FillInterestAuction (8)	a filler	the reserve’s, distributed to suppliers

The most consequential balance changes on the pool are third-party actions the observed wallet never authored. An indexer that only records what a wallet does to itself has a hole shaped exactly like its liquidations.

One correction to the story we first told ourselves, because it matters and because it is the whole lesson in miniature: this did not corrupt position balances. Our state fold reads the resulting ledger entries — the user’s positions map, the reserve’s config and data — and is entirely independent of event names. A liquidation still rewrites the liquidated user’s Positions entry, and the fold still picks that up. So the balances stayed right while the activity feed went blind. The concrete user-facing consequence is narrower than “your position is wrong”, and stranger: a user could be liquidated, watch their collateral vanish from the balance, and find no activity explaining it.

The path that read state survived. The path that waited for events did not. That is the thesis of this post, and we did not arrive at it by argument.

The keyword matcher is gone. The decoder now maps event names exactly rather than by substring, and reads the auction subtype from its u32 topic discriminator — the liquidation, bad-debt and interest cases the symbol itself never distinguishes.

Interest is a number that changes while nothing happens

The same split shows up before you get anywhere near a liquidation. A Blend supplier’s balance is not stored as a token amount. It is stored as a share count, and the amount is derived: balance = shares × b_rate, where the reserve’s b-rate (and the borrower-side d-rate) is an exchange rate that drifts upward as interest accrues.

The important structural detail is where that rate lives. The b-rate is reserve state, carried in the reserve’s data entry — not in the user’s entry. Our own decoder tests pin that separation explicitly: a reserve’s config payload carries c_factor but not b_rate, because config and data are different entries with different lifecycles.

ledger N     :  shares = 1,000   b_rate = 1.020000   →  balance = 1,020.00
ledger N+50k :  shares = 1,000   b_rate = 1.020411   →  balance = 1,020.41
             ^ no transaction, no event on the user's entry — only the rate moved

A poll-the-events indexer sees nothing between those two ledgers and reports the balance unchanged, because the only thing that moved was a rate on the reserve. To get this right you cannot wait for a deposit — you have to read the reserve’s rate state and revalue the share count. The balance is a function of state, not the sum of a stream of events.

That same fact — that interest ticks write the reserve’s entries and not the user’s — is what set up the second failure, and it is the one that actually reached served data.

The ledger evicts your state while you sleep

Every Soroban contract-data entry has a time-to-live. A Blend wallet’s persistent Positions entry has one clock, and that clock anchors at its last on-chain write — a deposit, borrow, withdraw, or repay. Interest ticks do not refresh it; they write the reserve’s entries, not the wallet’s. Leave a position untouched past its TTL and the protocol evicts it: while evicted, the entry does not exist on-ledger at all. A later restore brings it back. None of this is a transaction the user sends.

These are not soft limits. Read over RPC, mainnet’s archival settings put the persistent-entry minimum at 2,073,600 ledgers and the maximum entry life at 3,110,400 ledgers.

Our fold kept every position in memory and never consumed the ledger’s eviction list. So it sailed straight through the evicted window, re-emitting share × reserve_rate at its flush cadence for an entry the chain no longer held. The share count was right — it is the retained pre-eviction share — but the valuation was a fiction: it priced a balance that was off-ledger.

What the replay actually found

We only know the size of this because we built a second implementation and compared. A full-history cross-engine replay covered 37,973 wallets across 6,848,288 mainnet ledgers — roughly fifteen months — running the production fold against an independent engine. Of 885,796 rows compared: 94.21% matched exactly, 5.78% were value-consistent at a cadence offset, and 100 rows — 0.011% — were real divergences. Every one of the 100 was a fold defect; the independent engine was right.

The 100 split into two families: 58 where the fold valued entries after they had been evicted, and 42 phantom-zero emissions.

Getting to that conclusion took three rounds, and our first read was wrong. The initial dossier called the divergences a revaluation-cadence difference and treated the second engine’s silence as the anomaly. An independent refutation then argued TTL archival was impossible for most of the sample. Only the third pass — reconstructing each entry’s write history — settled it, on three independent quantitative signatures that agree:

• Across a 33-row sample, every entry sat unwritten across its window longer than the protocol’s guaranteed persistent lifetime. 19 of 33 sat unwritten longer than the maximum lifetime any entry can hold, making eviction protocol-forced rather than merely possible.
• The independent engine stops valuing each entry at a median of 2,096,298 ledgers past that entry’s own last write — the minimum persistent TTL plus a small scan lag. A cadence model has no reason to halt at a protocol constant, reproduced across unrelated wallets each anchored at its own write ledger.
• Valuation floors cluster: eight unrelated wallets stop at the same ledger within a ~1,700-ledger band, which is what a single batched eviction-scan pass looks like and is not what a per-wallet cadence produces.

Walked end to end on one public wallet: it deposits at ledger 59,476,041, and its Positions entry is not written again for 3,481,895 ledgers — past the maximum entry life, so eviction is forced. The reserve kept ticking interest the whole time, but those ticks wrote the pool’s reserve entry, not this wallet’s. The independent engine emits nothing after 61,552,737. The fold emitted a row deep inside that window, carrying the retained share at a fresher rate.

The fix has two halves. The fold now decodes the ledger close meta’s evicted-key list and synthesizes the removals the ordinary change stream never reports; and the adapter archives a TTL-lapsed entry in place rather than purging it, so the fold stops revaluing it instead of carrying it forward. A regression test pins that an archived, untouched entry emits exactly zero rows across its evicted window.

Through the evicted window	Event-driven in-memory fold	Archival-aware fold
Share count	retained (correct)	retained (correct)
Valuation	share × fresher_rate (fiction)	none — sealed at eviction
Entry state	still live	archived, with the archival ledger recorded

The over-correction

We then got it wrong in the other direction, which is the part most worth writing down.

Having identified 58 bad rows, we retracted all 58 from served data under a row-count guard. Re-adjudication later showed the classifier behind that retraction had mishandled a subclass: rows written at a restore ledger are the entry coming back on-ledger with chain-exact values, not ghosts. On the final split, only 6 of the 58 were genuine post-eviction ghosts. The other 52 were chain-true rows we deleted.

The evidence was unambiguous once we looked for it — decoded close-meta showing LedgerEntryRestored with exactly the retracted share, and an RPC lastModifiedLedgerSeq equal to the row’s own ledger. Our classifier’s restore-detection field was simply empty for those rows; it had not found the restore that was plainly there. Serving data has since been re-derived end to end from the deploy floor, so the 52 re-enter through the fold itself and the 6 never do.

Both mistakes have the same shape, and it is not a coding error. Distinguishing an eviction from a close, and a restore from a deposit, cannot be done from current state. getLedgerEntries serves only now: an archived entry looks identical to one that never existed, and a restore write looks exactly like continuous liveness. It takes the entry’s full write history to know whether a gap was an eviction. We went wrong first by ignoring that history and then by reading it with a classifier that missed half of it.

What you observe	What it might mean	The trap
A balance disappears	evicted (archived), not closed to zero	an archived entry is indistinguishable from one that never existed
An entry reappears	a restore, not a new deposit	counting the restore ledger as activity invents a deposit that never happened
A read inside a gap	the entry was off-ledger then	re-emitting a value through the gap prices a balance the chain did not hold

Build for the actions nobody announces

One discipline covers all of it: derive state from what the chain holds; don’t accumulate it from the events you happen to receive.

Interest accrues with no event on your entry, so revalue from reserve rate state rather than waiting for a deposit. Liquidations arrive as numeric auction fills initiated by third parties, so enumerate what the protocol can do from its RequestType enum instead of trusting what your handlers happen to catch. And the ledger archives entries on its own clock, so treat a vanished balance as possibly-archived and a reappearance as possibly-a-restore until the write history says otherwise.

The part we would emphasise to anyone building the same thing: none of these failures announced themselves. Nothing crashed, no error rate moved, and the pipeline reported healthy throughout. We found the eviction defect because we had a second implementation to disagree with the first, and we found the over-correction because someone re-checked a fix that had already shipped. On a chain where state changes without emitting an event, a check that only watches the event stream is a check that passes by not looking.