Rearchitecting For Segregated Witness

I switched my lightning prototype to use segregated witness for the anchor inputs, making the anchor immalleable. But that’s just the start of what SW offers, so I began switching the other transactions similarly: the commitment transactions, and the mutual close transaction(s).

This is where my infrastructure started showing its cracks. For easy prototyping, I hook into bitcoind’s wallet and “importaddress” the P2SH address which the anchor transaction pays into: this make “listtransactions” report any spends to and from that address, which covers the anchor, commitment transactions, and mutual close. I poll this every 30 seconds by default.

Since there’s no address format for segregated witness P2WSH addresses (yet), this approach doesn’t work. And using listtransactions is actually quite awkward; handling reorganizations seems particularly problematic. Finally, there’s no “forgetaddress” so these addresses would build up over time.

Thus I switched to directly quering the blocks as they come in, looking for interesting transactions. This means we have to do more work (tracking topology) but we can be more precise and easily divest the code from bitcoind later if we want.

This is made far easier by segregated witness, since not only are they immalleable, but the transaction ID is known before the tx is signed. When this is ubiquitous it will make writing bitcoin software much easier. And since “getblock” gives you the transactions IDs, you don’t need to dig deeper; just look for every transaction you handed out a signature for.

Not only are segwit txs immalleable, but the ID is known before the tx is signed.

But now we have a chicken and egg problem: I wanted to test the topology code with callbacks was working before I convert transactions to segregated witness, but you can’t say “watch for this txid” with an unsigned transaction without segwit! The solution I came up with in the interim was to create normalized transaction ids, which is the txid of the transaction with empty input scripts (in fact, this is basically how segregated witness works, and what the transactions I generate internally look like). When I get a block from bitcoind, I query all the transactions using gettransaction/getrawtransaction and produce a normalized txid.

Except that I need the real txid of the anchor transaction, which is already converted to segregated witness. So I actually pretend the block has twice as many transactions in it: the normalized txids and the standard txids. This works for the moment, and allows me to debug and develop the topology based callbacks; once the segwit conversion is complete I’ll eliminate the word “normalized” from the source altogether.

This also allows me to correctly interpret the concept of “confirmations” of a transaction. This is simple when there’s only one chain, but what about forks? If the transaction is in both forks, you don’t care, but if it’s only in one, don’t treat it as confirmed! What you really want to know is: how many blocks would it take to unspend this transaction? The naive algorithm for this is quite simple: measure the height of the tx on every branch (count it as branch-length + 1 if it’s not on the branch), and take the largest of those heights: subtract that from the highest block plus one, and there’s your answer.

What you really want to know is: how many blocks would it take to unspend this transaction?

The result is a fairly nice series of patches, and a much narrower interface to bitcoind: getchaintips, getblock, getrawtransaction, estimatefee and sendrawtransaction. I expect to finish segregated witness and push this branch out early next week.