Segregated Witness Hints For Developers

OK, now I’ve fully converted c-lightning to use segregated witness (which open-codes all the bitcoin transactions), I’ve jotted down a few signposts for other developers looking at segregated witness support in their clients.

Witness data: you need to store this somewhere! I added a u8** witness to my struct bitcoin_tx for ease of encoding, with NULL meaning no witness. This is not bitcoin script, but the raw stack data from bottom to top. So whereas you might have previously had a scriptSig for a multisig P2SH of “OP_0 PUSH(sig1) PUSH(sig2) PUSH(redeemscript)” your equivalent witness is the stack that results, ie “[] [sig1] [sig2] [redeemscript]” which gets encoded as simply as possible: a varint for the number of items (ie. 3 here), then a varint length followed by the item for each one. And note that OP_0 pushes an empty element on the stack, not a zero byte!

BIP 141: The basic Segregated Witness BIP is pretty dense, but you’ll need to read it. In particular, you’ll need to write new encode/decode routines for transactions with witnesses (which are made to look like transactions with no inputs, normally invalid). Fortunately, this is fairly easy, and has a flags field for future extensions; see tx.c’s add_tx().

BIP 143: You need to write a new serializer for your transactions for any signatures you’re going to put inside a witness; OP_CHECKSIG works differently! The BIP is pretty clear, but you can also see tx.c which has a self-contained hash_for_segwit() if you just want SIGHASH_ALL support.

Fee estimation: presumably the semantics of bitcoin’s “estimatefee” RPC call won’t change, which means you should create a routine which measures the cost; that’s the non-witness encoded length, plus the witness length divided by 4. That’s comparable against the old estimatefee “per byte” cost. (Strictly you should multiply the old length by 4 and add the witness length as my routine does, but for estimatefee that precision is overkill).

Warming up regtest: on Pieter’s segwit4 branch at least, you need to generate 432 blocks to activate the segregated witness BIPs (via BIP9 versionbits). I was scratching my head until Pieter told me.

Converting from P2SH to P2WSH: This is probably the easiest thing to do. On the input the redeemscript is now the witnessscript, and the other push-ops are replaced by the literal stack (for small pushes the opcode is the same as the length, so it’s even bitwise identical). On the output use a zero byte followed by the witness program: SHA256(witnessscript). This replaces the old p2sh program’s OP_HASH RIPEMD(SHA256(redeemscript)) OP_EQUAL.

Getting Witness Inputs: the easiest way to get witness inputs from another wallet is via p2sh. This means the zero byte and witness program above goes in a single push in the scriptSig, and that’s the redeemscript for the p2sh in the output. This is less efficient, but good for compatibility with older code.

Live Testing: You can use Nicolas Dorier’s transaction tester for testing the transactions you make; even if you’re using regtest and the input txs are unknown, at least it will tell you if you’ve encoded them correctly.

Remember, you only get the malleability benefits of Segregated Witness if all your inputs are segwit! That probably means going via p2sh for the moment, but don’t make a habit of it, as “native” segwit is more compact and simpler!

(Thanks to Greg Sanders for contributing additional semantic precision!)