Coming Soon: An Easier Way To Bootstrap Your Node’s UTXO Set

Yesterday marks coming up to the finishing line after almost four years of work. Assumeutxo was merged into the Bitcoin Core master branch by Andrew Chow yesterday. This is the first major non-consensus update to the Bitcoin reference implementation in quite a while, and is going to have a lot of interesting implications for users bootstrapping nodes.

To give a little context about the feature, I think it’ll be useful to give a little history lesson. When a Bitcoin Core node first turns on and starts syncing the blockchain it doesn’t actually validate the entire historical chain. It can, but it doesn’t by default, the reason being to save time and resources when bootstrapping. It validates the entire set of blockchain headers to make sure the proof-of-work is valid, but it doesn’t actually validate the signatures or other witness data for really old blocks. The logic is that for really old blocks, say five years old, it’s overkill to actually verify and run through the signature and other witness checks for those blocks. If someone was able to mine an invalid block five years ago spending coins without a valid signature, then Bitcoin is essentially broken.

Bitcoin Core used to skip validation on these old blocks with a hardcoded checkpoint, i.e. developers would literally just include the blockhash for a known valid block in the past directly into the validation code and Core would not check witness data for anything before that. It was a very hacky solution, and it was not possible for users to change this behavior without actually modifying the source code to the client themselves. In 2017 Greg Maxwell introduced the assumevalid function to deal with these issues. Instead of a hardcoded value in the source code, assumevalid was a setting that could be modified on startup where users could manually pass a blockhash themselves and decide which block to start validating witness data at. Core still hardcodes a default block for this feature, but the key difference is now users can easily override that without modifying sourcecode. Simply run “bitcoind –assumevalid=[theblockhash] and users can decide themselves what to validate or not. You can run it with assumevalid=0 to validate every single block since Genesis.

Assumeutxo brings a very similar functionality to handling the UTXO set. Everything your node does to verify the historical blockchain is done with the sole purpose of building the UTXO set. You can’t really know if a transaction is valid unless you know if the coin has been spent before, and the UTXO set is the cache a node builds up to keep track of all the coins that haven’t been spent. Users have put together many hacky tools in the last few years to simply skip ahead and start from a known good UTXO set from some point in time, most notably BTCPay Server. These are all very ugly hacks though that essentially boil down to copy and pasting the file directory holding the UTXO set. Now that functionality will be supported in Bitcoin Core proper.

The implementation James O’Beirne put together has implemented all the logic for importing and starting to use a UTXO snapshot, as well as tools to create one in the first place. Very much like assumevalid, assumeutxo will simply accept an existing snapshot file as an argument and load that into the user’s node and start running from there. This will add a massive amount of flexibility to node operators, both on the end user side as well as businesses.

Users can create a snapshot of their current UTXO set and store that somewhere; if their node ever crashes or any of the databases corrupt, they can simply grab that UTXO snapshot and restart their node from that blockheight instead of having to restart from the Genesis block. Users who have trusted relationships with each other can simply help one another start up a node from where things are now. No more blockchain syncing necessary at all. If you trust me, I can give you the current UTXO set from my node, and within a margin of a few hours worth of blocks you can have a node up and running in no time. Businesses who have to run multiple nodes for different reasons can spin them up much quicker and easier from UTXO snapshots.

Now, this isn’t quite ready to go live in the next Bitcoin Core release, but as James put it on Twitter the hurdle to cross to get to that point is literally five lines of code. All of the core logic and implementation is merged, all that needs to be done is change some flags so that Core will allow users to make use of assumeutxo when running their node on mainnet.

So, congratulations to James on the merge. Many Bitcoiners are looking forward to having more flexibility and control over their own node. 

3 October 2023 14:30