Bitcoin Optech Newsletter #217
This week’s newsletter includes our regular section with the summary of a Bitcoin Core PR Review Club meeting, a list of new software releases and release candidates, and summaries of notable changes to popular Bitcoin infrastructure projects.
No significant news this week.
Bitcoin Core PR Review Club
In this monthly section, we summarize a recent Bitcoin Core PR Review Club meeting, highlighting some of the important questions and answers. Click on a question below to see a summary of the answer from the meeting.
Reduce bandwidth during initial headers sync when a block is found is a PR by Suhas Daftuar that reduces a node’s network bandwidth requirements while synchronizing the blockchain with peers, including during Initial Block Download (IBD). An important part of the Bitcoin ethos is minimizing resource demands of running a fully-validating node, including networking resources, to encourage more users to run full nodes. Speeding up the sync time furthers this goal as well.
Blockchain synchronization occurs in two phases: First, the node receives block headers from peers; these headers are sufficient to determine the (likely) best chain (the one with the most work). Second, the node uses this best chain of headers to request and download the corresponding full blocks. This PR affects only the first phase (headers download).
Why do nodes (mostly) receive
invblock announcements while they are doing initial headers sync, even though they indicated preference for headers announcements (BIP 130)?
A node will not announce a new block to a peer using a headers message if the peer has not previously sent a header to which the new header connects to, and syncing nodes do not send headers. ➚
Why is bandwidth wasted (during initial headers sync) by adding all peers that announce a block to us via an
invas headers sync peers?
Each of these peers will then begin sending us the same stream of headers: the
getheadersto the same peer, and its
headersreply triggers another
getheadersfor the immediately following range of block headers. Receiving duplicate headers is harmless except for the additional bandwidth. ➚
What would be your estimate (lower/upper bound) of how much bandwidth is wasted?
Upper bound (in bytes):
(number_peers - 1) * number_blocks * 81; lower bound: zero (if no new blocks arrive during headers sync; if the syncing peer and the network are fast, downloading all 700k+ headers takes only a few minutes) ➚
What’s the purpose of CNodeState’s members
PeerManagerImpl::nSyncStarted? If we start syncing headers with peers that announce a block to us via an
inv, why do we not increase
fSyncStarted = trueand update
nSyncStartedcounts the number of peers whose
fSyncStartedis true, and this number can’t be greater than 1 until the node has headers close to (within one day of) the current time. This (arbitrary) peer will be our initial headers-sync peer. If this peer is slow, the node times it out (
m_headers_sync_timeout) and finds another ‘initial’ headers syncing peer. But if, during headers sync, a node sends an
invmessage that announces blocks, then without this PR, the node will start requesting headers from this peer as well, without setting its
fSyncStartedflag. This is the source of the redundant headers messages, and was probably not intended, but has the benefit of allowing headers sync to proceed even if the initial headers-sync peer is malicious, broken, or very slow. With this PR, the node requests headers from only one additional peer (rather than from all peers who announced the new block). ➚
An alternative to the approach taken in the PR would be to add an additional headers sync peer after a timeout (fixed or random). What is the benefit of the approach taken in the PR over this alternative?
One benefit is that peers that announce an
invto us have a higher probability of being responsive. Another is that a peer that manages to send us the block
invfirst is often also a very fast peer. So we’d not pick another slow peer if for some reason our initial peer is slow.
Releases and release candidates
New releases and release candidates for popular Bitcoin infrastructure projects. Please consider upgrading to new releases or helping to test release candidates.
- ● LDK 0.0.111 adds support for creating, receiving, and relaying onion messages, among several other new features and bug fixes.
Notable code and documentation changes
Notable changes this week in Bitcoin Core, Core Lightning, Eclair, LDK, LND, libsecp256k1, Hardware Wallet Interface (HWI), Rust Bitcoin, BTCPay Server, BDK, Bitcoin Improvement Proposals (BIPs), and Lightning BOLTs.
● Bitcoin Core #25614 builds on Bitcoin Core #24464, allowing the ability to add and trace logs with specific severity levels in addrdb, addrman, banman, i2p, mempool, netbase, net, net_processing, timedata, and torcontrol.
● Bitcoin Core #25768 fixes a bug where the wallet wouldn’t always rebroadcast unconfirmed transactions’ child transactions. Bitcoin Core’s built-in wallet periodically attempts to broadcast any of its transactions which haven’t been confirmed yet. Some of those transactions may spend outputs from other unconfirmed transactions. Bitcoin Core was randomizing the order of transactions before sending them to a different Bitcoin Core subsystem that expected to receive unconfirmed parent transactions before child transactions (or, more generally, all unconfirmed ancestors before any descendants). When a child transaction was received before its parent, it was internally rejected instead of being rebroadcast.
● Bitcoin Core #19602 adds a
migratewalletRPC that will convert a wallet to natively using descriptors. This works for pre-HD wallets (those created before BIP32 existed or was adopted by Bitcoin Core), HD wallets, and watch-only wallets without private keys. Before calling this function, read the documentation and be aware that there are some API differences between non-descriptor wallets and those that natively support descriptors.
● Eclair #2406 adds an option for configuring the experimental interactive funding protocol implementation to require that channel open transactions only include confirmed inputs—inputs which spend outputs that are a part of a confirmed transaction. If enabled, this can prevent an initiator from delaying a channel open by basing it off of a large unconfirmed transaction with a low feerate.
● Eclair #2190 removes support for the original fixed-length onion data format, which is also proposed for removal from the LN specification in BOLTs #962. The upgraded variable-length format was added to the specification more than three years ago and network scanning results mentioned in the BOLTs #962 PR indicate that it is supported by all but 5 out of over 17,000 publicly advertised nodes. Core Lightning also removed support earlier this year (see Newsletter #193).
● Rust Bitcoin #1196 modifies the previously-added
LockTimetype (see Newsletter #211) to be a
absolute::LockTimeand adds a new
relative::LockTimefor representing locktimes used with BIP68 and BIP112.