This release marks the first major release in the
v0.12.x series! As this is a major release several new features are included in this release including: anchor commitment types are now the default, anchor commitment support for watchtowers, new arguments to auto compact the database as well as drop the
wtxmgr state, generic wallet PSBT crafting+signing, and much more! As usual, this release contains several important bug fixes, so we recommend all users update.
- TODO: describe "peers-bucket"
This release contains a migration to initialize a top-level-bucket for an outpoint index. There is also a subsequent migration that populates this index with an outpoint's status. This will cut down on expensive
bbolt transactions throughout the codebase. The migration process should look something like this upon initial start up:
2020-12-21 10:45:07.256 [INF] LTND: Version: 0.12.0-beta commit=v0.12.0-beta, build=production, logging=default
2020-12-21 10:45:07.257 [INF] LTND: Active chain: Bitcoin (network=mainnet)
2020-12-21 10:45:07.257 [INF] LTND: Opening the main database, this might take a few minutes...
2020-12-21 10:45:07.257 [INF] LTND: Opening bbolt database, sync_freelist=false, auto_compact=false
2020-12-21 10:45:07.304 [INF] CHDB: Checking for schema update: latest_version=20, db_version=17
2020-12-21 10:45:07.304 [INF] CHDB: Performing database schema migration
2020-12-21 10:45:07.304 [INF] CHDB: Applying migration #18
2020-12-21 10:45:07.304 [INF] CHDB: Creating top-level bucket: "peers-bucket" ...
2020-12-21 10:45:07.305 [INF] CHDB: Created top-level bucket: "peers-bucket"
2020-12-21 10:45:07.305 [INF] CHDB: Applying migration #19
2020-12-21 10:45:07.305 [INF] CHDB: Creating top-level bucket: "outpoint-bucket" ...
2020-12-21 10:45:07.305 [INF] CHDB: Created top-level bucket: "outpoint-bucket"
2020-12-21 10:45:07.305 [INF] CHDB: Applying migration #20
2020-12-21 10:45:07.324 [INF] LTND: Database now open (time_to_open=67.71764ms)!
Verifying the Release
In order to verify the release, you'll need to have
gpg2 installed on your system. Once you've obtained a copy (and hopefully verified that as well), you'll first need to import the keys that have signed this release if you haven't done so already:
curl https://keybase.io/roasbeef/pgp_keys.asc | gpg --import
Once you have the required PGP keys, you can verify the release (assuming
manifest-v0.12.0-beta.rc2.txt.sig are in the current directory) with:
gpg --verify manifest-v0.12.0-beta.rc2.txt.sig
You should see the following if the verification was successful:
gpg: assuming signed data in 'manifest-v0.12.0-beta.rc2.txt'
gpg: Signature made Tue Dec 15 18:57:27 2020 PST
gpg: using RSA key 60A1FA7DA5BFF08BDCBBE7903BBD59E99B280306
gpg: Good signature from "Olaoluwa Osuntokun <email@example.com>" [ultimate]
That will verify the signature of the manifest file, which ensures integrity and authenticity of the archive you've downloaded locally containing the binaries. Next, depending on your operating system, you should then re-compute the
sha256 hash of the archive with
shasum -a 256 <filename>, compare it with the corresponding one in the manifest file, and ensure they match exactly.
For this release roasbeef's signature is the secondary signature which can be verified with the following command:
gpg --verify roasbeef-manifest-v0.12.0-beta.rc2.txt.sig manifest-v0.12.0-beta.rc2.txt
gpg: Signature made Wed Sep 30 17:35:20 2020 PDT
gpg: using RSA key 4AB7F8DA6FAEBB3B70B1F903BC13F65E2DC84465
gpg: Good signature from "Olaoluwa Osuntokun <firstname.lastname@example.org>" [ultimate]
Verifying the Release Timestamp
From this new version onwards, in addition to time-stamping the git tag with OpenTimeStamps, we'll also now timestamp the manifest file along with its signature. Two new files are now included along with the rest of our release artifacts:
Assuming you have the opentimestamps client installed locally, the timestamps can be verified with the following commands:
ots verify manifest-v0.12.0-beta.rc2.txt.ots
ots verify manifest-v0.12.0-beta.rc2.txt.sig.ots -f roasbeef-manifest-v0.12.0-beta.rc2.txt.sig
Alternatively, the open timestamps website can be used to verify timestamps if one doesn't have a
bitcoind instance accessible locally.
These timestamps should give users confidence in the integrity of this release even after the key that signed the release expires.
Verifying the Release Binaries
Our release binaries are fully reproducible. Third parties are able to verify that the release binaries were produced properly without having to trust the release manager(s). See our reproducible builds guide for how this can be achieved.
The release binaries are compiled with
go1.15.6, which is required by verifiers to arrive at the same ones.
They include the following build tags:
watchtowerrpc. Note that these are already included in the release script, so they do not need to be provided.
make release command can be used to ensure one rebuilds with all the same flags used for the release. If one wishes to build for only a single platform, then
make release sys=<os-arch> tag=<tag> can be used.
Finally, you can also verify the tag itself with the following command:
$ git verify-tag v0.12.0-beta.rc2
gpg: Signature made Tue 15 Dec 2020 10:31:06 PM UTC using RSA key ID 9B280306
gpg: Good signature from "Olaoluwa Osuntokun <email@example.com>"
Building the Contained Release
Users are able to rebuild the target release themselves without having to fetch any of the dependencies. In order to do so, assuming
lnd-source-v0.12.0-beta.rc2.tar.gz are in the current directory, follow these steps:
tar -xvzf vendor.tar.gz
tar -xvzf lnd-source-v0.12.0-beta.rc2.tar.gz
GO111MODULE=on go install -v -mod=vendor -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.12.0-beta.rc2" ./cmd/lnd
GO111MODULE=on go install -v -mod=vendor -ldflags "-X github.com/lightningnetwork/lnd/build.Commit=v0.12.0-beta.rc2" ./cmd/lncli
-mod=vendor flag tells the
go build command that it doesn't need to fetch the dependencies, and instead, they're all enclosed in the local vendor directory.
Additionally, it's now possible to use the enclosed
release.sh script to bundle a release for a specific system like so:
make release sys="linux-arm64 darwin-amd64"
⚡️⚡️⚡️ OK, now to the rest of the release notes! ⚡️⚡️⚡️
New Default Autopilot Heuristic
In this version of
lnd, the default heursitic for autopilot has been changed from preferential attachment, to a version that will attempt to optimize for the betweeness centrality of the node. At a high level, this change means that rather than trying to connect (stochastically) to the nodes that have the most channels,
lnd will instead attempt to connect to the nodes that appear most often in the shortest paths within the network. This change will serve to step as a stepping stone to further diffuse the graph to make it more resilient.
lnd will now properly penalize attempts of larger "wumbo" sized payments proportionally. This will serve to ensure that clients with less active failure information are able to properly prune the search space by increasing the attempt cost for larger payments. New flags has been added to allow users to configure the attempt cost for this value (
attemptcostppm). We encourage users taht frequently send larger payments to tweak these parameters to find what works best, and ideally communicate this information back to the maintainers of
lnd so we can better tune the current default value.
Graph Download Optimizations
lnd will now batch all insertion operations related to channel graph which should greatly speed up initial graph download. Initial becnhmarks show this change resluting in a 3x speed increase, with further gains likely being seen on mobile and more constrained platforms.
Peer to Peer Updates
A new flag has been added to
lnd to enforce a global connection timeout when trying to connect out to new peers. Setting a lower value for this new command line option (
timeout) will mean that
lnd will give up on unrechable peers much sooner than before, which can be useful when attempting to connect to a set of addresses to open chnnel to a peer.
Automatic Database Compaction
The most important data of any
lnd node is stored in its channel database (
channel.db). The database library currently used for this DB is
bbolt which by design does not give back free space to the file system, even if data is deleted from the DB. This can lead to large DB files and slow startup times. Compaction is the process of creating a fresh copy of a
bbolt database that only contains data and no "reserved free space". This process also de-fragments and validates the integrity of the data.
Automatic compaction of the
channel.db can now be turned on with the flag
--db.bolt.auto-compact. By default this will compact on startup, if the last compaction was more than a week ago. The flag
--db.bolt.auto-compact-min-age can also be set to
0 to force compaction on every startup, independent of how long ago it happened last.
Anchor Output Channels
lnd will now open the new channel type dubbed "anchor channels" by default if both peers support it. This is a channel type that has been available to advanced users since lnd v0.10, but it has seen a few updates that makes it even safer and useful in high fee scenarios, and it is now in line with a proposed BOLT change.
The anchor channel type is a new type of channel that is much safer in high fee scenarios, as it allows bumping the fees after the channel has been force closed, instead of making the peers agreeing on a future close fee. This is also a nice UX improvement, as less of the channel capacity needs to go towards the commitment fee reserve, and can instead be used for payments. In addition it allows bundling multiple HTLC transactions together into one, potentially saving on chain fees in force close scenarios.
The commitment transaction still needs to be signed up front with a fee that ensures its mempool acceptance, and this fee now defaults to 10 sat/vbyte. This can be tuned by the
--max-commit-fee-rate-anchors flag, but this should be used with caution. One can opt-out of the anchor channel type for new channels by setting the
Static Remote Key Feature Bit Required
This new version of
lnd now requires channels that use a static remote key,
AKA "tweakless commitments". This change
improves safety and security for users as now when a channel is force closed by
the remote party, the funds will go directly to a user control key. Prior
lnd have supported this channel type, but
lnd will now
only allow this type of channel when making channels with new peer.
Lnd will waive this requirement in the case where it still has legacy channels with a peer. This ensures that lnd can still
connect to nodes it has existing channels with, even if they do not understand
the feature bit.
Improved End to End Payment Security
The MPP protocol upgrade included a so called "payment address" that improves
end-to-end payment security by requiring the sender to include a special nonce
in the onion payload specify by the receiver. As intermediate nodes can't guess
this secret ahead of time, and it's encrypted in the onion only to the finally
receiver, they thwarts a large class of probing and de-anonymization attacks.
This new release of
lnd will now require this feature bit set in any new
invoices it creates, which
means all payments that don't include this new payment secret will be rejected.
The internal wallet can now create and sign PSBTs. In combination with the
ListUnspent RPC this allows RPC users to implement full coin control.
This feature also takes us one step closer to the goal of supporting watch-only on-chain wallets in
lnd where an online node would only have public keys to track the UTXOs and would delegate the signing to a non-networked
lnd node that has the private keys, all through using PSBTs.
Read more about the possible use cases and dive into the examples in our PSBT documentation.
Leveraging the power of GitHub Workflows, we now automatically build and push docker images of all our releases to Docker Hub. This includes images for
The distinction between the production
Dockerfile and the development
dev.Dockerfile were made more clear in the documentation.
The release binaries for all OS/architectures are now also built by a GitHub Workflow. The deterministic build system introduced in a previous release allows us to independently build and sign the binaries locally. Signatures of more than just one developer will be added to releases in the future.
The experiential build tag has been removed for the
assumechanvalid flag that is used to prevent long rescans for neutrino nodes.
Our continuous integration pipeline, most notably our integration tests, has received a number of improvements and bug fixes making them considerably faster and somewhat more stable:
- An integration test suite running against a
bitcoind with the TX index disabled was added.
- The ~70 integration tests are now split into 4 parts and run in parallel reducing the execution time by ~50%.
- Log files are only uploaded to termbin.com and file.io for failed runs and the
bitcoind binaries are extracted from a docker image instead of being downloaded, shaving off a few more minutes from the total itest execution time.
- The test harness for the
btcd node used as the mining node were improved to fix port collisions which resulted in flaky tests.
- A check was added that forces new command line flags to also be documented in the
- A new make target for itest flake hunting was added.
- New make targets for running fuzz tests were added.
- Build tags were removed from the integration test files, allowing the linter to check those as well.
Encode functions now have corresponding fuzz tests in the
brontide fuzz tests have been fixed.
- Fuzz testing has been optimized to instruct
gofuzz to always mutate the input.
Contract Court Performance Improvements
Performance improvements were made to the contract court subsystem which is responsible for closing out channels on chain and taking on-chain actions required to fully resolve the channel. The number of database transactions required to start up the subsystem has been reduced from one per channel to a single transaction, which reduces startup time. Improvements to the way the subsystem consumes new blocks from its backing bitcoin node have also improved the memory footprint of the system.
Extended Health Checks
A new optional
healthcheck has been added to insturct
lnd to restart itself in order to refresh an expired RPC TLS cert. This change is useful in containerized contexts such as k8s, where an auto restarting
lnd is able to propagate any auth changes in a decoupled manner upon restart.
Database contention has been reduced in the link by batching removal of forwarding packages. The removal timer has also been increased from 1 minute to 1 hour.
A bug has been fixed in our non-strict forwarding randomization to ensure we explciitly randomize our link sleection rather than relying on the undefined ordering of map interation in the Go spec.
Peer Flap Rate Tracking
An update to the channel fitness subsystem has introduced tracking of the number of times lnd is connected and disconnected from each of its peers. This information is surfaced in the output of the
The flap rate we have recorded for peers is also used to rate limit the amount of data lnd will store to track the peer’s uptime. If a peer has a high flap rate, lnd will reduce the amount of data it stores in memory, resulting in more aggregated uptime information. This change is intended to protect against constantly flapping peers, and will have little effect on peers that are consistently online with the occasional restart. To ensure that we do not permanently punish a peer for a period of instability long in the past, the flap rate we track for peers is exponentially cooled down over time.
RPC Enchancements & Bug Fixes
Uniform Unconfirmed Coin Selection for SendCoins+
lnd now allows all RPC calls that craft and send transactions to spend unconfirmed coins.
This change the following RPCs:
We've added two new parameters for these methods, following the same format as used for
Macaroon Root ID Key Rotation
lnd now supports root ID key rotation. This allows the baker (creator) of a set of macaroons to invalidate them all by deleting and regenerating the root key used to generate the macaroons. This feature is a useful security tool, as if an application/system that uses lnd's macaroons in a fine grained manner is compromised, the admin is able to revoke all generated macaroons.
Several new calls have been added to allow users to take advantage of this feature, namely:
lncli bakemacaroon call now accepts a new parameter:
root_key_id. This new field is an integer that can be used to rotate root ID keys.
* A new
lncli listmacaroonids command has been added to allow callers to monitor all their existing allocated root IDs.
* A new
lncli deletemacaroonid call has been added which implements macaroon revocation by allowing the caller to delete a specified root key ID.
New Verbose Output for
lncli channebalance call now returns much more information than before in order to give users more insight w.r.t exactly how their funds are distributed off-chain. An output of the new call resmbles the following:
Note that the first two fields (
pending_open_balance) are now deprecated and will be removed in the future. Callers should use the new fields that return both
Raw Key Support for
DeriveSharedKey now accepts a raw public key in addition to key locator.
Additional HTLC Information in ListChannels
ListChannels call will now return additional information about the set of linked HTLCs in a channel. Namely, we'll now return:
htlc_index of the HTLC within the channel
forwarding_channel, or the channel that forwarded the HTLC to the targte channel
forwarding_htlc_index, or the HTLC index on the forwarded channel.
Automated Let's Encrypt Certificates
A new series of command line flags have been added to
lnd which allows users to automatically obtain and renew a Let's Encrypt Certificate for the RPC interface of their
lnd node. With this change, in certain configurations, callers will be able to hit an
lnd now without having to manually store and update the
New flags added to the
lnd command line and
--letsencryptport: The port on which lnd will listen for Let's Encrypt challenges. Let's Encrypt will always try to contact on port 80. Often non-root processes are not allowed to bind to ports lower than 1024. This configuration option allows a different port to be used, but must be used in combination with port forwarding from port 80.
--letsencryptdir: The directory to store Let's Encrypt certificates within. By default this is
--letsencryptdomain: Request a Let's Encrypt certificate for the domain specified using this flag.
lncli cannot find a
tls.cert file, it will assume the server has a valid (Let's Encrypt) certificate. It is important to pass the domain name as a command line flag to
lncli --rpcserver my.domain.org:10009
This is necessary as well when connecting to
Custom Routing Hints for AddHoldInvoice
AddHoldInvoice RPC call now allows the users to specify their own custom routing hints.
Allow No RPC Auth on Private Addresses
A new config evaluation has been added to allow user to instruct
lnd that it should allow RPC requests with no authentiation only if
lnd is listening on a private address. This makes certain Docker based configurations more user friendly, as any dependent containers no longer need to obtain and update lnd's RPC authentication information. Assuming
lnd is only listening on a non-public private interface, then the
--no-macaroons config option is now valid.
New Channel Acceptor Parameters
Additional fields have been added to the
ChannelAcceptor API, which allow custom setting of custom errors for the remote peer, an upfront shutdown address for the channel (if supported by the peer), and more. Note that the error provided will be sent to the peer verbatim, so should not contain sensitive information.
Maximum Local CSV
When opening a channel, the remote party can specify the CSV delay for your funds. This value determines the amount of time that your balance will be unavailable in the case where your force close the channel. A
max_local_csv parameter has been added to allow setting of custom limitations on this value. For outgoing channels, this can be set using the
max_local_csv field in the
OpenChannel request. The
maxlocaldelay config value can be used to set a default maximum value for all channels.
Disable TLS for REST
It is now possible to disable TLS for REST RPC using
This release sees the removal of several components from the main
fundingmanager.go and tests are moved to the
chainparams.go have been moved to the
mock.go has been removed in favor of the
- A global variable
activeNetParams has been removed.
peer package's dependency on
brontide has been removed.
The DNS servers to use for initial peer bootstrapping can now be specified to overwrite the hard coded default values.
All supported command line flags are now also properly documented in the
A new flag has been added to instruct
lnd to timeout early if it can't obtain the file lock on bolt DB.
Multi node management
Hosting nodes on non-trusted (cloud) hardware was made safer by adding a stateless initialization mode that instructs
lnd to not store any unencrypted macaroons on the host's file system. Instead, the admin macaroon is returned in the response of the wallet creation request and must be stored by the caller.
To support the stateless initialization mode mentioned above on the client side as well, configuration profiles for
lncli can now be created. Those profiles make it easy to interact with multiple nodes from the same client machine. For additional security the macaroons stored in the profiles can optionally be encrypted with a password.
Forcing the on-chain wallet to rescan its state from chain was made easier by adding the
--reset-wallet-transactions flag to
lnd that replaces the functionality previously only available in the external tool
Individual subsystem log levels
A change that makes it possible to set the log level for individual subsystems was merged. One can now specify a global log level, and subsystem log levels that will override the global setting:
Contributors (Alphabetical Order)
Johan T. Halseth
Juan Pablo Civile
Yaacov Akiba Slama