lnd: v0.5.1-beta-rc1

This release is minor release of lnd, which includes several fixes and optimizations to make lnd even better. This time around one of the points of focus has been around the reliability, robustness and speed of the Neutrino backend, which is now in a state where it can be used for building applications for the Bitcoin testnet. This will let us sort out the last quirks and performance bottlenecks before it is ready to be enabled for mainnet.

Breaking changes

  • Remove NewWitnessAddress: The RPC NewWitnessAddress has been removed. Since we are only using witness addresses, addresses can be fetched using NewAddress.

A few RPCs have changed their behavior slightly, see the the RPC changes section.

Migrations

We now store the wallet’s birthday block to speed up rescans. If you are upgrading from an older version of lnd you should at startup see something like

2018-11-15 13:11:15.636 [INF] LTND: Version: 0.5.0-beta commit=v0.5.1-beta-rc1, build=production, logging=default
2018-11-15 13:11:15.636 [INF] LTND: Active chain: Bitcoin (network=testnet)
2018-11-15 13:11:15.638 [INF] CHDB: Checking for schema update: latest_version=6, db_version=6
2018-11-15 13:11:15.649 [INF] RPCS: password RPC server listening on 127.0.0.1:10009
2018-11-15 13:11:15.649 [INF] RPCS: password gRPC proxy started at 127.0.0.1:8080
2018-11-15 13:11:15.650 [INF] LTND: Waiting for wallet encryption password. Use `lncli create` to create a wallet, `lncli unlock` to unlock an existing wallet, or `lncli changepassword` to change the password of an existing wallet and unlock it.
2018-11-15 13:11:27.207 [INF] LNWL: Applying wallet transaction manager migration #2
2018-11-15 13:11:27.207 [INF] LNWL: Dropping wallet transaction history
2018-11-15 13:11:27.208 [INF] LNWL: Applying wallet address manager migration #6
2018-11-15 13:11:27.209 [INF] LNWL: Setting the wallet's birthday block from timestamp=2018-11-12 19:15:05 +0100 CET
2018-11-15 13:11:27.211 [INF] LNWL: Estimated birthday block from timestamp=2018-11-12 19:15:05 +0100 CET: height=408929, hash=0000000000114718f816515d333731e0f9982a4e703665771a192beadeb88398
2018-11-15 13:11:27.211 [INF] LNWL: Applying wallet address manager migration #7
2018-11-15 13:11:28.319 [INF] LNWL: Opened wallet

Note that it will then perform a rescan from the birthday, which might take a while. By setting the debug level to debug you’ll be able to track the process of this rescan.

Verifying the Release

In order to verify the release, you’ll need to have gpg or gpg2 installed on your system. This release candidate is only signed by halseth:

curl https://keybase.io/halseth/pgp_keys.asc | gpg --import

Once you have his PGP key you can verify the release (assuming manifest-v0.5.1-beta-rc1.txt and manifest-v0.5.1-beta-rc1.txt.sig are in the current directory) with:

gpg --verify manifest-v0.5.1-beta-rc1.txt.sig

You should see the following if the verification was successful:

gpg: assuming signed data in 'manifest-v0.5.1-beta-rc1.txt'
gpg: Signature made Thu Nov 15 10:25:29 2018 CET
gpg:                using RSA key 7AB3D7F5911708842796513415BAADA29DA20D26
gpg: Good signature from "Johan T Halseth <johanth@gmail.com>" [unknown]

That will verify the signature on the main manifest page which ensures integrity and authenticity of the binaries you’ve downloaded locally. Next, depending on your operating system you should then re-calculate the sha256 sum of the binary, and compare that with the following hashes (which are

cat manifest-v0.5.1-beta-rc1.txt

One can use the shasum -a 256 <file name here> tool in order to re-compute the sha256 hash of the target binary for your operating system. The produced hash should be compared with the hashes listed above and they should match exactly.

Finally, you can also verify the tag itself with the following command:

git verify-tag v0.5.1-beta-rc1

⚡️⚡️⚡️ OK, now to the rest of the release notes! ⚡️⚡️⚡️

Notable changes

Neutrino

Neutrino is a new light client for Bitcoin, that is more private and tailored for use on the Lightning Network than previous light clients. For more information, check out this blog post.

Since the previous release of lnd, the version of neutrino used has gained a lot in terms of stability and speed. We now start catching up to the necessary filter hashes the moment we have enough block headers to verify them. This let us do most of the header fetching in parallel, drastically speeding up initial sync.

A number of fixes has also been deployed to ensure neutrino correctly handles header and filter responses from multiple peers in parallel, and even peers serving invalid filters.

Height hint cache

When lnd opens channels to other nodes, it must always make sure the counterparty hasn’t unilaterally closed the channel without its consent. This is done by scanning all blocks following the opening of the channel, to ensure no closing transaction has been confirmed. This is a quick check on full nodes, but since neutrino doesn’t keep the entire chain around, it must request filters (and potentially blocks) from the network to perform this check.

lnd performs this scan on every restart, checking if any of its channels have been closed while offline. With the addition of height hint caches we are now able to cache the results of previous such scans, letting us start the scan from where we left off, instead of scanning the chain all the way from the block where the channel was opened. With these additions, both txid confirmations and spend detection is now cached. This saves a lot on time and bandwidth at startup, letting light client users get back into sync with the network faster.

Validating received channel updates

When sending a payment, there exists situation where the graph information the client has is not up to date with the nodes it is attempting to route through. In these cases the routing nodes will send an error back along with the updated information, such that the client can retry the payment with the correct parameters.

Previously we didn’t check the signatures on this updated information, making it possible for nodes to respond with information for channels they didn’t control. With an added validity check, this is no longer the case, and all graph information must be signed by the controlling parties.

forgetchannel RPC

A new RPC has been added for debug builds, that let users forcefully remove channels from their databases. This must be used with caution, as all state for the targeted channels will be lost, essentially making it impossible to recover any funds kept in these channels. This RPC is added to let advanced users get rid of leftover channels from earlier versions of lnd, where channels were not always cleaned up properly. Channels that had their funding transactions broadcast with a fee too small can also be removed with this RPC. To make sure no one uses this functionality by accident, it is only possible to use in debug builds.

Build package

A new package build has been added to lnd, greatly simplifying the process of adding build dependent changes, and improving logging during unit tests. The version string of lnd will with this change include the version tag, commit hash and number of commits since the tag. Logs from unit tests can be inspected by passing log="stdout <loglevel>" to make unit.

go 1.11 support

The recommended go version has been upgraded to v1.11! The project now supports go 1.10 and go 1.11.

gRPC 1.15.0

lnd uses gRPC for its RPC interface, making it easy to gain access to the functionality of lnd from a wide set of languages. The gRPC version used was due for an upgrade, and has been updated to v1.15.0, up from v1.5.2. You shouldn’t be seeing any breaking changes with this upgrade.

Reject insane timelocks

We now check that forwarded payments don’t impose a timelock that is too far in the future. Earlier an attacker willing to lock up his own funds could craft routes where funds got locked up for a very long time.

New sweep package

A new package sweep has been introduced, intended to take care of all kinds of sweeps within lnd, such as retrieving funds from closed channels. This is part of an ongoing effort to reduce the statefulness of funds sweeping, and be smarter about batching sweeps together in bigger transactions to save on chain fees.

rejectpush option

When opening a channel to a peer on the network, you have the option to immediately send some funds as part of the opening process (what is called push amount). For most users accepting incoming channels this is not a problem (hey, free money!), but some merchants have seen customers pushing funds to them by mistake, resulting in an annoying refund process. Now there’s a new configuration option --rejectpush that makes the node reject any incoming channels that include a push amount.

Use SendToRoute with private channels

SendToRoute is an RPC that is used by advanced users to send payments along custom paths. Previously it had the restriction that the edges used in these custom paths were known to lnd. A new option pub_key in the route description given to SendToRoute lets lnd create the route without relying on the channel graph, making it possible to attempt payments even for channels unknown to lnd, such as private channels. This works since the payment is encrypted with the given public key, and now all information needed to construct the payload can be supplied over the RPC.

RPC changes

  • Deprecating untyped value fields: Since RPC fields are not always typed to indicate their unit, there has been confusion especially whether satoshis or millisatoshis are used for certain RPCs. Going forward fields should indicate their type (ending with _sat or _msat), and you might see the existing, untyped fields getting deprecated.
  • IncludeUnannounced flag for DescribeGraph: The DescribeGraph RPC is used to get a list of the current information found in the local graph, including all known nodes and channels. Earlier this would include information about private nodes and channels you knew about. With this recent change, the caller must explicitly set the IncludeUnannounced flag if it wishes this private information to be part of the output, to avoid this information being leaked involuntarily.
  • Prevent spending unconfirmed funds by default: The SpendUnconfirmed flag must now be set for the wallet to use unconfirmed inputs when opening channels.
  • Reversed QueryInvoices consistency: When querying for an offset of invoices in the reverse order, we would earlier list the first available invoice found after this offset. This was inconsistent with the non-reversed case, and has been changed.
  • Number of inactive channels in GetInfo: The GetInfo RPC call now shows the number of inactive channels in addition to the number of active.

Changelog

The full list of changes since 0.5-beta can be found here: * https://github.com/lightningnetwork/lnd/compare/v0.5-beta...v0.5.1-beta-rc1

Contributors (Alphabetical Order)

AdamISZ Alex Bosworth Boblechinois CirroStorm Conner Fromknecht Daniel McNally Dave Kerr Desuuuu ErikEk Harald Nordgren Johan T. Halseth Joost Jager Olaoluwa Osuntokun Oliver Gugger Oscar Lafarga Patrick Roei Erez Vincent Woo Wilmer Paulino Xavi Soler bluetegu bob-333 chokoboko maurycy sevastos tailnode whythat