Bitcoin: v0.17.0

Bitcoin Core version 0.17.0 is now available from:

https://bitcoincore.org/bin/bitcoin-core-0.17.0/

Notable changes

Changed configuration options

  • -includeconf=<file> can be used to include additional configuration files. Only works inside the bitcoin.conf file, not inside included files or from command-line. Multiple files may be included. Can be disabled from command- line via -noincludeconf. Note that multi-argument commands like -includeconf will override preceding -noincludeconf, i.e. noincludeconf=1 includeconf=relative.conf

as bitcoin.conf will still include relative.conf.

GUI changes

  • Block storage can be limited under Preferences, in the Main tab. Undoing this setting requires downloading the full blockchain again. This mode is incompatible with -txindex and -rescan.

External wallet files

The -wallet=<path> option now accepts full paths instead of requiring wallets to be located in the -walletdir directory.

Newly created wallet format

If -wallet=<path> is specified with a path that does not exist, it will now create a wallet directory at the specified location (containing a wallet.dat data file, a db.log file, and database/log.?????????? files) instead of just creating a data file at the path and storing log files in the parent directory. This should make backing up wallets more straightforward than before because the specified wallet path can just be directly archived without having to look in the parent directory for transaction log files.

For backwards compatibility, wallet paths that are names of existing data files in the -walletdir directory will continue to be accepted and interpreted the same as before.

Dynamic loading and creation of wallets

Previously, wallets could only be loaded or created at startup, by specifying -wallet parameters on the command line or in the bitcoin.conf file. It is now possible to load, create and unload wallets dynamically at runtime:

  • Existing wallets can be loaded by calling the loadwallet RPC. The wallet can be specified as file/directory basename (which must be located in the walletdir directory), or as an absolute path to a file/directory.
  • New wallets can be created (and loaded) by calling the createwallet RPC. The provided name must not match a wallet file in the walletdir directory or the name of a wallet that is currently loaded.
  • Loaded wallets can be unloaded by calling the unloadwallet RPC.

This feature is currently only available through the RPC interface.

Coin selection

Partial spend avoidance

When an address is paid multiple times the coins from those separate payments can be spent separately which hurts privacy due to linking otherwise separate addresses. A new -avoidpartialspends flag has been added (default=false). If enabled, the wallet will always spend existing UTXO to the same address together even if it results in higher fees. If someone were to send coins to an address after it was used, those coins will still be included in future coin selections.

Configuration sections for testnet and regtest

It is now possible for a single configuration file to set different options for different networks. This is done by using sections or by prefixing the option with the network, such as:

main.uacomment=bitcoin
test.uacomment=bitcoin-testnet
regtest.uacomment=regtest
[main]
mempoolsize=300
[test]
mempoolsize=100
[regtest]
mempoolsize=20

If the following options are not in a section, they will only apply to mainnet: addnode=, connect=, port=, bind=, rpcport=, rpcbind= and wallet=. The options to choose a network (regtest= and testnet=) must be specified outside of sections.

‘label’ and ‘account’ APIs for wallet

A new ‘label’ API has been introduced for the wallet. This is intended as a replacement for the deprecated ‘account’ API. The ‘account’ can continue to be used in V0.17 by starting bitcoind with the ‘-deprecatedrpc=accounts’ argument, and will be fully removed in V0.18.

The label RPC methods mirror the account functionality, with the following functional differences:

  • Labels can be set on any address, not just receiving addresses. This functionality was previously only available through the GUI.
  • Labels can be deleted by reassigning all addresses using the setlabel RPC method.
  • There isn’t support for sending transactions from a label, or for determining which label a transaction was sent from.
  • Labels do not have a balance.

Here are the changes to RPC methods:

Deprecated Method New Method Notes
getaccount getaddressinfo getaddressinfo returns a json object with address information instead of just the name of the account as a string.
getaccountaddress n/a There is no replacement for getaccountaddress since labels do not have an associated receive address.
getaddressesbyaccount getaddressesbylabel getaddressesbylabel returns a json object with the addresses as keys, instead of a list of strings.
getreceivedbyaccount getreceivedbylabel no change in behavior
listaccounts listlabels listlabels does not return a balance or accept minconf and watchonly arguments.
listreceivedbyaccount listreceivedbylabel Both methods return new label fields, along with account fields for backward compatibility.
move n/a no replacement
sendfrom n/a no replacement
setaccount setlabel Both methods now:
  • allow assigning labels to any address, instead of raising an error if the address is not receiving address.
  • delete the previous label associated with an address when the final address using that label is reassigned to a different label, instead of making an implicit getaccountaddress call to ensure the previous label still has a receiving address.
Changed Method Notes
addmultisigaddress Renamed account named parameter to label. Still accepts account for backward compatibility if running with ‘-deprecatedrpc=accounts’.
getnewaddress Renamed account named parameter to label. Still accepts account for backward compatibility. if running with ‘-deprecatedrpc=accounts’
listunspent Returns new label fields. account field will be returned for backward compatibility if running with ‘-deprecatedrpc=accounts’
sendmany The account named parameter has been renamed to dummy. If provided, the dummy parameter must be set to the empty string, unless running with the -deprecatedrpc=accounts argument (in which case functionality is unchanged).
listtransactions The account named parameter has been renamed to dummy. If provided, the dummy parameter must be set to the string *, unless running with the -deprecatedrpc=accounts argument (in which case functionality is unchanged).
getbalance account, minconf and include_watchonly parameters are deprecated, and can only be used if running with ‘-deprecatedrpc=accounts’

BIP 174 Partially Signed Bitcoin Transactions support

BIP 174 PSBT is an interchange format for Bitcoin transactions that are not fully signed yet, together with relevant metadata to help entities work towards signing it. It is intended to simplify workflows where multiple parties need to cooperate to produce a transaction. Examples include hardware wallets, multisig setups, and CoinJoin transactions.

Overall workflow

Overall, the construction of a fully signed Bitcoin transaction goes through the following steps:

  • A Creator proposes a particular transaction to be created. He constructs a PSBT that contains certain inputs and outputs, but no additional metadata.
  • For each input, an Updater adds information about the UTXOs being spent by the transaction to the PSBT.
  • A potentially other Updater adds information about the scripts and public keys involved in each of the inputs (and possibly outputs) of the PSBT.
  • Signers inspect the transaction and its metadata to decide whether they agree with the transaction. They can use amount information from the UTXOs to assess the values and fees involved. If they agree, they produce a partial signature for the inputs for which they have relevant key(s).
  • A Finalizer is run for each input to convert the partial signatures and possibly script information into a final scriptSig and/or scriptWitness.
  • An Extractor produces a valid Bitcoin transaction (in network format) from a PSBT for which all inputs are finalized.

Generally, each of the above (excluding Creator and Extractor) will simply add more and more data to a particular PSBT. In a naive workflow, they all have to operate sequentially, passing the PSBT from one to the next, until the Extractor can convert it to a real transaction. In order to permit parallel operation, Combiners can be employed which merge metadata from different PSBTs for the same unsigned transaction.

The names above in bold are the names of the roles defined in BIP174. They’re useful in understanding the underlying steps, but in practice, software and hardware implementations will typically implement multiple roles simultaneously.

RPCs

  • converttopsbt (Creator) is a utility RPC that converts an unsigned raw transaction to PSBT format. It ignores existing signatures.
  • createpsbt (Creator) is a utility RPC that takes a list of inputs and outputs and converts them to a PSBT with no additional information. It is equivalent to calling createrawtransaction followed by converttopsbt.
  • walletcreatefundedpsbt (Creator, Updater) is a wallet RPC that creates a PSBT with the specified inputs and outputs, adds additional inputs and change to it to balance it out, and adds relevant metadata. In particular, for inputs that the wallet knows about (counting towards its normal or watch-only balance), UTXO information will be added. For outputs and inputs with UTXO information present, key and script information will be added which the wallet knows about. It is equivalent to running createrawtransaction, followed by fundrawtransaction, and converttopsbt.
  • walletprocesspsbt (Updater, Signer, Finalizer) is a wallet RPC that takes as input a PSBT, adds UTXO, key, and script data to inputs and outputs that miss it, and optionally signs inputs. Where possible it also finalizes the partial signatures.
  • finalizepsbt (Finalizer, Extractor) is a utility RPC that finalizes any partial signatures, and if all inputs are finalized, converts the result to a fully signed transaction which can be broadcast with sendrawtransaction.
  • combinepsbt (Combiner) is a utility RPC that implements a Combiner. It can be used at any point in the workflow to merge information added to different versions of the same PSBT. In particular it is useful to combine the output of multiple Updaters or Signers.
  • decodepsbt is a diagnostic utility RPC which will show all information in a PSBT in human-readable form, as well as compute its eventual fee if known.

Upgrading non-HD wallets to HD wallets

Since Bitcoin Core 0.13.0, creating new BIP 32 Hierarchical Deterministic wallets has been supported by Bitcoin Core but old non-HD wallets could not be upgraded to HD. Now non-HD wallets can be upgraded to HD using the -upgradewallet command line option. This upgrade will result in the all keys in the keypool being marked as used and a new keypool generated. A new backup must be made when this upgrade is performed.

Additionally, -upgradewallet can be used to upgraded from a non-split HD chain (all keys generated with m/0'/0'/i') to a split HD chain (receiving keys generated with 'm/0'/0'/i' and change keys generated with m'/0'/1'/i'). When this upgrade occurs, all keys already in the keypool will remain in the keypool to be used until all keys from before the upgrade are exhausted. This is to avoid issues with backups and downgrades when some keys may come from the change key keypool. Users can begin using the new split HD chain keypools by using the newkeypool RPC to mark all keys in the keypool as used and begin using a new keypool generated from the split HD chain.

HD Master key rotation

A new RPC, sethdseed, has been introduced which allows users to set a new HD seed or set their own HD seed. This allows for a new HD seed to be used. A new backup must be made when a new HD seed is set.

Low-level RPC changes

  • The new RPC scantxoutset can be used to scan the UTXO set for entries that match certain output descriptors. Refer to the output descriptors reference documentation for more details. This call is similar to listunspent but does not use a wallet, meaning that the wallet can be disabled at compile or run time. This call is experimental, as such, is subject to changes or removal in future releases.

  • The createrawtransaction RPC will now accept an array or dictionary (kept for compatibility) for the outputs parameter. This means the order of transaction outputs can be specified by the client.

  • The fundrawtransaction RPC will reject the previously deprecated reserveChangeKey option.

  • sendmany now shuffles outputs to improve privacy, so any previously expected behavior with regards to output ordering can no longer be relied upon.

  • The new RPC testmempoolaccept can be used to test acceptance of a transaction to the mempool without adding it.

  • JSON transaction decomposition now includes a weight field which provides the transaction’s exact weight. This is included in REST /rest/tx/ and /rest/block/ endpoints when in json mode. This is also included in getblock (with verbosity=2), listsinceblock, listtransactions, and getrawtransaction RPC commands.

  • New fees field introduced in getrawmempool, getmempoolancestors, getmempooldescendants and getmempoolentry when verbosity is set to true with sub-fields ancestor, base, modified and descendant denominated in BTC. This new field deprecates previous fee fields, such as fee, modifiedfee, ancestorfee and descendantfee.

  • The new RPC getzmqnotifications returns information about active ZMQ notifications.

  • When bitcoin is not started with any -wallet=<path> options, the name of the default wallet returned by getwalletinfo and listwallets RPCs is now the empty string "" instead of "wallet.dat". If bitcoin is started with any -wallet=<path> options, there is no change in behavior, and the name of any wallet is just its <path> string.

  • Passing an empty string ("") as the address_type parameter to getnewaddress, getrawchangeaddress, addmultisigaddress, fundrawtransaction RPCs is now an error. Previously, this would fall back to using the default address type. It is still possible to pass null or leave the parameter unset to use the default address type.

  • Bare multisig outputs to our keys are no longer automatically treated as incoming payments. As this feature was only available for multisig outputs for which you had all private keys in your wallet, there was generally no use for them compared to single-key schemes. Furthermore, no address format for such outputs is defined, and wallet software can’t easily send to it. These outputs will no longer show up in listtransactions, listunspent, or contribute to your balance, unless they are explicitly watched (using importaddress or importmulti with hex script argument). signrawtransaction* also still works for them.

  • The getwalletinfo RPC method now returns an hdseedid value, which is always the same as the incorrectly-named hdmasterkeyid value. hdmasterkeyid will be removed in V0.18.

  • The getaddressinfo RPC method now returns an hdseedid value, which is always the same as the incorrectly-named hdmasterkeyid value. hdmasterkeyid will be removed in V0.18.

  • Parts of the validateaddress RPC method have been deprecated and moved to getaddressinfo. Clients must transition to using getaddressinfo to access this information before upgrading to v0.18. The following deprecated fields have moved to getaddressinfo and will only be shown with -deprecatedrpc=validateaddress: ismine, iswatchonly, script, hex, pubkeys, sigsrequired, pubkey, addresses, embedded, iscompressed, account, timestamp, hdkeypath, hdmasterkeyid.

  • signrawtransaction is deprecated and will be fully removed in v0.18. To use signrawtransaction in v0.17, restart bitcoind with -deprecatedrpc=signrawtransaction. Projects should transition to using signrawtransactionwithkey and signrawtransactionwithwallet before upgrading to v0.18.

Other API changes

  • The inactivehdmaster property in the dumpwallet output has been corrected to inactivehdseed

Logging

  • The log timestamp format is now ISO 8601 (e.g. “2018-02-28T12:34:56Z”).

  • When running bitcoind with -debug but without -daemon, logging to stdout is now the default behavior. Setting -printtoconsole=1 no longer implicitly disables logging to debug.log. Instead, logging to file can be explicitly disabled by setting -debuglogfile=0.

Transaction index changes

The transaction index is now built separately from the main node procedure, meaning the -txindex flag can be toggled without a full reindex. If bitcoind is run with -txindex on a node that is already partially or fully synced without one, the transaction index will be built in the background and become available once caught up. When switching from running -txindex to running without the flag, the transaction index database will not be deleted automatically, meaning it could be turned back on at a later time without a full resync.

Miner block size removed

The -blockmaxsize option for miners to limit their blocks’ sizes was deprecated in V0.15.1, and has now been removed. Miners should use the -blockmaxweight option if they want to limit the weight of their blocks.

Python Support

Support for Python 2 has been discontinued for all test files and tools.