Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wallet TX count and time indexing #605

Closed
wants to merge 70 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
680e20a
test: add utility for test fs cleanup
braydonf Sep 10, 2018
5746ee5
test: add utility to sleep for desired ms
braydonf Sep 10, 2018
248c3d4
wallet: add test cases for wallet tx count and time indexing
braydonf Sep 11, 2018
41a371f
wallet: add db layout for tx count and monotonic time
braydonf Sep 11, 2018
a53f966
wallet: implement monotonic time
braydonf Sep 11, 2018
fe9f3f3
wallet: add tx count index for confirmed txs
braydonf Sep 11, 2018
b5a9074
wallet: add rpc methods for tx count and monotonic time
braydonf Sep 11, 2018
4f96017
test: restore count to generateBlocks regtest util
braydonf Sep 12, 2018
c931791
docs: after is exclusive
braydonf Sep 12, 2018
c289780
wallet: monotonic time indexing
braydonf Sep 13, 2018
56ef021
test: add reorg to wallet time test
braydonf Sep 21, 2018
bbaa222
test: monotonic time reorg, incomplete
braydonf Oct 17, 2018
9b2c585
wallet: rollback monotonic time
braydonf Oct 23, 2018
0f99886
wallet: add spv monotonic time reorg
braydonf Oct 23, 2018
54ff89a
wallet: reorg tx monotonic time and count indexes
braydonf Oct 25, 2018
99a396e
wallet: refactor monotonic index method
braydonf Oct 27, 2018
2c5ddb0
wallet: options for list history methods
braydonf Oct 27, 2018
1def771
wallet: add indexing for unconfirmed count and time
braydonf Nov 6, 2018
f9cc58d
wallet: add methods to query for unconfirmed txs
braydonf Nov 8, 2018
ea7c27c
test: confirm and unconfirm txs
braydonf Nov 12, 2018
9adf54c
wallet: combine `getPending` and `listUnconfirmed`
braydonf Nov 12, 2018
69e2f1b
wallet: http endpoints for history and unconfirmed txs
braydonf Nov 20, 2018
d4afc1d
wallet: test spv node http endpoints and w/ fixes
braydonf Nov 21, 2018
dafad8d
test: remove `.skip` function call, it's not implemented in bmocha
braydonf Dec 5, 2018
9dc1673
test: wallet rescan test
braydonf Dec 18, 2018
e8c6410
wallet: use height + index for count index
braydonf Dec 19, 2018
30d79d4
test: remove spv from wallet rescan test
braydonf Dec 19, 2018
d8ff085
test: assertions for wallet rescan test
braydonf Dec 19, 2018
a0e5729
wallet: use median time past for time indexing
braydonf Dec 20, 2018
d9f4601
wallet: linting
braydonf Dec 20, 2018
857d07c
wallet: use brhash for http after param
braydonf Dec 20, 2018
7b41f51
test: fix spv test when cpu is loaded
braydonf Dec 20, 2018
6d1748c
wallet: minor cleanup
braydonf Dec 20, 2018
bd29e9f
wallet: cleanup indexes
braydonf Dec 20, 2018
00c652a
wallet: make configurable max transactions limits
braydonf Dec 20, 2018
4108b58
wallet: minor cleanup
braydonf Dec 20, 2018
43199ec
wallet: fix configurable txs limits
braydonf Dec 20, 2018
f66c89d
test: bump wallet http before hook time
braydonf Dec 20, 2018
be59b32
wallet: fix http time tx queries
braydonf Dec 21, 2018
ce7ca8b
wallet: minor linting
braydonf Dec 21, 2018
a803c98
wallet: check rpc min args length
braydonf Dec 21, 2018
49cc51b
test: minor linting
braydonf Dec 21, 2018
5b51439
wallet: fix sorting of time indexing
braydonf Dec 21, 2018
abe2353
wallet: use brhash instead of manually reversing
braydonf Dec 21, 2018
3f41dae
docs: minor linting
braydonf Dec 21, 2018
2fe41f2
wallet: fix sorting of confirmed time index
braydonf Dec 21, 2018
f386a39
wallet: refactor date handling for api
braydonf Dec 21, 2018
0974f72
test: refactor utils
braydonf Dec 21, 2018
8de45e5
wallet: minor cleanup and adjustments
braydonf Dec 21, 2018
cda883a
wallet: minor cleanup and update
braydonf Dec 21, 2018
561b1ae
wallet: update rpc methods with new history functions
braydonf Dec 22, 2018
3fd5216
wallet: use unix epoch time in seconds for rpc history cmds
braydonf Dec 23, 2018
fb46596
test: increase lookahead for test wallet
braydonf Dec 23, 2018
794b417
test: refactor initial blocks
braydonf Dec 25, 2018
1dda1a8
wallet client: add getMedianTime
pinheadmz Jan 8, 2019
cfb8359
wallet: list confirmed and unconfirmed history
braydonf Jan 30, 2019
f0cbe7f
test: update for segwit activation on regtest
braydonf Feb 12, 2019
10ee54f
changelog: update for transaction history api changes
braydonf Feb 12, 2019
7d3c403
wallet: check if options available
braydonf Mar 1, 2019
02b3141
test: listtransactions is deprecated
braydonf Mar 21, 2019
4c559f1
changelog: add note about max-txs
braydonf Mar 26, 2019
b14237d
wallet: minor, refactor and comments
braydonf Mar 26, 2019
62b26fd
test: use `testdir` for prefixes
braydonf May 15, 2019
6833408
test: updates wallet tests to use `forValue`
braydonf May 16, 2019
6024507
test: updates for nyc and ci
braydonf Jun 5, 2019
f339e8c
node: restore error logger argument
braydonf Jun 5, 2019
f75a157
wallet: remove http history date argument
braydonf Jun 5, 2019
1d8c94b
wallet: remove deprecated methods
braydonf Jun 5, 2019
069e3d6
test: minor, cleanup todos
braydonf Jun 5, 2019
7d8ddf8
test: use bsert instead of assert util
braydonf Jun 27, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
changelog: update for transaction history api changes
  • Loading branch information
braydonf committed Jun 27, 2019
commit 10ee54fbda6c3d29e8e6cd9c278d38c443ded858
89 changes: 81 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,8 +45,19 @@ before upgrading.

### Wallet API changes

The wallet has been updated to handle a large number of transactions, there
are several new API methods that have been added and modified to better
support transaction queries. This also includes the ability to query
for transaction history by median-time-past (MTP). It is necessary to
perform a wallet rescan to update the wallet indexes.

#### HTTP

These endpoints have been modified:

- `GET /wallet/:id/tx/history` - The params are now `time`, `after`,
`limit`, and `reverse`.
- `GET /wallet/:id/tx/unconfirmed` - The params are the same as above.
- `PUT /wallet/:id` Creating a watch-only wallet now requires an `accountKey`
argument. This is to prevent bcoin from generating keys and addresses the
user can not spend from.
Expand All @@ -58,19 +69,81 @@ before upgrading.
- Exposes `blocks`, which can will be used if there is no `rate` option.
- Exposes `sort` (Default `true`), that can be used to disable BIP69 sorting.

These endpoints have been deprecated:

- `GET /wallet/:id/tx/range` - Instead use the `time` param for the history and
unconfirmed endpoints.
- `GET /wallet/:id/tx/last` - Instead use `reverse` param for the history and
unconfirmed endpoints.

##### Examples

```
GET /wallet/:id/tx/history?after=<txid>&limit=50&reverse=false
GET /wallet/:id/tx/history?after=<txid>&limit=50&reverse=true
```
By using `after=<txid>` we can anchor pages so that results will not shift
when new blocks and transactions arrive. With `reverse=true` we can change
the order the transactions are returned as _latest to genesis_. The
`limit=<number>` specifies the maximum number of transactions to return
in the result.

```
GET /wallet/:id/tx/history?time=<median-time-past>&limit=50&reverse=false
GET /wallet/:id/tx/history?time=<median-time-past>&limit=50&reverse=true
```
The param `time` is in epoch seconds and indexed based on median-time-past
(MTP) and `date` is ISO 8601 format. Because multiple transactions can share
the same time, this can function as an initial query, and then switch to the
above `after` format for the following pages.

```
GET /wallet/:id/tx/unconfirmed?after=<txid>&limit=50&reverse=false
GET /wallet/:id/tx/unconfirmed?after=<txid>&limit=50&reverse=true
GET /wallet/:id/tx/unconfirmed?time=<time-received>&limit=50&reverse=false
```
The same will apply to unconfirmed transactions. The `time` is in epoch
seconds and indexed based on when the transaction was added to the wallet.

#### RPC

- Bug fix addresses for the `getnewaddress` command with various networks.
- Deprecate the `ismine` and `iswatchonly` fields from the `validateaddress`
command and add `isscript`, `iswitness`, `ischange`, `witness_version`
and `witness_program` to partially match the v0.18.0 Bitcoin Core release
The following methods have bug fixes:
- `getnewaddress` Now supports various networks.

The following have property deprecations and additions:
- `validateaddress` - Deprecate the `ismine` and `iswatchonly` properties,
use the new command `getaddressinfo` instead. The command now has
`isscript`, `iswitness`, `ischange`, `witness_version` and
`witness_program` to partially match the v0.18.0 Bitcoin Core release
(26a2000b0177fd2668b7d82e5aa52829cf2bfdf6)
- Add wallet RPC `getaddressinfo` to return `ismine` and `iswatchonly`
with the correct values instead of their previous values which were
hardcoded to false. Also returns `address`, `scriptPubKey`, `isscript`,
`iswitness`, `witness_version` and `witness_program`.

The following new methods have been added:

- `listhistory` - List history with a limit and in reverse order.
- `listhistoryafter` - List history after a txid _(subsequent pages)_.
- `listhistorybytime` - List history by giving a timestamp in epoch seconds
_(block median time past)_.
- `listunconfirmed` - List unconfirmed transactions with a limit and in
reverse order.
- `listunconfirmedafter` - List unconfirmed transactions after a txid
_(subsequent pages)_.
- `listunconfirmedbytime` - List unconfirmed transactions by time they
where added.
- `getaddressinfo` - Returns `ismine` and `iswatchonly` with the correct
values instead of their previous values from `validateaddress` which
were hardcoded to false. Also returns `address`, `scriptPubKey`,
`isscript`, `iswitness`, `witness_version` and `witness_program`.
(a28ffa272a3c4d90d0273d9aa223a23becc08e0e)

The following methods have been deprecated:

- `listtransactions` - Use `listhistory` and the related methods and the
`after` argument for results that do not shift when new blocks arrive.
- `importprunedfunds` - Would cause a shift in transaction counts, and better
handled by adding and removing blocks at a time via resetting and
rescanning pruned and spv nodes.
- `removeprunedfunds` - For similar reasons as `importprunedfunds`.

### Node API changes

#### HTTP
Expand Down