Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix sendpay aggregation and ordering in listpays #4567

Merged
merged 25 commits into from
Oct 13, 2021
Merged
Changes from 1 commit
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
db5181f
pytest: Reproduce #4482
cdecker May 28, 2021
9dc4726
db: Add groupid to the payments table
cdecker Sep 28, 2021
af371b6
jsonrpc: Add groupid to `sendpay` and `sendonion`
cdecker Sep 28, 2021
9f887f3
pyln: Add groupid to `sendpay` and `sendonion`
cdecker Sep 28, 2021
8163ae3
pay: Add `groupid` to the payment struct
cdecker Sep 29, 2021
e73564d
wallet: Add function to retrieve the latest groupid for a payment
cdecker Sep 30, 2021
779826a
db: Add `groupid` to HTLCs
cdecker Sep 29, 2021
2529daa
jsonrpc: Add missing `partid` in `listsendpays` schema
cdecker Sep 29, 2021
b83fd20
pay: Call `listsendpays` to find prior attempts and abort if needed
cdecker Sep 29, 2021
fc62b7f
doc: Add missing `amount_sent_msat` to `listpays`
cdecker Sep 29, 2021
38ec80d
jsonrpc: Add `groupid` to `waitsendpay`
cdecker Sep 29, 2021
95641e3
pay: Make `pay` idempotent
cdecker Sep 30, 2021
dd78d61
pay: Do not delete old sendpay attempts if we retry
cdecker May 31, 2021
6e1341e
paycore: Prevent multiple concurrent payment groups
cdecker Oct 4, 2021
99d5808
paycore: Default `groupid` to increment from last one
cdecker Oct 4, 2021
d555a99
pytest: Fix up `test_partial_payment` to use a single `groupid`
cdecker Oct 4, 2021
75a4f5b
pay: Fail a `sendpay` or `sendonion` that'd produce a DB collision
cdecker Oct 4, 2021
fc94852
pytest: Add `groupid` to `test_partial_payment_{timeout,restart}`
cdecker Oct 4, 2021
fea3d08
pytest: Fix `test_onchain_timeout` to use `groupid`
cdecker Oct 4, 2021
f726fc6
libplugin: Add callbacks for successful and failed payments
cdecker Oct 4, 2021
11d18c1
pytest: Adjust `test_sendpay` to the new semantics
cdecker Oct 5, 2021
b8280be
pay: Mark completed payments as such by nullifying `cmd`
cdecker Oct 5, 2021
0da9623
pay: Stash and forward results for duplicate `pay` calls
cdecker Oct 5, 2021
03847bd
pay: listpays groups by payment_hash and groupid
cdecker May 28, 2021
71752df
db: Fix a syntax error with the optional parameters
cdecker Oct 7, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
79 changes: 77 additions & 2 deletions plugins/pay.c
Original file line number Diff line number Diff line change
Expand Up @@ -1936,6 +1936,76 @@ static const char *init(struct plugin *p,
return NULL;
}

/* We are interested in any prior attempts to pay this payment_hash /
* invoice so we can set the `groupid` correctly and ensure we don't
* already have a pending payment running. */
static struct command_result *
payment_listsendpays_previous(struct command *cmd, const char *buf,
const jsmntok_t *result, struct payment *p)
{
size_t i;
const jsmntok_t *t, *arr, *err;
/* What was the groupid of an eventual previous attempt? */
u64 last_group = 0;
/* Do we have pending sendpays for the previous attempt? */
bool pending = false;
/* Did a prior attempt succeed? */
bool completed = false;

err = json_get_member(buf, result, "error");
if (err)
return command_fail(
cmd, LIGHTNINGD,
"Error retrieving previous pay attempts: %s",
json_strdup(tmpctx, buf, err));

arr = json_get_member(buf, result, "payments");
if (!arr || arr->type != JSMN_ARRAY)
return command_fail(
cmd, LIGHTNINGD,
"Unexpected non-array result from listsendpays");

/* We iterate through all prior sendpays, looking for the
* latest group and remembering what its state is. */
json_for_each_arr(i, t, arr)
{
u64 groupid;
const jsmntok_t *status, *grouptok;
grouptok = json_get_member(buf, t, "groupid");
json_to_u64(buf, grouptok, &groupid);

/* New group, reset what we collected. */
if (last_group != groupid) {
completed = false;
pending = false;
last_group = groupid;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes that listsendpays is listed in increasing groupid order? That assumption does not seem documented?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a listsendpays call limited to the payment_hash of the invoice. There cannot be multiple groups in flight concurrently. So ordering by id which we do internally in lightningd/pay.c will result in the desired ordering of groupid. I'll add a comment to that effect 👍


status = json_get_member(buf, t, "status");
completed |= json_tok_streq(buf, status, "complete");
pending |= json_tok_streq(buf, status, "pending");
}

if (completed) {
return command_fail(
cmd, PAY_RHASH_ALREADY_USED,
"The payment with payment_hash=%s was successful "
"(groupid=%" PRIu64 ")",
type_to_string(tmpctx, struct sha256, p->payment_hash),
last_group);
} else if (pending) {
return command_fail(
cmd, PAY_IN_PROGRESS,
"Already have payment with payment_hash=%s in progress "
"(groupid=%" PRIu64 ")",
type_to_string(tmpctx, struct sha256, p->payment_hash),
last_group);
}
p->groupid = last_group + 1;
payment_start(p);
return command_still_pending(cmd);
}

struct payment_modifier *paymod_mods[] = {
&local_channel_hints_pay_mod,
&exemptfee_pay_mod,
Expand Down Expand Up @@ -1986,6 +2056,7 @@ static struct command_result *json_paymod(struct command *cmd,
u64 invexpiry;
struct sha256 *local_offer_id;
const struct tlv_invoice *b12;
struct out_req *req;
#if DEVELOPER
bool *use_shadow;
#endif
Expand Down Expand Up @@ -2167,12 +2238,16 @@ static struct command_result *json_paymod(struct command *cmd,
shadow_route->use_shadow = *use_shadow;
#endif
p->label = tal_steal(p, label);
payment_start(p);
list_add_tail(&payments, &p->list);
/* We're keeping this around now */
tal_steal(cmd->plugin, p);

return command_still_pending(cmd);
req = jsonrpc_request_start(cmd->plugin, cmd, "listsendpays",
payment_listsendpays_previous,
payment_listsendpays_previous, p);

json_add_sha256(req->js, "payment_hash", p->payment_hash);
return send_outreq(cmd->plugin, req);
}

static const struct plugin_command commands[] = {
Expand Down