Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swap not monitored after connext failed sendpayment and xud restarted #1799

Closed
sangaman opened this issue Aug 6, 2020 · 1 comment · Fixed by #1822
Closed

Swap not monitored after connext failed sendpayment and xud restarted #1799

sangaman opened this issue Aug 6, 2020 · 1 comment · Fixed by #1822
Assignees
Labels

Comments

@sangaman
Copy link
Collaborator

sangaman commented Aug 6, 2020

#1761 (comment)

There is still no monitoring continuation for maker side after maker's xud restart.

Steps:

1. place 9 sell ETH/BTC orders as a maker, fill them all by 1 order of a taker -> most of swaps failed (**29/07/2020 15:33:58.510 in maker's log**)

2. monitoring messages in maker logs each 5 mins (**29/07/2020 15:47:33.970 in maker's log**)

3. restart maker's xud (**29/07/2020 16:01:13.444 in maker's log**)

4. wait -> No monitor messages

5. shutdown both envs -> swap err messages in the taker's xud log after disconnecting from all peers. (**29/07/2020 16:17:41.217 in taker's log**)

Screenshot from 2020-07-29 19-36-15
Screenshot from 2020-07-29 19-36-27

monitor_xud_taker.log
monitor_xud_maker.log

maker's xud db - gofile.io/d/7GKTmK
taker's xud db - gofile.io/d/uqMD6s

This is very helpful but after some investigation I'm pretty sure this is a separate issue from this PR and the issue it indends to fix. It's due to the fact that a connext sendpayment call can "fail" while the payment is still in limbo. It's actually somewhat related to #1794, the solution is to not fail the deal when a sendpayment calls, and to have a mechanism within swaps to monitor swaps that are still in an active status even after they've timed out and even after our attempts to send payment have failed, up until we get confirmation that all HTLCs are canceled/expired. Everything in SwapRecovery will then only be used for when xud crashes or is restarted while there are active swaps.

@kilrau
Copy link
Contributor

kilrau commented Aug 17, 2020

This one is definitely still open, opened another issue: #1818

sangaman added a commit that referenced this issue Aug 18, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1816.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1816.
sangaman added a commit that referenced this issue Aug 19, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1816.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1816.
sangaman added a commit that referenced this issue Aug 19, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1816.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1816.
sangaman added a commit that referenced this issue Aug 19, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1816.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1816.
sangaman added a commit that referenced this issue Aug 19, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1816.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1816.
sangaman added a commit that referenced this issue Aug 21, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1794.
sangaman added a commit that referenced this issue Aug 25, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1794.
sangaman added a commit that referenced this issue Aug 25, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1794.
sangaman added a commit that referenced this issue Aug 25, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

Fixes #1799. Fixes #1794.
sangaman added a commit that referenced this issue Aug 26, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.
sangaman added a commit that referenced this issue Aug 26, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.
sangaman added a commit that referenced this issue Aug 26, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.
sangaman added a commit that referenced this issue Aug 26, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.
sangaman added a commit that referenced this issue Aug 27, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.
sangaman added a commit that referenced this issue Aug 27, 2020
This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.
raladev pushed a commit that referenced this issue Aug 28, 2020
* feat(swaps): monitor pending payments before fail

This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.

* test(sim): handle no force close for taker

This modifies the simulation tests to account for the maker properly
canceling the invoice for its incoming payment once its outgoing payment
htlc expires. Rather than expecting the channel with the taker to maker
payment to expire and force close, it expects the invoice to be canceled
and for the channel to remain open. It ensures that the balances on the
channel do not change.
rsercano pushed a commit that referenced this issue Sep 11, 2020
* feat(swaps): monitor pending payments before fail

This monitors all swap client payments until their resolution without
putting deals into `SwapRecovery`. Previously, if a call to send payment
failed but the payment was still in pending status (as has been the case
with Connext), then we would fail the swap deal and monitor the payment
in `SwapRecovery`. This had several downsides, namely:

1. Since the deal is marked as having failed in the database, if xud
restarts while payment monitoring is ongoing, it won't resume monitoring
because it sees the swap as having failed in the database. We only
recover swaps that were "active" at the time xud shut down. See #1799.

2. When a deal is failed, the maker order it attempted to swap re-enters
the order book and is available to be matched again. However, since
the payment for the original deal is still pending, it may still go
through, meaning that the order can be "double filled" in such a case.
See #1794.

By monitoring all pending payments to their resolution, we ensure that
we don't fail deals that wind up completing.

This also fixes a case where the maker would not cancel its invoice when
an outgoing payment would fail and the very first `lookupPayment` call
would confirm that it had failed.

Fixes #1799. Fixes #1794. Fixes #1708.

* test(sim): handle no force close for taker

This modifies the simulation tests to account for the maker properly
canceling the invoice for its incoming payment once its outgoing payment
htlc expires. Rather than expecting the channel with the taker to maker
payment to expire and force close, it expects the invoice to be canceled
and for the channel to remain open. It ensures that the balances on the
channel do not change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants