Skip to content

Conversation

@thecodeboss
Copy link
Contributor

This addresses one of the comments in #20, specifically wrapping the Port.command operation in a try-rescue so that argument errors are not raised when the port gets closed.

This does not fix the fact that the port does not get re-opened, but at the very least things won't crash due to metrics errors. Re-opening the port on failure can be done in a separate PR.

@thecodeboss
Copy link
Contributor Author

@lexmag could you take a look?

@drozzy
Copy link

drozzy commented Aug 7, 2018

Forgive if this comment is irrelevant...

But I think the proper way to handle disconnection of some sort is to re-start the process and try to connect again. If connection fails, wait a few seconds (possibly with back-off) and try again. Keep doing it until you succeed. After that transition to normal mode of operation.

If you try to handle all possible errors it will never work "nicely".

Again, ignore my comment if i misunderstood something.

@edgurgel
Copy link
Contributor

edgurgel commented Aug 7, 2018

@drozzy , I think the missing argument here is that in general you don't care if you missed some metrics and you don't want to break your application because the port was closed.

So basically if some metrics were not sent, keep going. Much like a GenServer cast I guess?

@drozzy
Copy link

drozzy commented Aug 7, 2018

But will it work the next time the metric tries to get sent? Or will the process responsible for sending them already be "dead"?

@edgurgel
Copy link
Contributor

edgurgel commented Aug 7, 2018

From what I understand the port will be reestablished given that we reconnect like described here: #20 (comment)

I implemented something similar already and it works perfectly 👍

@electricshaman
Copy link

FWIW, I think an ideal solution to this issue in the general case would avoid making too many assumptions about how the end-application intends to handle failure. I agree that an application probably shouldn't crash or stop handling requests just because monitoring goes down, but I think what @drozzy is getting at is that you can already handle this kind of scenario very well in OTP by structuring your supervision tree such that the port/connection is separate from the client process that uses it. The client process could then still handle requests (though not fulfill them) like is described in this article. https://ferd.ca/it-s-about-the-guarantees.html

I realized I'm like 2 months late to the party, so feel free to ignore this. :)

@lexmag lexmag merged commit fb72bfb into lexmag:master Jul 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants