ENH: Exponential backoff retry strategy

[Gui-FernandesBR]

Is your feature request related to a problem? Please describe.

Currently the Environment class practices a bad retry by iterating inside a while loop to catch API response within a certain number of maximum attempts. This can be problematic because the user can get blocked by the server or the API may not have enough time to process the new request after . A good standard is to apply exponential backoff, meaning that the execution will give longer pauses as the number of attempts increases.

• Dedicated python libraries may be considered. I'm leaving here two alternatives: https://pypi.org/project/backoff/ and https://stackoverflow.com/a/35636367/19971645
• I'm not against being a bit more verbose with those API recalls. I'd like to know what is happening when trying to retrieve data from te different APIs
• Implementing such features would let the code safer and more reliable.
• I found 6 occurrences of such problem within the Environment class.

Additional context

• To read more about exponential backoff
• Example of problematic code lines:
  

[Gui-FernandesBR]

I see two options for us:

1. Start using a dedicated library to apply backoff retry in our requests
   • This is good because it let the code more compact, which is usually better for reading the code and also maintenance.
   • A useful library that could be used is: https://github.com/litl/backoff, however, it seems like the library hasn't been updated lately, so that's risky. Still we can see more than 27k applications using it, that's impressive.
2. Do the backoff by using for loops, in a pure python implementation
   • This might require more code line from us.
   • This doesn't generate a new dependency so it is easier to debug in case of things go wrong, but yet the code is larger.

Further discussions are welcome here.

EDIT: actually, I just thought about something here... We could also create our own @backoff decorator.

[lucasfourier]

I was messing around with some tests in the environment class and i think the best option might be to just use some custom Python code. For instance, consider this integration test:

Maybe one could substitute 'time.sleep(1)' by 'time.sleep(i)' or 'time.sleep(2*i)', which would result in a linear backoff retry strategy and could possibly work.

Is that too naive? It was however the first thing that came to my mind.

[lucasfourier]

@Gui-FernandesBR What do you think?