Retry until

[lvijnck]

Hi,

I'm wondering whether there exists functionality to keep retrying the test under certain condition. This is especially relevant for APIs with long running 'asynchronous' tasks, i.e., consider the following endpoints:

1. /create endpoint that accepts the task and immediately yields a 200 status
2. /status endpoint that returns the status of the job, e.g., IN_PROGRESS, SUCCESS and FAILED

I'd like to express a test that keeps re-trying the /status endpoint until the status is either SUCCESS or FAILED. This could indeed be achieved with the retry functionality, but that would result in a rather flaky test as the duration of the task varies heavily.

P.S. Happy to contribute after discussion.

[michaelboulton]

I'm not sure I get the utility of this suggestion - how would a 'retry until' be practically any different than just using the existing 'retry' key? They both involve running the same query until a certain condition is fulfilled, and with the existing retry key you can just add a delay before the stage which will let you set a time bound on how long to try (ie, setting a 1 second delay with 10 retries limits it to 10 seconds).

[lvijnck]

Hi @michaelboulton the difference here is that my long running task can possibly take an hour to complete, while failure along the way is very likely. So it would be relevant to specify additional stop (and failure) conditions.

[michaelboulton]

I suppose this is related to #621 then , some way of specifying that you don't want something to happen?

[lvijnck]

Yeah, exactly!

[puneeter]

Hi @michaelboulton ! I have a similar problem. The app's status can only be (Succeeded, Failed, Running) one of these. I was using this stage in my test but it seems like I could do an early quit in case of status=="Failed". In the current implementation, I can only see it working for MQTT. Any chance we can get this functionality in the standard tests?

- name: <name>
    delay_before: 10
    max_retries: 60
    request:
      URL: <URL>
      method: GET
      headers:
        content-type: application/json
    response:
      status_code: 200
      json:
        status: Succeeded
      # unexpected: true -> Can we use this here? If yes then how do we define the unexpected value?
      headers:
        content-type: application/json

[michaelboulton]

If anyone is following this I made something to try and address this issue without adding more random keys into the request/response: #1011 Especially see the docs here: https://github.com/taverntesting/tavern/blob/b8b87f0921beac7cf55cc585b73abd36c84e7741/docs/source/scripting.md

This is currently on pypi as version 3.4.0a1