Handling Rate Limiting With v2 Commands

[CyTheGuy]

Description

What is the best way to handle web API rate limiting with the new v2 client? The docs https://slack.dev/python-slackclient/basic_usage.html [slack.dev] don't seem to work anymore and the release notes and migration notes didn't suggest anything. I now get this error when hitting rate limiting but there appears no way to handle it like before since the response is never returned to me it is just output as an error.

slack.errors.SlackApiError: The request to the Slack API failed.
The server responded with: {'ok': False, 'error': 'ratelimited'}

What type of issue is this? (place an x in one of the [ ])

• question
• documentation related

Requirements (place an x in each of the [ ])

• [X ] I've read and understood the Contributing guidelines and have done my best effort to follow them.
• [X ] I've read and agree to the Code of Conduct.
• [X ] I've searched for any related issues and avoided creating a duplicate issue.

---

Reproducible in:

slackclient version: v2

python version: Python 3.7.3

OS version(s): Mac
Logs, screenshots, screencast, sample project, funny gif, etc.

[RodneyU215]

Hi @CyTheGuy, thanks for opening this Github issue! Providing support for better handling of rate limits is something that we're definitely interested in exploring. This is a feature we want to add, but it's important that we do so in a way that doesn't make it easy to abuse the API. I'm planning on spending more time on this over the next couple of weeks.

If you'd like to fix it locally in the time being here's some pseudo code that should get you close:

max_attempts = 3
for i in range(max_attempts):
    while True:
        try:
            client.chat_postMessage() # Whatever API call you're making.
        except client_err.SlackApiError as exception

            wait_time = float(exception.response["headers"]["Retry-After"])
            time.sleep(wait_time) # or await asyncio.sleep(wait_time) for async methods.
            continue
        break

This has also helped me identify an existing bug where the amount of time you should wait is not provided in the exception. I'll likely work on this very soon.

[CyTheGuy]

Woohoo! Thank you for getting back so quickly. I will give this a try thank you so much. Didn't realize I could handle the error myself like that. Previously I would just see if the error returned was "ratelimited" and sleep if it wasn't "ok" but this looks like a solid path forward for now. Thanks again

[DarrenN]

@RodneyU215 Howdy, just wanted to point out a small wrinkle in this strategy of throwing Exceptions for rate limiting. The new Response object is a generator / iterable, which is super useful for paging through conversations.list. When I use a for loop like:

        for response in sc.api_call("conversations.list", params=params):
            ...

I can't catch the SlackApiError with a try and use continue. I have to wrap the call in another generator, manually calling next and catching exceptions.

UPDATE I can't even do that successfully because in __iter__ you set self._iteration which will only be set if I use a for in loop, it won't be set if I call yield on the Response object. Example:

def gen_wrapper(response):
    while True:
        try:
            # Returns Response object
            yield next(response)
        except StopIteration:
            raise
        except SlackApiError as exn:
           # rate limiting code here

Will error object has no attribute '_iteration' because __iter__() was never called.

[lukwam]

I am also unable to handle any rate limit issues with the newest slackclient for python.

try:
    response = client.users_profile_get(user=uid)
except slack.errors.SlackApiError as e:
    if e.response['ok'] is False and e.response['headers']['Retry-After']:
        delay = int(e.response['headers']['Retry-After'])
        print('Rate limited. Retrying in ' + str(delay) + ' seconds')
        time.sleep(delay)
        response = client.users_profile_get(user=uid)

However, the SlackApiError response dict has no headers key.

Traceback (most recent call last):
  File "./test.py", line 71, in <module>
    main()
  File "./test.py", line 66, in main
    profiles = s.get_users_profiles()
  File "/usr/src/bits/slack/__init__.py", line 201, in get_users_profiles
    if e.response['ok'] is False and e.response['headers']['Retry-After']:
KeyError: 'headers'

I've tried getting the headers from the response instead, but that doesn't work because the client.users_profile_get(user=uid) call doesn't return a value, it raises an exception.

It seems like there is no way to gracefully handle rate limit errors using the python client?

How do I get the headers from the SlackApiError object?

[ddnomad]

@RodneyU215 Howdy, just wanted to point out a small wrinkle in this strategy of throwing Exceptions for rate limiting. The new Response object is a generator / iterable, which is super useful for paging through conversations.list. When I use a for loop like:

        for response in sc.api_call("conversations.list", params=params):
            ...

I can't catch the SlackApiError with a try and use continue. I have to wrap the call in another generator, manually calling next and catching exceptions.

UPDATE I can't even do that successfully because in __iter__ you set self._iteration which will only be set if I use a for in loop, it won't be set if I call yield on the Response object. Example:

def gen_wrapper(response):
    while True:
        try:
            # Returns Response object
            yield next(response)
        except StopIteration:
            raise
        except SlackApiError as exn:
           # rate limiting code here

Will error object has no attribute '_iteration' because __iter__() was never called.

So I've just encountered that thing and now wondering how to get around. conversations_history() method will raise at a random point in time and there is no obvious way of how to resume.

[lukwam]

I really don't see any way to handle rate limiting in this new version. Can someone show a working example of a loop where you hit the rate limit and then you handle it? My goal is to do the following:

• get all users in my workspace
• for each user get user profile

I always hit the rate limit half way through the user list but I haven't figured out how to handle it proper and proceed. How do you retrieve the response headers after you've received the exception?