Bug: new pagination logic increments page index incorrectly

[pylipp]

Checklist

• I have looked into the Readme and have not found a suitable solution or answer.
• I have looked into the API documentation and have not found a suitable solution or answer.
• I have searched the issues and have not found a suitable solution or answer.
• I have searched the Auth0 Community forums and have not found a suitable solution or answer.
• I agree to the terms within the Auth0 Code of Conduct.

Description

I'm using the new (v5.1.0) SDK and I want to iterate over my 108 roles.

Investigation

The pagination logic in the RawClient incorrectly increments the page index for the lambda function get_next:

auth0-python/src/auth0/management/roles/raw_client.py

Line 95 in 2f4906d

page=page + len(_items or []),

When I change it to page=page + 1, it correctly works.

Other paginated queries, e.g. listing users, and in the AsyncClient, also suffer from this. I found more than 60 occurrences of the pattern page = page + len(items or []) in the code base.

Reproduction

client = ManagementClient(...)
resp = client.roles.list()
for i, role in enumerate(resp):
    print(f"{i}: {role.name}")

Expected output

108 lines with the role names

Actual input

50 lines with the role names (50 is the default for the per_page parameter)

Additional context

No response

auth0-python version

5.1.0

Python version

3.12

[insolite]

Having something similar here:

roles = []
pager = await client.roles.list(per_page=3)
async for role in pager:
    roles.append(role)
assert len(roles) == 8 # <-- fails: it's always 3 regardles of expected 8 total

The client logs these:

/api/v2/roles?per_page=3&page=0&include_totals=true
/api/v2/roles?per_page=3&page=3&include_totals=true

I think page=3 clearly indicates there should be either offset page=1 or offset=3.

[pylipp]

Working around with a custom client:

import httpx


class _Client(httpx.Client):
    """Custom client class to work-around a pagination bug in the Auth0 SDK.
    Keep track of correct page number, instead of relying on incorrect page increment
    implemented in the library.
    Reset page count before using the `users.list` or `roles.list` methods.
    Cf. https://github.com/auth0/auth0-python/issues/783
    """

    def __init__(self, **kwargs):
        self._page = 0
        super().__init__(**kwargs)

    def request(self, method, url, **kwargs):
        if url.split("/")[-1] in ("users", "roles") and method == "GET":
            params = dict(kwargs.get("params", [("page", 0)]))
            params["page"] = self._page
            self._page += 1
            kwargs["params"] = list(params.items())
        return super().request(method, url, **kwargs)

interface = ManagementClient(domain=domain, client_id=client_id, client_secret=secret, httpx_client=_Client())
print(interface.roles.list(per_page=10))

# Reset page counter
interface._api._client_wrapper.httpx_client.httpx_client._page = 0
print(interface.users.list(per_page=10))