Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix async helpers import cycle #311

Merged
merged 5 commits into from
Mar 8, 2023
Merged

Conversation

aiudirog
Copy link
Contributor

@aiudirog aiudirog commented Mar 3, 2023

Description

Fixes the import cycle that prevents async helpers from being imported since v2.2.0

Issues Resolved

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

aiudirog added 2 commits March 3, 2023 15:39
Signed-off-by: Roger Aiudi <aiudirog@gmail.com>
…using an import cycle (fixes opensearch-project#310)

Signed-off-by: Roger Aiudi <aiudirog@gmail.com>
@aiudirog
Copy link
Contributor Author

aiudirog commented Mar 3, 2023

Also, when updating the changelog, I noticed that there isn't an entry for v2.2.0

Signed-off-by: Roger Aiudi <aiudirog@gmail.com>
CHANGELOG.md Outdated
@@ -1,6 +1,10 @@
# CHANGELOG
Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)

## [2.2.1]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe 2.1.1 is mislabeled and should have been 2.2.0? @harshavamsi

This should say "unreleased"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed


I know you didn't ask me, but looking at the release notes on GitHub, it looks like the release notes for both versions go merged. I think everything but the notes for the SigV4 fixes should be moved to a new block for 2.2.0. I can do that if you would like.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please! We made a mess.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Latest commit should have resolved this. Let me know if you want anything else done or changed.

Signed-off-by: Roger Aiudi <aiudirog@gmail.com>
@dblock
Copy link
Member

dblock commented Mar 7, 2023

Thanks! I think we need a test for this. Otherwise we'll reintroduce the same bug.

@aiudirog
Copy link
Contributor Author

aiudirog commented Mar 7, 2023

Thanks! I think we need a test for this. Otherwise we'll reintroduce the same bug.

Already done. The first commit adjusts the imports to use the public import (the one that was broken) rather than the private one. That caused the test cases to fail during import. The second commit actually fixes the problem.

Signed-off-by: Roger Aiudi <aiudirog@gmail.com>
@dblock dblock merged commit 49ae7c7 into opensearch-project:main Mar 8, 2023
@dblock
Copy link
Member

dblock commented Mar 8, 2023

@aiudirog I may have been a bit too fast to merge, https://github.com/opensearch-project/opensearch-py/actions/runs/4359333294/jobs/7621006576 seems to be related, care to take a look?

@aiudirog
Copy link
Contributor Author

aiudirog commented Mar 8, 2023

@dblock That's a very interesting error. This command is expected to fail but didn't:

run(
venv_python,
"-c",
f"from {dist_name}.helpers import async_scan, async_bulk, async_streaming_bulk, async_reindex",
expect_exit_code=256,
)

I think the reason is that there's no longer the import of AsyncOpenSearch into that file, so there's no reason for those helpers to fail since they don't themselves import aiohttp. We may be able to remove the except ImportError from helpers/__init__.py. I'll look into it.

@aiudirog
Copy link
Contributor Author

aiudirog commented Mar 8, 2023

@dblock Yes, that appears to be the issue. I'll open a new PR. Would you like me to adjust helpers/__init__.py accordingly? I think either of these changes would be good:

Option A:

try:
    # Asyncio only supported on Python 3.6+
    if sys.version_info < (3, 6):
        raise SyntaxError

    from .._async.helpers import (
        async_bulk,
        async_reindex,
        async_scan,
        async_streaming_bulk,
    )

    __all__ += ["async_scan", "async_bulk", "async_reindex", "async_streaming_bulk"]
except SyntaxError:
    pass

Option B:

# Asyncio only supported on Python 3.6+
if sys.version_info >= (3, 6):

    from .._async.helpers import (
        async_bulk,
        async_reindex,
        async_scan,
        async_streaming_bulk,
    )

    __all__ += ["async_scan", "async_bulk", "async_reindex", "async_streaming_bulk"]

I personally prefer Option B.

@dblock
Copy link
Member

dblock commented Mar 8, 2023

Most definitely don't rely on an exception to control flow! B is much better.

@aiudirog
Copy link
Contributor Author

aiudirog commented Mar 8, 2023

Cool, I'll push that change up on the other PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Async helpers cannot be imported due to import cycle
2 participants