Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove unused imports #2624

Merged
merged 4 commits into from
May 22, 2020
Merged

remove unused imports #2624

merged 4 commits into from
May 22, 2020

Conversation

jameslamb
Copy link

Thanks for contributing to Prefect!

  • adds new tests (if appropriate)
  • add a changelog entry in the changes/ directory (if appropriate)
  • updates docstrings for any new functions or function arguments, including docs/outline.toml for API reference docs (if appropriate)

What does this PR change?

Thanks for this great project! I've been getting familiar with prefect and really liking it, so I wanted to contribute back. Similar to #2567 , this PR proposes fixing another class of problem I saw running flake8 on this repo: unused imports.

The "unused imports" detection in flake8 can lead to a lot of false positives, for example in __init__.py files, so I tried to limit the changes I made here to things I'm fairly confident can be removed. You can generate the list of things I changed with this command:

flake8 \
    --ignore=E501,E231,E503,E266,W503 \
    | grep "F401" \
    | grep -v "__init" \
    | grep -v "'prefect" \
    | grep -v "'pytest'" \
    | grep -v "'asyncio" \
    | grep -v "sqlalchemy" \
    | grep -v "alembic" \
    | grep -v "'docker" \
    | grep -v "'azure\.stoorage" \
    | grep -v "'google\.cloud" \
    | grep -v "'graphviz" \
    | grep -v "\.fixtures\.database_fixtures" \
    | grep -v "'blig" \
    > flake.txt

Most of those ignored things are from these common false positive cases:

  • things in __init__.py files
  • things that need to be defined because they're mocked or patched
  • imports used to test the availability of a library

I also avoided sqlalchemy and alembic just because I don't understand the full magic of sqlalchemy and didn't know if some of the things flake8 turned about were actually unused.

Why is this PR important?

Imports add to the time it takes to load modules.

Unused imports can also make it seem like some dependencies are more widely used in the project than they really are, which can prevent maintainers from noticing limited-use dependencies and trying to factor them out.

To be clear...this PR removes imports, but it doesn't change the dependencies for prefect at all. All of the things I'm proposing removing come from libraries that are imported elsewhere in the project.

Thanks for your time and consideration!

@joshmeek
Copy link

Looks like it may have removed too much 😅

_________________ ERROR collecting docs/test_generate_docs.py __________________
docs/test_generate_docs.py:9: in <module>
    from prefect import Task, task
src/prefect/__init__.py:8: in <module>
    import prefect.triggers
src/prefect/triggers.py:48: in <module>
    from prefect.engine import signals
src/prefect/engine/__init__.py:9: in <module>
    from prefect.engine.task_runner import TaskRunner
src/prefect/engine/task_runner.py:55: in <module>
    class TaskRunner(Runner):
src/prefect/engine/task_runner.py:615: in TaskRunner
    ) -> Tuple[State, Dict[Edge, State]]:
E   NameError: name 'Tuple' is not defined

@jameslamb
Copy link
Author

haha ah! That's what I get for not running the tests one more time before pushing.

I just fixed that in f5b5b7a. Ran the tests locally and all but two passed. From the errors at first glance it seemed unrelated to these changes, and I'm hoping it's just one of those "my laptop looks different from the CI environment" things.

Let's see if I'm right 🤞

@jameslamb
Copy link
Author

Let's see if I'm right

I was not haha. Ok sorry, you can ignore this PR until I get CI working. Clearly there are some differences between the way I was running the tests locally and the way they're run in CI.

@jameslamb jameslamb changed the title remove unused imports [WIP] remove unused imports May 21, 2020
@cicdw
Copy link
Member

cicdw commented May 21, 2020

We do run an unused import check circle job so I'll be surprised if you find anything new

@jcrist
Copy link

jcrist commented May 21, 2020

We do run an unused import check circle job so I'll be surprised if you find anything new

This is only run on the src/prefect codebase, and ignores both server and the tests.

@jameslamb
Copy link
Author

We do run an unused import check circle job so I'll be surprised if you find anything new

This is only run on the src/prefect codebase, and ignores both server and the tests.

that makes sense! The only unused imports I picked up with flake8 in prefect/ were type hints, so it's possible the code you're using in that test doesn't know how to reason about type hints.

@codecov
Copy link

codecov bot commented May 22, 2020

Codecov Report

Merging #2624 into master will not change coverage.
The diff coverage is n/a.

@jameslamb jameslamb changed the title [WIP] remove unused imports remove unused imports May 22, 2020
@jameslamb
Copy link
Author

Ok this is passing CI and ready for a review! Thanks for your time.

@jcrist jcrist merged commit 0809432 into PrefectHQ:master May 22, 2020
@jcrist
Copy link

jcrist commented May 22, 2020

Thanks @jameslamb!

@jameslamb jameslamb deleted the misc/unused-imports branch September 15, 2020 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants