Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: begin linting imports #27011

Closed
wants to merge 2 commits into from
Closed

Conversation

kdmccormick
Copy link
Member

@kdmccormick kdmccormick commented Mar 15, 2021

Blockers

  1. https://github.com/edx/edx-platform/pull/27875 : ensure every python source dir has an __init__.py
  2. https://github.com/edx/edx-platform/pull/27874 : remove dependency of xmodule on cms
  3. refactor: import common/lib/ modules from canonical locations #30533 : normalize capa and xmodule imports
  4. refactor: move common/lib/capa/capa to xmodule/capa #30791
  5. Need to write an ADR to accompany this

Description

This PR installs and configures import-linter. We start with a fairly light set of constraints; for example, we declare that lms_should_not_depend_on_cms, with a few existing exceptions. Over time, we could make these constraints stricter. The linter is run as a GitHub Action on PR builds and the master branch.

Rationale

In order to empower refactoring, restructuring, and extraction of edx-platform code, we would like to be able to analyze and make assertions about what code depends on what. One way to do this is by computing the digraph of Python import statements in our codebase, and failing PR builds if unwanted imports exist in the graph. This is exactly what the tool import-linter will allow us to do.

Example

LMS-specific code belongs in ./lms, and CMS-specific code belongs in ./cms. Since we want to decouple LMS and CMS, we don’t want code in one of these folders to import code in the other.

Using import-linter , we can fail any PR that adds a cms import to a module within lms. We can add exceptions for existing imports.

Supporting information

See BOM-2576 for context and details.

Testing instructions

Run make lint-imports locally within lms-shell and studio-shell. Make sure the check passes.

Add an illegal import (for example, from cms.djangoapps.contentstore import views within any lms module). Make sure the check fails, and make sure the output is comprehensible.

Deadline

None.

Other information

  • Upon merging, we'll want to communicate this new check in the forums and Slack, so that developers now how to resolve any
    import-linter failures that they see on their PRs. We'll want to educate folks on the importance of these checks so that they don't get frustrated and/or just add their forbidden imports to the "ignore" list.
  • I had a more ambitious version of this over here. I closed that and decided to stick with rules that don't have so many exceptions.

@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch from bf4ccc9 to ec3d3b8 Compare March 15, 2021 21:07
@kdmccormick kdmccormick reopened this Mar 15, 2021
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch 2 times, most recently from d4341cb to f470f96 Compare March 16, 2021 00:11
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch 6 times, most recently from 2d0cb2f to 847e816 Compare June 7, 2021 19:52
@kdmccormick kdmccormick changed the title draft: build: begin linting imports build: begin linting imports Jun 7, 2021
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch 2 times, most recently from f9068d7 to a366784 Compare June 7, 2021 20:21
@kdmccormick kdmccormick changed the base branch from master to kdmccormick/import-as-xmodule June 7, 2021 20:22
@kdmccormick kdmccormick force-pushed the kdmccormick/import-as-xmodule branch from 119216c to d42b0c4 Compare June 7, 2021 20:36
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch 2 times, most recently from 2b74022 to 1e7179a Compare June 7, 2021 21:54
@kdmccormick kdmccormick changed the base branch from kdmccormick/import-as-xmodule to kdmccormick/xmodule-no-cms June 7, 2021 23:23
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch from 1e7179a to 10e0539 Compare June 7, 2021 23:29
@kdmccormick kdmccormick changed the base branch from kdmccormick/xmodule-no-cms to kdmccormick/init-dot-py June 7, 2021 23:29
@kdmccormick kdmccormick force-pushed the kdmccormick/init-dot-py branch 3 times, most recently from c893bf8 to cfc4415 Compare June 8, 2021 13:46
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch from 10e0539 to b70a45f Compare June 8, 2021 13:54
@kdmccormick kdmccormick changed the base branch from kdmccormick/init-dot-py to master June 8, 2021 13:54
@kdmccormick kdmccormick closed this Jun 8, 2021
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch 2 times, most recently from 699ed5e to a3b6b9a Compare June 2, 2022 18:18
@kdmccormick kdmccormick changed the base branch from master to kdmccormick/common-lib-import-fixes June 2, 2022 18:21
@kdmccormick kdmccormick force-pushed the kdmccormick/common-lib-import-fixes branch from 5d34f5e to 1a3bcc6 Compare June 2, 2022 18:32
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch 8 times, most recently from 5d23d41 to 01f8e22 Compare June 2, 2022 19:36
@kdmccormick kdmccormick marked this pull request as ready for review June 2, 2022 19:59
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch from 01f8e22 to eaeebcf Compare June 2, 2022 20:02
@kdmccormick kdmccormick self-assigned this Jun 2, 2022
@kdmccormick
Copy link
Member Author

Ugh, the pylint build just started failing with something wacky.

Once #30533 is through, I'll rebase, which'll hopefully resolve pylint.

Base automatically changed from kdmccormick/common-lib-import-fixes to master June 6, 2022 13:54
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch from eaeebcf to b7efeb8 Compare June 6, 2022 17:28
@kdmccormick kdmccormick force-pushed the kdmccormick/lint-imports branch from 05a65b0 to e0fcac5 Compare June 6, 2022 17:30
@kdmccormick
Copy link
Member Author

Huh, pylint's failing with something weird. I'll look into that soon.

@kdmccormick kdmccormick marked this pull request as draft July 29, 2022 16:38
@kdmccormick
Copy link
Member Author

Closed in favor of #31062

@kdmccormick kdmccormick deleted the kdmccormick/lint-imports branch December 20, 2022 18:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants