Python Requirements Update (with fix for pylint 2.15 compatibility) #208

bradenmacdonald · 2022-10-12T18:52:34Z

Description

This has all the version bumps from #206 and some additional minor ones, along with various little fixes to make this work with pylint 2.15.x.

The main error seen with pylint 2.15 (also copied below) was due to "" missing from sys.path. Why this happened is unknown, but I have put in a hack in pylintrc_tweaks which has worked around the issue for now.

Run make quality
/home/runner/work/blockstore/blockstore/venv/bin/pycodestyle --config=pycodestyle blockstore *.py
/home/runner/work/blockstore/blockstore/venv/bin/pylint --django-settings-module=blockstore.settings.test --rcfile=pylintrc blockstore *.py
Traceback (most recent call last):
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.[8](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:9)/site-packages/pylint_django/checkers/foreign_key_strings.py", line [9](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:10)2, in open
    django.setup()
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/django/__init__.py", line 19, in setup
    configure_logging(settings.LOGGING_CONFIG, settings.LOGGING)
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 82, in __getattr__
    self._setup(name)
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 63, in _setup
    raise ImproperlyConfigured(
django.core.exceptions.ImproperlyConfigured: Requested setting LOGGING_CONFIG, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure() before accessing settings.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint_django/checkers/foreign_key_strings.py", line 120, in open
    settings.configure(Settings(self.config.django_settings_module))
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/django/conf/__init__.py", line 170, in __init__
    mod = importlib.import_module(self.SETTINGS_MODULE)
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line [10](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:11)14, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 961, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'blockstore'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/runner/work/blockstore/blockstore/venv/bin/pylint", line 8, in <module>
    sys.exit(run_pylint())
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/__init__.py", line 35, in run_pylint
    PylintRun(argv or sys.argv[1:])
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/lint/run.py", line 207, in __init__
    linter.check(args)
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 685, in check
    with self._astroid_module_checker() as check_astroid_module:
  File "/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/contextlib.py", line [11](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:12)3, in __enter__
    return next(self.gen)
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 982, in _astroid_module_checker
    checker.open()
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint_django/checkers/foreign_key_strings.py", line [12](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:13)5, in open
    self.add_message(
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/checkers/base_checker.py", line 164, in add_message
    self.linter.add_message(
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line [13](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:14)04, in add_message
    self._add_one_message(
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/lint/pylinter.py", line 1237, in _add_one_message
    self.stats.increase_single_module_message_count(
  File "/home/runner/work/blockstore/blockstore/venv/lib/python3.8/site-packages/pylint/utils/linterstats.py", line 309, in increase_single_module_message_count
    self.by_module[modname][type_name] += increase
KeyError: 'Command line or configuration file'
make: *** [Makefile:1[14](https://github.com/openedx/blockstore/actions/runs/3216133753/jobs/5257725616#step:6:15): quality] Error 1
Error: Process completed with exit code 2.

openedx-webhooks · 2022-10-12T18:52:37Z

Thanks for the pull request, @bradenmacdonald!

As a core committer in this repo, you can merge this once the pull request is approved per the core committer reviewer requirements and according to the agreement with your edX Champion.

bradenmacdonald · 2022-10-12T19:01:01Z

@Agrendalath Could you please give this a quick look/review before I merge?

bradenmacdonald · 2022-10-12T20:52:25Z

Makefile

@@ -111,7 +111,7 @@ html_coverage: ## Generate HTML coverage report

 quality: ## Run quality checks
 	${VENV_BIN}/pycodestyle --config=pycodestyle blockstore *.py
-	${VENV_BIN}/pylint --django-settings-module=blockstore.settings.test --rcfile=pylintrc blockstore *.py
+	${VENV_BIN}/pylint --django-settings-module=blockstore.settings.test blockstore *.py


It's not necessary to specify pylintrc as it's the default config file used.

Agrendalath

👍

I tested this: checked that the quality tests are passing locally
I read through the code
I checked for accessibility issues: n/a
Includes documentation
I made sure any change in configuration variables is reflected in the corresponding client's configuration-secure repository: n/a

blockstore/apps/bundles/store.py

Agrendalath · 2022-10-17T12:49:24Z

requirements/docs.txt

@@ -1,5 +1,5 @@
 #
-# This file is autogenerated by pip-compile with python 3.8
+# This file is autogenerated by pip-compile with python 3.10


Nit: why is the header updated to Python 3.10 in local.txt, docs.txt, and test.txt, but not, e.g. for production.txt? I thought the make upgrade command would upgrade it for all files.
Using Python 3.10 with make upgrade could potentially update packages to versions that are no longer compatible with Python 3.8, which is fine for the CI, but not so fine for production requirements. We may want to be consistent here and use only Python 3.8 unless we test both versions in the CI.

Whoops, my bad. I tried not to commit the "3.10" but missed a couple. And you're right, better to run it with 3.8 in the first place, so that's what I've done now. I don't think it changed anything besides a comment about which packages depend on typing-extensions

openedx-webhooks · 2022-10-17T17:23:11Z

@jristau1984: thought you might like to know that bradenmacdonald merged this pull request.

openedx-webhooks · 2022-10-17T17:23:12Z

@bradenmacdonald 🎉 Your pull request was merged! Please take a moment to answer a two question survey so we can improve your experience in the future.

bradenmacdonald · 2022-10-17T17:25:53Z

Thanks for the great reviews @Agrendalath .

edx-requirements-bot and others added 2 commits October 9, 2022 20:30

chore: Updating Python Requirements

1224dc5

chore: pin pylint to < 2.15.0 because the new release is not working

7994665

openedx-webhooks added core committer open-source-contribution PR author is not from Axim or 2U labels Oct 12, 2022

bradenmacdonald mentioned this pull request Oct 12, 2022

Python Requirements Update #206

Closed

bradenmacdonald force-pushed the requirements-manual-bump branch from 701ffe2 to e5a3cd4 Compare October 12, 2022 20:37

bradenmacdonald changed the title ~~Python Requirements Update (with pylint version temporarily pinned)~~ Python Requirements Update (with fix for pylint 2.15 compatibility) Oct 12, 2022

bradenmacdonald force-pushed the requirements-manual-bump branch from e5a3cd4 to c092840 Compare October 12, 2022 20:46

chore: un-pin pylint (required hack in pylintrc_tweaks)

c2a782b

bradenmacdonald force-pushed the requirements-manual-bump branch from c092840 to c2a782b Compare October 12, 2022 20:49

bradenmacdonald commented Oct 12, 2022

View reviewed changes

bradenmacdonald mentioned this pull request Oct 12, 2022

fix: bump pylint version openedx/edx-platform#31084

Merged

Agrendalath approved these changes Oct 17, 2022

View reviewed changes

chore: remove pyblake2, use correct python version

188656a

bradenmacdonald force-pushed the requirements-manual-bump branch from 1fe9b48 to 188656a Compare October 17, 2022 17:20

bradenmacdonald merged commit 35d7d17 into master Oct 17, 2022

bradenmacdonald deleted the requirements-manual-bump branch October 17, 2022 17:23

bradenmacdonald mentioned this pull request Oct 17, 2022

Python Requirements Update #209

Closed

UsamaSadiq mentioned this pull request Oct 27, 2022

Fix pylint-django-settings plugin openedx/edx-platform#31198

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python Requirements Update (with fix for pylint 2.15 compatibility) #208

Python Requirements Update (with fix for pylint 2.15 compatibility) #208

bradenmacdonald commented Oct 12, 2022 •

edited

Loading

openedx-webhooks commented Oct 12, 2022

bradenmacdonald commented Oct 12, 2022

bradenmacdonald Oct 12, 2022

Agrendalath left a comment

Agrendalath Oct 17, 2022

bradenmacdonald Oct 17, 2022

openedx-webhooks commented Oct 17, 2022

openedx-webhooks commented Oct 17, 2022

bradenmacdonald commented Oct 17, 2022

Python Requirements Update (with fix for pylint 2.15 compatibility) #208

Python Requirements Update (with fix for pylint 2.15 compatibility) #208

Conversation

bradenmacdonald commented Oct 12, 2022 • edited Loading

Description

openedx-webhooks commented Oct 12, 2022

bradenmacdonald commented Oct 12, 2022

bradenmacdonald Oct 12, 2022

Choose a reason for hiding this comment

Agrendalath left a comment

Choose a reason for hiding this comment

Agrendalath Oct 17, 2022

Choose a reason for hiding this comment

bradenmacdonald Oct 17, 2022

Choose a reason for hiding this comment

openedx-webhooks commented Oct 17, 2022

openedx-webhooks commented Oct 17, 2022

bradenmacdonald commented Oct 17, 2022

bradenmacdonald commented Oct 12, 2022 •

edited

Loading