Remove Python 2 support from CLI #5512

ofek · 2020-01-17T23:15:14Z

Motivation

codecov · 2020-01-17T23:55:55Z

Codecov Report

Merging #5512 into master will decrease coverage by 3.22%.
The diff coverage is 94.28%.

Impacted Files	Coverage Δ
...checks_dev/datadog_checks/dev/tooling/constants.py	`96.15% <0%> (-0.15%)`	⬇️
...v/datadog_checks/dev/tooling/configuration/core.py	`100% <100%> (ø)`	⬆️
...tadog_checks/dev/tooling/configuration/template.py	`100% <100%> (ø)`	⬆️
...cks/dev/tooling/configuration/consumers/example.py	`100% <100%> (ø)`	⬆️
...og_checks_dev/datadog_checks/dev/tooling/create.py	`76.11% <100%> (ø)`	⬆️
...tadog_checks_dev/datadog_checks/dev/tooling/git.py	`93.84% <100%> (ø)`	⬆️
...v/datadog_checks/dev/tooling/configuration/spec.py	`100% <100%> (ø)`	⬆️
.../datadog_checks/dev/tooling/configuration/utils.py	`100% <100%> (ø)`	⬆️
...cks_dev/datadog_checks/dev/tooling/requirements.py	`72.64% <50%> (ø)`	⬆️
datadog_checks_dev/datadog_checks/dev/compat.py	`50% <0%> (-50%)`	⬇️
... and 823 more

therve

Looks good, one small change. There is a unicode_literals import to remove too. Do we need the ensure_unicode calls, or can we replace by by a direct encode?

datadog_checks_dev/datadog_checks/dev/tooling/commands/create.py

datadog_checks_dev/datadog_checks/dev/tooling/commands/agent/changelog.py

florimondmanca · 2020-01-20T15:44:46Z

datadog_checks_dev/datadog_checks/dev/tooling/cli.py

-            repo = 'datadog-agent' if repo_choice == 'agent' else 'integrations-{}'.format(repo_choice)
-            echo_warning('`{}` directory `{}` does not exist, defaulting to the current location.'.format(repo, root))
+            repo = 'datadog-agent' if repo_choice == 'agent' else f'integrations-{repo_choice}'
+            echo_warning(f'`{repo}` directory `{root}` does not exist, defaulting to the current location.')


Note: another way to show "this is the output of a string" than using backticks is the !r specifier, e.g. f"some {string!r}" is formatted as some 'string'... We could consider migrating to that as a follow-up PR? 😄

datadog_checks_dev/datadog_checks/dev/tooling/create.py

datadog_checks_dev/setup.py

* Remove Python 2 support from CLI * oops * Update tox.ini * test all * . * ok * address * nice

fmpro12 · 2020-05-27T15:14:13Z

Hey, can be same issue in agent version 7.19?

ofek · 2020-05-27T17:38:21Z

@fmpro12 Hello! Yes Agent 7.x is Python 3-only. You may use Agent 6 if you cannot immediately update custom checks to support Python 3.

Was that your question?

fmpro12 · 2020-05-27T18:33:17Z

Okay, let me explain issue on high level.
I'm trying to install integrations-extras package on top of the latest agent, during installation I got the error:

root@ip-172-31-42-147:~/dd# ddev config set repo extras
Traceback (most recent call last):
File "/usr/local/bin/ddev", line 7, in
from datadog_checks.dev.tooling.cli import ddev
File "/usr/local/lib/python2.7/dist-packages/datadog_checks/dev/tooling/cli.py", line 39
echo_warning(f'Unable to create config file located at {CONFIG_FILE}. Please check your permissions.')
^
SyntaxError: invalid syntax

How I can fix that?

ofek · 2020-05-27T20:31:42Z

You'll need to use Python 3.

fmpro12 · 2020-05-28T08:34:38Z

And hot exactly to do that?

I'm following this instruction - https://docs.datadoghq.com/integrations/ping/.

See commands I'm running on my ubuntu 18.04:

sudo apt update
sudo apt install python3.8
sudo apt-get install python3-pip -y
sudo apt-get install python3.8-dev
sudo python3.8 -m pip install "datadog-checks-dev[cli]"
sudo mkdir $HOME/dd && cd $HOME/dd
sudo git clone https://github.com/DataDog/integrations-extras.git

Then #ddev config set extras ./integrations-extras and i got:

Traceback (most recent call last):
File "/usr/local/bin/ddev", line 7, in
from datadog_checks.dev.tooling.cli import ddev
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/cli.py", line 6, in
from .commands import ALL_COMMANDS
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/init.py", line 6, in
from .clean import clean
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/clean.py", line 42, in
def clean(ctx, check, compiled_only, all_matches, force, verbose):
File "/usr/lib/python3/dist-packages/click/decorators.py", line 151, in decorator
_param_memo(f, ArgumentClass(param_decls, **attrs))
File "/usr/lib/python3/dist-packages/click/core.py", line 1699, in init
Parameter.init(self, param_decls, required=required, **attrs)
TypeError: init() got an unexpected keyword argument 'autocompletion'
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
from apport.fileutils import likely_packaged, get_recent_crashes
File "/usr/lib/python3/dist-packages/apport/init.py", line 5, in
from apport.report import Report
File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in
import apport.fileutils
File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in
from apport.packaging_impl import impl as packaging
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in
import apt
File "/usr/lib/python3/dist-packages/apt/init.py", line 23, in
import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
File "/usr/local/bin/ddev", line 7, in
from datadog_checks.dev.tooling.cli import ddev
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/cli.py", line 6, in
from .commands import ALL_COMMANDS
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/init.py", line 6, in
from .clean import clean
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/clean.py", line 42, in
def clean(ctx, check, compiled_only, all_matches, force, verbose):
File "/usr/lib/python3/dist-packages/click/decorators.py", line 151, in decorator
_param_memo(f, ArgumentClass(param_decls, **attrs))
File "/usr/lib/python3/dist-packages/click/core.py", line 1699, in init
Parameter.init(self, param_decls, required=required, **attrs)
TypeError: init() got an unexpected keyword argument 'autocompletion'
root@ip-172-31-17-211:~/dd# ddev config set extras ./integrations-extras
Traceback (most recent call last):
File "/usr/local/bin/ddev", line 7, in
from datadog_checks.dev.tooling.cli import ddev
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/cli.py", line 6, in
from .commands import ALL_COMMANDS
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/init.py", line 6, in
from .clean import clean
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/clean.py", line 42, in
def clean(ctx, check, compiled_only, all_matches, force, verbose):
File "/usr/lib/python3/dist-packages/click/decorators.py", line 151, in decorator
_param_memo(f, ArgumentClass(param_decls, **attrs))
File "/usr/lib/python3/dist-packages/click/core.py", line 1699, in init
Parameter.init(self, param_decls, required=required, **attrs)
TypeError: init() got an unexpected keyword argument 'autocompletion'
Error in sys.excepthook:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport_python_hook.py", line 63, in apport_excepthook
from apport.fileutils import likely_packaged, get_recent_crashes
File "/usr/lib/python3/dist-packages/apport/init.py", line 5, in
from apport.report import Report
File "/usr/lib/python3/dist-packages/apport/report.py", line 30, in
import apport.fileutils
File "/usr/lib/python3/dist-packages/apport/fileutils.py", line 23, in
from apport.packaging_impl import impl as packaging
File "/usr/lib/python3/dist-packages/apport/packaging_impl.py", line 24, in
import apt
File "/usr/lib/python3/dist-packages/apt/init.py", line 23, in
import apt_pkg
ModuleNotFoundError: No module named 'apt_pkg'

Original exception was:
Traceback (most recent call last):
File "/usr/local/bin/ddev", line 7, in
from datadog_checks.dev.tooling.cli import ddev
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/cli.py", line 6, in
from .commands import ALL_COMMANDS
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/init.py", line 6, in
from .clean import clean
File "/usr/local/lib/python3.8/dist-packages/datadog_checks/dev/tooling/commands/clean.py", line 42, in
def clean(ctx, check, compiled_only, all_matches, force, verbose):
File "/usr/lib/python3/dist-packages/click/decorators.py", line 151, in decorator
_param_memo(f, ArgumentClass(param_decls, **attrs))
File "/usr/lib/python3/dist-packages/click/core.py", line 1699, in init
Parameter.init(self, param_decls, required=required, **attrs)
TypeError: init() got an unexpected keyword argument 'autocompletion'

therve · 2020-05-28T08:43:55Z

It looks like you're using an outdated version of click. Maybe try to uninstall the package and install it using pip. It would make some sense to use a virtualenv too.

It may be worth pursuing this discussion in an issue too. Thanks!

fmpro12 · 2020-05-28T08:53:15Z

Thanks for your response, indeed updating of click fixed the issue.

However now I stuck on:

ddev -e release build ping
Building ping...
Traceback (most recent call last):
File "setup.py", line 4, in
from setuptools import setup
ImportError: No module named setuptools

I installed setuptools but still, get this error.

fmpro12 · 2020-06-01T09:24:44Z

Okay, I was able to fix that wasn't so simple, but the working procedure is below:

apt update
apt-get install python3-pip -y
git clone https://github.com/DataDog/integrations-extras.git
pip3 install "datadog-checks-dev[cli]"
pip3 install "click" --upgrade
cd /usr/bin/
ln -fs python3.6 python
cd /home/ubuntu/
ddev config set extras ./integrations-extras
ddev -e release build ping

There is a mistake in your procedure:

https://docs.datadoghq.com/developers/integrations/new_check_howto/?tab=configurationfile#developer-toolkit

It's should be pip3 install and not pip install - because of python3.
Additionally to pip3 need to upgrade click to version 7
Need to connect create a link between python and python 3.6

* no more validation for py3 nor logos * update Click range #5512 (comment) * ensure non-ancient appdirs https://pypi.org/project/appdirs/#history

ofek · 2020-06-03T12:55:58Z

This should be fixed in 3.8.0 https://pypi.org/project/datadog-checks-dev/3.8.0/

Remove Python 2 support from CLI

c752d92

ofek added the changelog/Changed label Jan 17, 2020

ofek requested review from a team as code owners January 17, 2020 23:15

github-actions bot added dev/tooling documentation integration/datadog_checks_dev labels Jan 17, 2020

ofek added 2 commits January 17, 2020 18:29

oops

7311f3f

Update tox.ini

21548e9

test all

54fb2c2

ofek requested review from a team as code owners January 18, 2020 14:38

github-actions bot added the integration/datadog_checks_base label Jan 18, 2020

ofek added 2 commits January 18, 2020 09:47

.

104e5f3

ok

9e4a3b7

ofek removed the integration/datadog_checks_base label Jan 19, 2020

therve requested changes Jan 20, 2020

View reviewed changes

florimondmanca reviewed Jan 20, 2020

View reviewed changes

ofek added 2 commits January 20, 2020 14:58

address

14396f4

nice

68d26c3

florimondmanca approved these changes Jan 20, 2020

View reviewed changes

therve approved these changes Jan 20, 2020

View reviewed changes

ofek merged commit 4c22dea into master Jan 21, 2020

ofek deleted the ofek/d3 branch January 21, 2020 01:12

AlexandreYang pushed a commit that referenced this pull request Jan 30, 2020

Remove Python 2 support from CLI (#5512)

7f35d3e

* Remove Python 2 support from CLI * oops * Update tox.ini * test all * . * ok * address * nice

ofek mentioned this pull request Feb 1, 2020

Python 2.7 Docker image SyntaxError: invalid syntax DataDog/integrations-extras#530

Closed

ofek mentioned this pull request May 28, 2020

Build packages with the current Python #6770

Merged

ofek added a commit that referenced this pull request Jun 1, 2020

update Click range #5512 (comment)

53153e5

ofek added a commit that referenced this pull request Jun 1, 2020

Update CLI dependencies (#6784)

be94cde

* no more validation for py3 nor logos * update Click range #5512 (comment) * ensure non-ancient appdirs https://pypi.org/project/appdirs/#history

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove Python 2 support from CLI #5512

Remove Python 2 support from CLI #5512

ofek commented Jan 17, 2020

codecov bot commented Jan 17, 2020 •

edited

Loading

therve left a comment

florimondmanca Jan 20, 2020

fmpro12 commented May 27, 2020

ofek commented May 27, 2020

fmpro12 commented May 27, 2020

ofek commented May 27, 2020

fmpro12 commented May 28, 2020

therve commented May 28, 2020

fmpro12 commented May 28, 2020

fmpro12 commented Jun 1, 2020

ofek commented Jun 3, 2020

Remove Python 2 support from CLI #5512

Remove Python 2 support from CLI #5512

Conversation

ofek commented Jan 17, 2020

Motivation

codecov bot commented Jan 17, 2020 • edited Loading

Codecov Report

therve left a comment

Choose a reason for hiding this comment

florimondmanca Jan 20, 2020

Choose a reason for hiding this comment

fmpro12 commented May 27, 2020

ofek commented May 27, 2020

fmpro12 commented May 27, 2020

ofek commented May 27, 2020

fmpro12 commented May 28, 2020

therve commented May 28, 2020

fmpro12 commented May 28, 2020

fmpro12 commented Jun 1, 2020

ofek commented Jun 3, 2020

codecov bot commented Jan 17, 2020 •

edited

Loading