Skip to content

bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt #16366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Sep 25, 2019
Merged

bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt #16366

merged 1 commit into from
Sep 25, 2019

Conversation

vstinner
Copy link
Member

@vstinner vstinner commented Sep 24, 2019

Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed
to test pyvenv.cfg and pybuilddir.txt configuration files.

Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8
encoding, not ASCII.

https://bugs.python.org/issue38234

Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed
to test pyvenv.cfg and pybuilddir.txt configuration files.

Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8
encoding, not ASCII.
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot x86-64 High Sierra 3.x has failed when building commit 52ad33a.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/145/builds/2469) and take a look at the build logs.
  4. Check if the failure is related to this commit (52ad33a) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/145/builds/2469

Failed tests:

  • test_embed

Failed subtests:

  • test_init_pybuilddir - test.test_embed.InitConfigTests
  • test_init_pyvenv_cfg - test.test_embed.InitConfigTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

402 tests OK.

10 slowest tests:

  • test_multiprocessing_spawn: 4 min 13 sec
  • test_concurrent_futures: 3 min 33 sec
  • test_tools: 3 min 15 sec
  • test_tokenize: 3 min 11 sec
  • test_multiprocessing_forkserver: 2 min 34 sec
  • test_lib2to3: 2 min 26 sec
  • test_unicodedata: 1 min 56 sec
  • test_asyncio: 1 min 54 sec
  • test_io: 1 min 9 sec
  • test_pickle: 1 min 9 sec

1 test failed:
test_embed

16 tests skipped:
test_devpoll test_epoll test_gdb test_ioctl test_msilib
test_multiprocessing_fork test_ossaudiodev test_spwd
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_embed

Total duration: 27 min 56 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 1182, in test_init_pyvenv_cfg
    self.check_all_configs("test_init_compat_config", config,
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[655 chars]ne, 'module_search_paths': ['/Users/buildbot/b[1416 chars]': 1} != {'_co[655 chars]ne, 'executable': '/var/folders/sy/9hwmqyx14s1[1400 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmprrz3csek',
-  'base_executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpczgnz4r3/_testembed',
?                      --------


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 1182, in test_init_pyvenv_cfg
    self.check_all_configs("test_init_compat_config", config,
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[655 chars]ne, 'module_search_paths': ['/Users/buildbot/b[1416 chars]': 1} != {'_co[655 chars]ne, 'executable': '/var/folders/sy/9hwmqyx14s1[1400 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpnfvp7q8k',
-  'base_executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp3ihnccdy/_testembed',
?                      --------


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 1130, in test_init_pybuilddir
    self.check_all_configs("test_init_compat_config", config,
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[655 chars]ne, 'module_search_paths': ['/Users/buildbot/b[1407 chars]': 1} != {'_co[655 chars]ne, 'executable': '/var/folders/sy/9hwmqyx14s1[1383 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
-  'base_executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmp2a98rb5k/_testembed',
?                      --------


Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 1130, in test_init_pybuilddir
    self.check_all_configs("test_init_compat_config", config,
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/Users/buildbot/buildarea/3.x.billenstein-sierra/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[655 chars]ne, 'module_search_paths': ['/Users/buildbot/b[1407 chars]': 1} != {'_co[655 chars]ne, 'executable': '/var/folders/sy/9hwmqyx14s1[1383 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/Users/buildbot/buildarea/3.x.billenstein-sierra/build/target',
-  'base_executable': '/private/var/folders/sy/9hwmqyx14s11577cvgzwf2v40000gt/T/tmpdn3tru0j/_testembed',
?                      --------

@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 FreeBSD CURRENT Shared 3.x has failed when building commit 52ad33a.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/168/builds/1510) and take a look at the build logs.
  4. Check if the failure is related to this commit (52ad33a) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/168/builds/1510

Failed tests:

  • test_embed

Failed subtests:

  • test_init_pybuilddir - test.test_embed.InitConfigTests
  • test_init_pyvenv_cfg - test.test_embed.InitConfigTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

402 tests OK.

10 slowest tests:

  • test_tools: 6 min 35 sec
  • test_multiprocessing_spawn: 6 min 19 sec
  • test_tokenize: 6 min 2 sec
  • test_lib2to3: 5 min 16 sec
  • test_concurrent_futures: 4 min 44 sec
  • test_unicodedata: 3 min 57 sec
  • test_regrtest: 3 min 18 sec
  • test_pickle: 3 min 9 sec
  • test_multiprocessing_forkserver: 3 min 8 sec
  • test_subprocess: 2 min 34 sec

1 test failed:
test_embed

16 tests skipped:
test_dbm_gnu test_devpoll test_epoll test_gdb test_ioctl
test_msilib test_ossaudiodev test_spwd test_startfile test_tix
test_tk test_ttk_guionly test_winconsoleio test_winreg
test_winsound test_zipfile64

1 re-run test:
test_embed

Total duration: 27 min 1 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 1130, in test_init_pybuilddir
    self.check_all_configs("test_init_compat_config", config,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[671 chars]ne, 'module_search_paths': ['/usr/home/buildbo[1297 chars]': 1} != {'_co[671 chars]ne, 'executable': '/tmp/tmpzyou017j/_testembed[1285 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
-  'base_executable': '/var/tmp/tmpzyou017j/_testembed',
?                      ----


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 1130, in test_init_pybuilddir
    self.check_all_configs("test_init_compat_config", config,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[671 chars]ne, 'module_search_paths': ['/usr/home/buildbo[1297 chars]': 1} != {'_co[671 chars]ne, 'executable': '/tmp/tmpsa4mgwz2/_testembed[1285 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/target',
-  'base_executable': '/var/tmp/tmpsa4mgwz2/_testembed',
?                      ----


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 1182, in test_init_pyvenv_cfg
    self.check_all_configs("test_init_compat_config", config,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[671 chars]ne, 'module_search_paths': ['/usr/home/buildbo[1216 chars]': 1} != {'_co[671 chars]ne, 'executable': '/tmp/tmp4yyuft16/_testembed[1208 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/tmp/tmpkqpl7bdt',
-  'base_executable': '/var/tmp/tmp4yyuft16/_testembed',
?                      ----


Traceback (most recent call last):
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 1182, in test_init_pyvenv_cfg
    self.check_all_configs("test_init_compat_config", config,
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 681, in check_all_configs
    self.check_config(configs, expected_config)
  File "/usr/home/buildbot/python/3.x.koobs-freebsd-current/build/Lib/test/test_embed.py", line 613, in check_config
    self.assertEqual(config, expected)
AssertionError: {'_co[671 chars]ne, 'module_search_paths': ['/usr/home/buildbo[1216 chars]': 1} != {'_co[671 chars]ne, 'executable': '/tmp/tmpszvtbyq4/_testembed[1208 chars]': 1}
  {'_config_init': 1,
   '_init_main': 1,
   '_install_importlib': 1,
   'argv': [''],
   'base_exec_prefix': '/tmp/tmpr0gyjdqs',
-  'base_executable': '/var/tmp/tmpszvtbyq4/_testembed',
?                      ----

vstinner added a commit that referenced this pull request Sep 26, 2019
)

* bpo-38234: Py_SetPath() uses the program full path (GH-16357)

Py_SetPath() now sets sys.executable to the program full path
(Py_GetProgramFullPath()), rather than to the program name
(Py_GetProgramName()).

Fix also memory leaks in pathconfig_set_from_config().

(cherry picked from commit 1ce152a)

* bpo-38234: Add tests for Python init path config (GH-16358)


(cherry picked from commit bb6bf7d)

* bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt (GH-16366)

Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed
to test pyvenv.cfg and pybuilddir.txt configuration files.

Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8
encoding, not ASCII.

(cherry picked from commit 52ad33a)

* bpo-38234: Cleanup getpath.c (GH-16367)

* search_for_prefix() directly calls reduce() if found is greater
  than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path

(cherry picked from commit 221fd84)

* bpo-38234: Fix test_embed pathconfig tests (GH-16390)

bpo-38234: On macOS and FreeBSD, the temporary directory can be
symbolic link. For example, /tmp can be a symbolic link to /var/tmp.
Call realpath() to resolve all symbolic links.

(cherry picked from commit 00508a7)

* bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402)

* Add test_embed.test_init_setpath_config(): test Py_SetPath()
  with PyConfig.
* test_init_setpath() and test_init_setpythonhome() no longer call
  Py_SetProgramName(), but use the default program name.
* _PyPathConfig: isolated, site_import  and base_executable
  fields are now only available on Windows.
* If executable is set explicitly in the configuration, ignore
  calculated base_executable: _PyConfig_InitPathConfig() copies
  executable to base_executable.
* Complete path config documentation.

(cherry picked from commit 8bf39b6)

* bpo-38234: Complete init config documentation (GH-16404)


(cherry picked from commit 88feaec)

* bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406)

Explicitly preinitializes with a Python preconfiguration to avoid
Py_SetPath() implicit preinitialization with a compat
preconfiguration.

Fix also test_init_setpath() and test_init_setpythonhome() on macOS:
use self.test_exe as the executable (and base_executable), rather
than shutil.which('python3').

(cherry picked from commit 49d99f0)

* bpo-38234: Py_Initialize() sets global path configuration (GH-16421)

* Py_InitializeFromConfig() now writes PyConfig path configuration to
  the global path configuration (_Py_path_config).
* Add test_embed.test_get_pathconfig().
* Fix typo in _PyWideStringList_Join().

(cherry picked from commit 12f2f17)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants