Description
Is your feature request related to a problem? Please describe.
We run Linux systems that contain multiple Pipenv projects which span multiple versions of Python. We use pyenv
to manage those versions. We set PYENV_ROOT
to a common directory in /opt
so that they can share Python builds. We set that PYENV_ROOT=/opt/pyenv in an
.env` file checked into each project. This has two benefits:
- We can refer to that
.env
file as a common "vars that should be set for this project" from other programs that understand such files, e.g.EnvironmentFile
directives for Systemd services. - We can invoke
pipenv
from shells that are not specially configured, e.g. human users, service launcher scripts and makefiles.
However, when Pipenv wants to use Pyenv, it expects it to be on PATH
. Users of these systems then have to modify their PATH (by whatever means they wish) to point at the /opt/pyenv
directory. Despite respecting PYENV_ROOT
, Pipenv does not use that variable to resolve the pyenv
binary itself.
Describe the solution you'd like
First change: When Pipenv wants to install Python via pipenv
or asdf
, it should try additional locations to find an installer executable. At present, it searches PATH
. I would like it to search the following, in order:
PATH
- The respective "installer root" environment variable +
/bin/{installer_name}
. The variables in question areASDF_DIR
andPYENV_ROOT
. - The default recommended source-control based install location for the installer +
/bin/{installer_name}
.
If an executable file of the expected name is found in any of those locations it should be used.
Second change: whether or not an executable installer can be found according to the above logic should be used to decide whether or not to try the other installer (the pyenv
-> asdf
fallback). At present, pythonfinder
's methods which only check for whether or not an installer's environment variables are configured will be used. On users' systems with both pyenv
and asdf
, where only asdf
is activated, this will be a behavior change; pyenv
will now be used for installation.
Describe alternatives you've considered
- Manual workaround: every human user could ensure
pyenv
/asdf
was on their PATH. Every launcher script could appripriately modifyPATH
before launching. Pros: requires no Pipenv changes. Cons: all code which wants to do a hands-freepipenv install
on a system without an installer on the path will have to be aware of this and be modified. - Common shell hacks: a common
PATH
modification could be added to systemwide shell profile, sourced by all users, to make the installers available. Pros: Requires no Pipenv changes. Cons: doesn't work for programs which invoke Pipenv which don't run in a shell (e.g. Systemd); doesn't work for users with esoteric shells. - Make
.env
files dynamic. If we could writePATH=/opt/pyenv/bin:$PATH
in an.env
file and have the$PATH
reference be substituted, the problem would be solved. Libraries exist to add this dynamic behavior to.env
files. Pros: solves this at a lower level than the "find an installer binary" logic. Cons: many other programs consume.env
files and do not perform variable substitution. Only Laravel performs that substitution explicitly to my knowledge (andsource .env
does as well, but that's not treating it as an.env
file but rather a shell script).
Additional context
When running pipenv shell
without a pyenv
binary on the PATH
, even with PYENV_ROOT
set properly, the following trace is emitted:
Loading .env environment variables...
Warning: Python 3.8 was not found on your system...
Traceback (most recent call last):
File "/home/zacbentley/.local/bin/pipenv", line 8, in <module>
sys.exit(cli())
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 764, in __call__
return self.main(*args, **kwargs)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 717, in main
rv = self.invoke(ctx)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/click/core.py", line 555, in invoke
return callback(*args, **kwargs)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/cli/command.py", line 390, in shell
pypi_mirror=state.pypi_mirror,
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 2156, in do_shell
three=three, python=python, validate=False, pypi_mirror=pypi_mirror,
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 574, in ensure_project
pypi_mirror=pypi_mirror,
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 494, in ensure_virtualenv
python = ensure_python(three=three, python=python)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/core.py", line 419, in ensure_python
version = pyenv.find_version_to_install(python)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 99, in find_version_to_install
), key=operator.attrgetter('cmpkey'))
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 96, in <genexpr>
inst_version
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 79, in iter_installable_versions
for name in self._pyenv('install', '--list').out.splitlines():
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/pyenv.py", line 70, in _pyenv
c = delegator.run(args, block=False, timeout=timeout)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 336, in run
c.run(block=block, binary=binary, cwd=cwd, env=env)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/delegator.py", line 204, in run
s = PopenSpawn(self._popen_args, **pexpect_kwargs)
File "/home/zacbentley/.local/lib/python2.7/site-packages/pipenv/vendor/pexpect/popen_spawn.py", line 53, in __init__
self.proc = subprocess.Popen(cmd, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
$ pipenv --support
Pipenv version: '2018.11.26'
Pipenv location: '/home/zacbentley/.local/lib/python2.7/site-packages/pipenv'
Python location: '/usr/bin/python'
Python installations found:
3.5.2
:/usr/bin/python3.5
3.5.2
:/usr/bin/python3.5m
2.7.12
:/usr/bin/python
PEP 508 Information:
{'implementation_name': 'cpython',
'implementation_version': '0',
'os_name': 'posix',
'platform_machine': 'x86_64',
'platform_python_implementation': 'CPython',
'platform_release': '4.4.0-1102-aws',
'platform_system': 'Linux',
'platform_version': '#113-Ubuntu SMP Wed Jan 29 14:54:54 UTC 2020',
'python_full_version': '2.7.12',
'python_version': '2.7',
'sys_platform': 'linux2'}
System environment variables:
LC_TERMINAL_VERSION
PYTHONDONTWRITEBYTECODE
LESSOPEN
SSH_CLIENT
LOGNAME
USER
PATH
HOME
LANG
LC_TERMINAL
TERM
SHELL
SHLVL
PYTHONFINDER_IGNORE_UNSUPPORTED
XDG_RUNTIME_DIR
SSH_AUTH_SOCK
TMUX
PIP_PYTHON_PATH
XDG_SESSION_ID
_
SSH_CONNECTION
LESSCLOSE
SSH_TTY
OLDPWD
XDG_DATA_DIRS
PWD
PIP_SHIMS_BASE_MODULE
PIP_DISABLE_PIP_VERSION_CHECK
MAIL
LS_COLORS
TMUX_PANE
Pipenv–specific environment variables:
Debug–specific environment variables:
PATH
:/home/zacbentley/bin:/home/zacbentley/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
SHELL
:/bin/bash
LANG
:en_US.UTF-8
PWD
:/home/zacbentley/cloudwatch_test
Contents of Pipfile
('/home/zacbentley/cloudwatch_test/Pipfile'):
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
flask = "*"
pyopenssl = "*"
validatesns = "*"
boto3 = "*"
[requires]
python_version = "3.8"
Contents of Pipfile.lock
('/home/zacbentley/cloudwatch_test/Pipfile.lock'):
{
"_meta": {
"hash": {
"sha256": "965842e951cb3f46daf1ce0b8f0203fdef361b2da8c54d8f27b99cc13b9d32c0"
},
"pipfile-spec": 6,
"requires": {
"python_version": "3.8"
},
"sources": [
{
"name": "pypi",
"url": "https://pypi.org/simple",
"verify_ssl": true
}
]
},
"default": {
"asn1crypto": {
"hashes": [
"sha256:5a215cb8dc12f892244e3a113fe05397ee23c5c4ca7a69cd6e69811755efc42d",
"sha256:831d2710d3274c8a74befdddaf9f17fcbf6e350534565074818722d6d615b315"
],
"version": "==1.3.0"
},
"boto3": {
"hashes": [
"sha256:989ede38b9f69743e2536ccf371941354e77103b47b37d5ba90f77718368a248",
"sha256:dc4d17a9b0bd6fb03b2650a0f7c7b6a458583fe7bebbcd6bbefd299d7169fb5b"
],
"index": "pypi",
"version": "==1.13.3"
},
"botocore": {
"hashes": [
"sha256:d291035e643c353029df8985cbc0bbdcdf9117fff81c715dd688aadd51816f41",
"sha256:e2c384f378d96c09079d9fa44be4f6a849c4d5295de68c9c8afc8783f7a5a2a2"
],
"version": "==1.16.3"
},
"cffi": {
"hashes": [
"sha256:001bf3242a1bb04d985d63e138230802c6c8d4db3668fb545fb5005ddf5bb5ff",
"sha256:00789914be39dffba161cfc5be31b55775de5ba2235fe49aa28c148236c4e06b",
"sha256:028a579fc9aed3af38f4892bdcc7390508adabc30c6af4a6e4f611b0c680e6ac",
"sha256:14491a910663bf9f13ddf2bc8f60562d6bc5315c1f09c704937ef17293fb85b0",
"sha256:1cae98a7054b5c9391eb3249b86e0e99ab1e02bb0cc0575da191aedadbdf4384",
"sha256:2089ed025da3919d2e75a4d963d008330c96751127dd6f73c8dc0c65041b4c26",
"sha256:2d384f4a127a15ba701207f7639d94106693b6cd64173d6c8988e2c25f3ac2b6",
"sha256:337d448e5a725bba2d8293c48d9353fc68d0e9e4088d62a9571def317797522b",
"sha256:399aed636c7d3749bbed55bc907c3288cb43c65c4389964ad5ff849b6370603e",
"sha256:3b911c2dbd4f423b4c4fcca138cadde747abdb20d196c4a48708b8a2d32b16dd",
"sha256:3d311bcc4a41408cf5854f06ef2c5cab88f9fded37a3b95936c9879c1640d4c2",
"sha256:62ae9af2d069ea2698bf536dcfe1e4eed9090211dbaafeeedf5cb6c41b352f66",
"sha256:66e41db66b47d0d8672d8ed2708ba91b2f2524ece3dee48b5dfb36be8c2f21dc",
"sha256:675686925a9fb403edba0114db74e741d8181683dcf216be697d208857e04ca8",
"sha256:7e63cbcf2429a8dbfe48dcc2322d5f2220b77b2e17b7ba023d6166d84655da55",
"sha256:8a6c688fefb4e1cd56feb6c511984a6c4f7ec7d2a1ff31a10254f3c817054ae4",
"sha256:8c0ffc886aea5df6a1762d0019e9cb05f825d0eec1f520c51be9d198701daee5",
"sha256:95cd16d3dee553f882540c1ffe331d085c9e629499ceadfbda4d4fde635f4b7d",
"sha256:99f748a7e71ff382613b4e1acc0ac83bf7ad167fb3802e35e90d9763daba4d78",
"sha256:b8c78301cefcf5fd914aad35d3c04c2b21ce8629b5e4f4e45ae6812e461910fa",
"sha256:c420917b188a5582a56d8b93bdd8e0f6eca08c84ff623a4c16e809152cd35793",
"sha256:c43866529f2f06fe0edc6246eb4faa34f03fe88b64a0a9a942561c8e22f4b71f",
"sha256:cab50b8c2250b46fe738c77dbd25ce017d5e6fb35d3407606e7a4180656a5a6a",
"sha256:cef128cb4d5e0b3493f058f10ce32365972c554572ff821e175dbc6f8ff6924f",
"sha256:cf16e3cf6c0a5fdd9bc10c21687e19d29ad1fe863372b5543deaec1039581a30",
"sha256:e56c744aa6ff427a607763346e4170629caf7e48ead6921745986db3692f987f",
"sha256:e577934fc5f8779c554639376beeaa5657d54349096ef24abe8c74c5d9c117c3",
"sha256:f2b0fa0c01d8a0c7483afd9f31d7ecf2d71760ca24499c8697aeb5ca37dc090c"
],
"version": "==1.14.0"
},
"click": {
"hashes": [
"sha256:d2b5255c7c6349bc1bd1e59e08cd12acbbd63ce649f2588755783aa94dfb6b1a",
"sha256:dacca89f4bfadd5de3d7489b7c8a566eee0d3676333fbb50030263894c38c0dc"
],
"version": "==7.1.2"
},
"cryptography": {
"hashes": [
"sha256:091d31c42f444c6f519485ed528d8b451d1a0c7bf30e8ca583a0cac44b8a0df6",
"sha256:18452582a3c85b96014b45686af264563e3e5d99d226589f057ace56196ec78b",
"sha256:1dfa985f62b137909496e7fc182dac687206d8d089dd03eaeb28ae16eec8e7d5",
"sha256:1e4014639d3d73fbc5ceff206049c5a9a849cefd106a49fa7aaaa25cc0ce35cf",
"sha256:22e91636a51170df0ae4dcbd250d318fd28c9f491c4e50b625a49964b24fe46e",
"sha256:3b3eba865ea2754738616f87292b7f29448aec342a7c720956f8083d252bf28b",
"sha256:651448cd2e3a6bc2bb76c3663785133c40d5e1a8c1a9c5429e4354201c6024ae",
"sha256:726086c17f94747cedbee6efa77e99ae170caebeb1116353c6cf0ab67ea6829b",
"sha256:844a76bc04472e5135b909da6aed84360f522ff5dfa47f93e3dd2a0b84a89fa0",
"sha256:88c881dd5a147e08d1bdcf2315c04972381d026cdb803325c03fe2b4a8ed858b",
"sha256:96c080ae7118c10fcbe6229ab43eb8b090fccd31a09ef55f83f690d1ef619a1d",
"sha256:a0c30272fb4ddda5f5ffc1089d7405b7a71b0b0f51993cb4e5dbb4590b2fc229",
"sha256:bb1f0281887d89617b4c68e8db9a2c42b9efebf2702a3c5bf70599421a8623e3",
"sha256:c447cf087cf2dbddc1add6987bbe2f767ed5317adb2d08af940db517dd704365",
"sha256:c4fd17d92e9d55b84707f4fd09992081ba872d1a0c610c109c18e062e06a2e55",
"sha256:d0d5aeaedd29be304848f1c5059074a740fa9f6f26b84c5b63e8b29e73dfc270",
"sha256:daf54a4b07d67ad437ff239c8a4080cfd1cc7213df57d33c97de7b4738048d5e",
"sha256:e993468c859d084d5579e2ebee101de8f5a27ce8e2159959b6673b418fd8c785",
"sha256:f118a95c7480f5be0df8afeb9a11bd199aa20afab7a96bcf20409b411a3a85f0"
],
"version": "==2.9.2"
},
"docutils": {
"hashes": [
"sha256:6c4f696463b79f1fb8ba0c594b63840ebd41f059e92b31957c46b74a4599b6d0",
"sha256:9e4d7ecfc600058e07ba661411a2b7de2fd0fafa17d1a7f7361cd47b1175c827",
"sha256:a2aeea129088da402665e92e0b25b04b073c04b2dce4ab65caaa38b7ce2e1a99"
],
"version": "==0.15.2"
},
"flask": {
"hashes": [
"sha256:4efa1ae2d7c9865af48986de8aeb8504bf32c7f3d6fdc9353d34b21f4b127060",
"sha256:8a4fdd8936eba2512e9c85df320a37e694c93945b33ef33c89946a340a238557"
],
"index": "pypi",
"version": "==1.1.2"
},
"itsdangerous": {
"hashes": [
"sha256:321b033d07f2a4136d3ec762eac9f16a10ccd60f53c0c91af90217ace7ba1f19",
"sha256:b12271b2047cb23eeb98c8b5622e2e5c5e9abd9784a153e9d8ef9cb4dd09d749"
],
"version": "==1.1.0"
},
"jinja2": {
"hashes": [
"sha256:89aab215427ef59c34ad58735269eb58b1a5808103067f7bb9d5836c651b3bb0",
"sha256:f0a4641d3cf955324a89c04f3d94663aa4d638abe8f733ecd3582848e1c37035"
],
"version": "==2.11.2"
},
"jmespath": {
"hashes": [
"sha256:695cb76fa78a10663425d5b73ddc5714eb711157e52704d69be03b1a02ba4fec",
"sha256:cca55c8d153173e21baa59983015ad0daf603f9cb799904ff057bfb8ff8dc2d9"
],
"version": "==0.9.5"
},
"markupsafe": {
"hashes": [
"sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473",
"sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161",
"sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235",
"sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5",
"sha256:13d3144e1e340870b25e7b10b98d779608c02016d5184cfb9927a9f10c689f42",
"sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff",
"sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b",
"sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1",
"sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e",
"sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183",
"sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66",
"sha256:596510de112c685489095da617b5bcbbac7dd6384aeebeda4df6025d0256a81b",
"sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1",
"sha256:6788b695d50a51edb699cb55e35487e430fa21f1ed838122d722e0ff0ac5ba15",
"sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1",
"sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e",
"sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b",
"sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905",
"sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735",
"sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d",
"sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e",
"sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d",
"sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c",
"sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21",
"sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2",
"sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5",
"sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b",
"sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6",
"sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f",
"sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f",
"sha256:cdb132fc825c38e1aeec2c8aa9338310d29d337bebbd7baa06889d09a60a1fa2",
"sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7",
"sha256:e8313f01ba26fbbe36c7be1966a7b7424942f670f38e666995b88d012765b9be"
],
"version": "==1.1.1"
},
"oscrypto": {
"hashes": [
"sha256:2f53f50813ecc5114fd126b89766fabdb9448e0bb080814451cc4bb92acafca4",
"sha256:e9ea9401294b20ee17225502cc308660814ba8de386f94bc9b1dcf518d5399d9"
],
"version": "==1.2.0"
},
"pycparser": {
"hashes": [
"sha256:2d475327684562c3a96cc71adf7dc8c4f0565175cf86b6d7a404ff4c771f15f0",
"sha256:7582ad22678f0fcd81102833f60ef8d0e57288b6b5fb00323d101be910e35705"
],
"version": "==2.20"
},
"pyopenssl": {
"hashes": [
"sha256:621880965a720b8ece2f1b2f54ea2071966ab00e2970ad2ce11d596102063504",
"sha256:9a24494b2602aaf402be5c9e30a0b82d4a5c67528fe8fb475e3f3bc00dd69507"
],
"index": "pypi",
"version": "==19.1.0"
},
"python-dateutil": {
"hashes": [
"sha256:73ebfe9dbf22e832286dafa60473e4cd239f8592f699aa5adaf10050e6e1823c",
"sha256:75bb3f31ea686f1197762692a9ee6a7550b59fc6ca3a1f4b5d7e32fb98e2da2a"
],
"version": "==2.8.1"
},
"s3transfer": {
"hashes": [
"sha256:2482b4259524933a022d59da830f51bd746db62f047d6eb213f2f8855dcb8a13",
"sha256:921a37e2aefc64145e7b73d50c71bb4f26f46e4c9f414dc648c6245ff92cf7db"
],
"version": "==0.3.3"
},
"six": {
"hashes": [
"sha256:236bdbdce46e6e6a3d61a337c0f8b763ca1e8717c03b369e87a7ec7ce1319c0a",
"sha256:8f3cd2e254d8f793e7f3d6d9df77b92252b52637291d0f0da013c76ea2724b6c"
],
"version": "==1.14.0"
},
"urllib3": {
"hashes": [
"sha256:3018294ebefce6572a474f0604c2021e33b3fd8006ecd11d62107a5d2a963527",
"sha256:88206b0eb87e6d677d424843ac5209e3fb9d0190d0ee169599165ec25e9d9115"
],
"markers": "python_version != '3.4'",
"version": "==1.25.9"
},
"validatesns": {
"hashes": [
"sha256:8596c0d0df88e9ba01da657b784262381ac0ff87097808d408bd883c36710493"
],
"index": "pypi",
"version": "==0.1.1"
},
"werkzeug": {
"hashes": [
"sha256:2de2a5db0baeae7b2d2664949077c2ac63fbd16d98da0ff71837f7d1dea3fd43",
"sha256:6c80b1e5ad3665290ea39320b91e1be1e0d5f60652b964a3070216de83d2e47c"
],
"version": "==1.0.1"
}
},
"develop": {}
}