pipenv install should not bring back old versions of dependent packages, if newer versions are already installed

[vvaradarajan]

Be sure to check the existing issues (both open and closed!), and make sure you are running the latest version of Pipenv.

Check the diagnose documentation for common issues before posting! We may close your issue if it is very similar to one of them. Please be considerate, or be on your way.

Make sure to mention your debugging experience if the documented solution failed.

Issue description

pipenv install sqlalchemy #the Pipfile is updated with sqlalchemy = "*"
pip install sqlalchemy #This correctly installs 'current' sqlalchemy 2.0 ( "version": "==2.0.28")
pipenv install dataset #This changes the previously installed sqlalchemy to "version": "==1.4.52"

Expected result

pipenv install dataset should error out, indicating that sqlalchemy is installed and at a later incompatible version.
Discussion: The application was working with the installed sqlalchemy version 2.0.. After installing dataset, the sqlalchemy version went backward to 1.4.52 and application failed.
In general the newer versions offer new features, and retain some compatibility with older versions. So upgrading packages during pipenv dependency resolution is acceptible. But downgrading packages is not..

---

$ pipenv --support

Pipenv version: '2023.12.1'

Pipenv location: '/home/vv/junk/venv/lib/python3.12/site-packages/pipenv'

Python location: '/home/vv/junk/venv/bin/python3.12'

OS Name: 'posix'

User pip version: '23.3.2'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.12.1',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '6.3.0-060300-generic',
 'platform_system': 'Linux',
 'platform_version': '#202304232030 SMP PREEMPT_DYNAMIC Sun Apr 23 20:37:49 '
                     'UTC 2023',
 'python_full_version': '3.12.1',
 'python_version': '3.12',
 'sys_platform': 'linux'}

System environment variables:

• SHELL
• SESSION_MANAGER
• QT_ACCESSIBILITY
• COLORTERM
• XDG_CONFIG_DIRS
• XDG_SESSION_PATH
• TERM_PROGRAM_VERSION
• GNOME_DESKTOP_SESSION_ID
• LANGUAGE
• SSH_AUTH_SOCK
• CINNAMON_VERSION
• DESKTOP_SESSION
• GTK_MODULES
• XDG_SEAT
• PWD
• XDG_SESSION_DESKTOP
• LOGNAME
• QT_QPA_PLATFORMTHEME
• XDG_SESSION_TYPE
• GPG_AGENT_INFO
• XAUTHORITY
• VSCODE_GIT_ASKPASS_NODE
• XDG_GREETER_DATA_DIR
• GJS_DEBUG_TOPICS
• GDM_LANG
• HOME
• LANG
• XDG_CURRENT_DESKTOP
• VIRTUAL_ENV
• GIT_ASKPASS
• XDG_SEAT_PATH
• CHROME_DESKTOP
• GJS_DEBUG_OUTPUT
• VSCODE_GIT_ASKPASS_EXTRA_ARGS
• XDG_SESSION_CLASS
• TERM
• GTK_OVERLAY_SCROLLING
• USER
• VSCODE_GIT_IPC_HANDLE
• DISPLAY
• SHLVL
• XDG_VTNR
• XDG_SESSION_ID
• VIRTUAL_ENV_PROMPT
• XDG_RUNTIME_DIR
• PS1
• VSCODE_GIT_ASKPASS_MAIN
• GTK3_MODULES
• XDG_DATA_DIRS
• GDK_BACKEND
• PATH
• GDMSESSION
• ORIGINAL_XDG_CURRENT_DESKTOP
• DBUS_SESSION_BUS_ADDRESS
• GIO_LAUNCHED_DESKTOP_FILE_PID
• GIO_LAUNCHED_DESKTOP_FILE
• OLDPWD
• TERM_PROGRAM
• _
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONDONTWRITEBYTECODE
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /home/vv/junk/venv/bin:/home/vv/.vscode/extensions/ms-python.python-2024.2.0/pythonFiles/deactivate/bash:/home/vv/git/OBD/venv/bin:/home/vv/.local/bin:/home/vv/.vscode/extensions/ms-python.python-2024.2.0/pythonFiles/deactivate/bash:/home/vv/git/OBD/venv/bin:/home/vv/.local/bin:/home/vv/.local/bin:/home/vv/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
• SHELL: /bin/bash
• LANG: en_US.UTF-8
• PWD: /home/vv/junk/venv
• VIRTUAL_ENV: /home/vv/junk/venv

---

Contents of Pipfile ('/home/vv/junk/venv/Pipfile'):

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
sqlalchemy = "*"
dataset = "*"

[dev-packages]

[requires]
python_version = "3.12"

Contents of Pipfile.lock ('/home/vv/junk/venv/Pipfile.lock'):

{
    "_meta": {
        "hash": {
            "sha256": "1f667241d0154cea18e01babaad381a7a09e36e4c55f0ba33329889ca852a494"
        },
        "pipfile-spec": 6,
        "requires": {
            "python_version": "3.12"
        },
        "sources": [
            {
                "name": "pypi",
                "url": "https://pypi.org/simple",
                "verify_ssl": true
            }
        ]
    },
    "default": {
        "alembic": {
            "hashes": [
                "sha256:2edcc97bed0bd3272611ce3a98d98279e9c209e7186e43e75bbb1b2bdfdbcc43",
                "sha256:4932c8558bf68f2ee92b9bbcb8218671c627064d5b08939437af6d77dc05e595"
            ],
            "markers": "python_version >= '3.8'",
            "version": "==1.13.1"
        },
        "banal": {
            "hashes": [
                "sha256:2fe02c9305f53168441948f4a03dfbfa2eacc73db30db4a93309083cb0e250a5",
                "sha256:877aacb16b17f8fa4fd29a7c44515c5a23dc1a7b26078bc41dd34829117d85e1"
            ],
            "version": "==1.0.6"
        },
        "dataset": {
            "hashes": [
                "sha256:77d362118f67a8cbb4848dbd30ab362b9fa7cfebdbfaf426c9c500cb38969a99",
                "sha256:dcca9ba7658473d3082b1adf87a650252a1cd665705b73fa7d4ee32116a107b9"
            ],
            "index": "pypi",
            "version": "==1.6.2"
        },
        "greenlet": {
            "hashes": [
                "sha256:43374442353259554ce33599da8b692d5aa96f8976d567d4badf263371fbe491",
                "sha256:881b7db1ebff4ba09aaaeae6aa491daeb226c8150fc20e836ad00041bcb11230",
                "sha256:fcd2469d6a2cf298f198f0487e0a5b1a47a42ca0fa4dfd1b6862c999f018ebbf"
            ],
            "markers": "python_version >= '3' and platform_machine == 'aarch64' or (platform_machine == 'ppc64le' or (platform_machine == 'x86_64' or (platform_machine == 'amd64' or (platform_machine == 'AMD64' or (platform_machine == 'win32' or platform_machine == 'WIN32')))))",
            "version": "==3.0.3"
        },
        "mako": {
            "hashes": [
                "sha256:2a0c8ad7f6274271b3bb7467dd37cf9cc6dab4bc19cb69a4ef10669402de698e",
                "sha256:32a99d70754dfce237019d17ffe4a282d2d3351b9c476e90d8a60e63f133b80c"
            ],
            "markers": "python_version >= '3.8'",
            "version": "==1.3.2"
        },
        "markupsafe": {
            "hashes": [
                "sha256:00e046b6dd71aa03a41079792f8473dc494d564611a8f89bbbd7cb93295ebdcf",
                "sha256:075202fa5b72c86ad32dc7d0b56024ebdbcf2048c0ba09f1cde31bfdd57bcfff",
                "sha256:0e397ac966fdf721b2c528cf028494e86172b4feba51d65f81ffd65c63798f3f",
                "sha256:17b950fccb810b3293638215058e432159d2b71005c74371d784862b7e4683f3",
                "sha256:1f3fbcb7ef1f16e48246f704ab79d79da8a46891e2da03f8783a5b6fa41a9532",
                "sha256:2174c595a0d73a3080ca3257b40096db99799265e1c27cc5a610743acd86d62f",
                "sha256:2b7c57a4dfc4f16f7142221afe5ba4e093e09e728ca65c51f5620c9aaeb9a617",
                "sha256:2d2d793e36e230fd32babe143b04cec8a8b3eb8a3122d2aceb4a371e6b09b8df",
                "sha256:30b600cf0a7ac9234b2638fbc0fb6158ba5bdcdf46aeb631ead21248b9affbc4",
                "sha256:397081c1a0bfb5124355710fe79478cdbeb39626492b15d399526ae53422b906",
                "sha256:3a57fdd7ce31c7ff06cdfbf31dafa96cc533c21e443d57f5b1ecc6cdc668ec7f",
                "sha256:3c6b973f22eb18a789b1460b4b91bf04ae3f0c4234a0a6aa6b0a92f6f7b951d4",
                "sha256:3e53af139f8579a6d5f7b76549125f0d94d7e630761a2111bc431fd820e163b8",
                "sha256:4096e9de5c6fdf43fb4f04c26fb114f61ef0bf2e5604b6ee3019d51b69e8c371",
                "sha256:4275d846e41ecefa46e2015117a9f491e57a71ddd59bbead77e904dc02b1bed2",
                "sha256:4c31f53cdae6ecfa91a77820e8b151dba54ab528ba65dfd235c80b086d68a465",
                "sha256:4f11aa001c540f62c6166c7726f71f7573b52c68c31f014c25cc7901deea0b52",
                "sha256:5049256f536511ee3f7e1b3f87d1d1209d327e818e6ae1365e8653d7e3abb6a6",
                "sha256:58c98fee265677f63a4385256a6d7683ab1832f3ddd1e66fe948d5880c21a169",
                "sha256:598e3276b64aff0e7b3451b72e94fa3c238d452e7ddcd893c3ab324717456bad",
                "sha256:5b7b716f97b52c5a14bffdf688f971b2d5ef4029127f1ad7a513973cfd818df2",
                "sha256:5dedb4db619ba5a2787a94d877bc8ffc0566f92a01c0ef214865e54ecc9ee5e0",
                "sha256:619bc166c4f2de5caa5a633b8b7326fbe98e0ccbfacabd87268a2b15ff73a029",
                "sha256:629ddd2ca402ae6dbedfceeba9c46d5f7b2a61d9749597d4307f943ef198fc1f",
                "sha256:656f7526c69fac7f600bd1f400991cc282b417d17539a1b228617081106feb4a",
                "sha256:6ec585f69cec0aa07d945b20805be741395e28ac1627333b1c5b0105962ffced",
                "sha256:72b6be590cc35924b02c78ef34b467da4ba07e4e0f0454a2c5907f473fc50ce5",
                "sha256:7502934a33b54030eaf1194c21c692a534196063db72176b0c4028e140f8f32c",
                "sha256:7a68b554d356a91cce1236aa7682dc01df0edba8d043fd1ce607c49dd3c1edcf",
                "sha256:7b2e5a267c855eea6b4283940daa6e88a285f5f2a67f2220203786dfa59b37e9",
                "sha256:823b65d8706e32ad2df51ed89496147a42a2a6e01c13cfb6ffb8b1e92bc910bb",
                "sha256:8590b4ae07a35970728874632fed7bd57b26b0102df2d2b233b6d9d82f6c62ad",
                "sha256:8dd717634f5a044f860435c1d8c16a270ddf0ef8588d4887037c5028b859b0c3",
                "sha256:8dec4936e9c3100156f8a2dc89c4b88d5c435175ff03413b443469c7c8c5f4d1",
                "sha256:97cafb1f3cbcd3fd2b6fbfb99ae11cdb14deea0736fc2b0952ee177f2b813a46",
                "sha256:a17a92de5231666cfbe003f0e4b9b3a7ae3afb1ec2845aadc2bacc93ff85febc",
                "sha256:a549b9c31bec33820e885335b451286e2969a2d9e24879f83fe904a5ce59d70a",
                "sha256:ac07bad82163452a6884fe8fa0963fb98c2346ba78d779ec06bd7a6262132aee",
                "sha256:ae2ad8ae6ebee9d2d94b17fb62763125f3f374c25618198f40cbb8b525411900",
                "sha256:b91c037585eba9095565a3556f611e3cbfaa42ca1e865f7b8015fe5c7336d5a5",
                "sha256:bc1667f8b83f48511b94671e0e441401371dfd0f0a795c7daa4a3cd1dde55bea",
                "sha256:bec0a414d016ac1a18862a519e54b2fd0fc8bbfd6890376898a6c0891dd82e9f",
                "sha256:bf50cd79a75d181c9181df03572cdce0fbb75cc353bc350712073108cba98de5",
                "sha256:bff1b4290a66b490a2f4719358c0cdcd9bafb6b8f061e45c7a2460866bf50c2e",
                "sha256:c061bb86a71b42465156a3ee7bd58c8c2ceacdbeb95d05a99893e08b8467359a",
                "sha256:c8b29db45f8fe46ad280a7294f5c3ec36dbac9491f2d1c17345be8e69cc5928f",
                "sha256:ce409136744f6521e39fd8e2a24c53fa18ad67aa5bc7c2cf83645cce5b5c4e50",
                "sha256:d050b3361367a06d752db6ead6e7edeb0009be66bc3bae0ee9d97fb326badc2a",
                "sha256:d283d37a890ba4c1ae73ffadf8046435c76e7bc2247bbb63c00bd1a709c6544b",
                "sha256:d9fad5155d72433c921b782e58892377c44bd6252b5af2f67f16b194987338a4",
                "sha256:daa4ee5a243f0f20d528d939d06670a298dd39b1ad5f8a72a4275124a7819eff",
                "sha256:db0b55e0f3cc0be60c1f19efdde9a637c32740486004f20d1cff53c3c0ece4d2",
                "sha256:e61659ba32cf2cf1481e575d0462554625196a1f2fc06a1c777d3f48e8865d46",
                "sha256:ea3d8a3d18833cf4304cd2fc9cbb1efe188ca9b5efef2bdac7adc20594a0e46b",
                "sha256:ec6a563cff360b50eed26f13adc43e61bc0c04d94b8be985e6fb24b81f6dcfdf",
                "sha256:f5dfb42c4604dddc8e4305050aa6deb084540643ed5804d7455b5df8fe16f5e5",
                "sha256:fa173ec60341d6bb97a89f5ea19c85c5643c1e7dedebc22f5181eb73573142c5",
                "sha256:fa9db3f79de01457b03d4f01b34cf91bc0048eb2c3846ff26f66687c2f6d16ab",
                "sha256:fce659a462a1be54d2ffcacea5e3ba2d74daa74f30f5f143fe0c58636e355fdd",
                "sha256:ffee1f21e5ef0d712f9033568f8344d5da8cc2869dbd08d87c84656e6a2d2f68"
            ],
            "markers": "python_version >= '3.7'",
            "version": "==2.1.5"
        },
        "sqlalchemy": {
            "hashes": [
                "sha256:1296f2cdd6db09b98ceb3c93025f0da4835303b8ac46c15c2136e27ee4d18d94",
                "sha256:1e135fff2e84103bc15c07edd8569612ce317d64bdb391f49ce57124a73f45c5",
                "sha256:1f8e1c6a6b7f8e9407ad9afc0ea41c1f65225ce505b79bc0342159de9c890782",
                "sha256:24bb0f81fbbb13d737b7f76d1821ec0b117ce8cbb8ee5e8641ad2de41aa916d3",
                "sha256:29d4247313abb2015f8979137fe65f4eaceead5247d39603cc4b4a610936cd2b",
                "sha256:2c286fab42e49db23c46ab02479f328b8bdb837d3e281cae546cc4085c83b680",
                "sha256:2f251af4c75a675ea42766880ff430ac33291c8d0057acca79710f9e5a77383d",
                "sha256:346ed50cb2c30f5d7a03d888e25744154ceac6f0e6e1ab3bc7b5b77138d37710",
                "sha256:3491c85df263a5c2157c594f54a1a9c72265b75d3777e61ee13c556d9e43ffc9",
                "sha256:427988398d2902de042093d17f2b9619a5ebc605bf6372f7d70e29bde6736842",
                "sha256:427c282dd0deba1f07bcbf499cbcc9fe9a626743f5d4989bfdfd3ed3513003dd",
                "sha256:49e3772eb3380ac88d35495843daf3c03f094b713e66c7d017e322144a5c6b7c",
                "sha256:4dae6001457d4497736e3bc422165f107ecdd70b0d651fab7f731276e8b9e12d",
                "sha256:5b5de6af8852500d01398f5047d62ca3431d1e29a331d0b56c3e14cb03f8094c",
                "sha256:5bbce5dd7c7735e01d24f5a60177f3e589078f83c8a29e124a6521b76d825b85",
                "sha256:5bed4f8c3b69779de9d99eb03fd9ab67a850d74ab0243d1be9d4080e77b6af12",
                "sha256:618827c1a1c243d2540314c6e100aee7af09a709bd005bae971686fab6723554",
                "sha256:6ab773f9ad848118df7a9bbabca53e3f1002387cdbb6ee81693db808b82aaab0",
                "sha256:6e41cb5cda641f3754568d2ed8962f772a7f2b59403b95c60c89f3e0bd25f15e",
                "sha256:7027be7930a90d18a386b25ee8af30514c61f3852c7268899f23fdfbd3107181",
                "sha256:763bd97c4ebc74136ecf3526b34808c58945023a59927b416acebcd68d1fc126",
                "sha256:7d0dbc56cb6af5088f3658982d3d8c1d6a82691f31f7b0da682c7b98fa914e91",
                "sha256:80e63bbdc5217dad3485059bdf6f65a7d43f33c8bde619df5c220edf03d87296",
                "sha256:80e7f697bccc56ac6eac9e2df5c98b47de57e7006d2e46e1a3c17c546254f6ef",
                "sha256:84e10772cfc333eb08d0b7ef808cd76e4a9a30a725fb62a0495877a57ee41d81",
                "sha256:853fcfd1f54224ea7aabcf34b227d2b64a08cbac116ecf376907968b29b8e763",
                "sha256:99224d621affbb3c1a4f72b631f8393045f4ce647dd3262f12fe3576918f8bf3",
                "sha256:a251146b921725547ea1735b060a11e1be705017b568c9f8067ca61e6ef85f20",
                "sha256:a551d5f3dc63f096ed41775ceec72fdf91462bb95abdc179010dc95a93957800",
                "sha256:a5d2e08d79f5bf250afb4a61426b41026e448da446b55e4770c2afdc1e200fce",
                "sha256:a752bff4796bf22803d052d4841ebc3c55c26fb65551f2c96e90ac7c62be763a",
                "sha256:afb1672b57f58c0318ad2cff80b384e816735ffc7e848d8aa51e0b0fc2f4b7bb",
                "sha256:bcdfb4b47fe04967669874fb1ce782a006756fdbebe7263f6a000e1db969120e",
                "sha256:bdb7b4d889631a3b2a81a3347c4c3f031812eb4adeaa3ee4e6b0d028ad1852b5",
                "sha256:c124912fd4e1bb9d1e7dc193ed482a9f812769cb1e69363ab68e01801e859821",
                "sha256:c294ae4e6bbd060dd79e2bd5bba8b6274d08ffd65b58d106394cb6abbf35cf45",
                "sha256:ca5ce82b11731492204cff8845c5e8ca1a4bd1ade85e3b8fcf86e7601bfc6a39",
                "sha256:cb8f9e4c4718f111d7b530c4e6fb4d28f9f110eb82e7961412955b3875b66de0",
                "sha256:d2de46f5d5396d5331127cfa71f837cca945f9a2b04f7cb5a01949cf676db7d1",
                "sha256:d913f8953e098ca931ad7f58797f91deed26b435ec3756478b75c608aa80d139",
                "sha256:de9acf369aaadb71a725b7e83a5ef40ca3de1cf4cdc93fa847df6b12d3cd924b",
                "sha256:e93983cc0d2edae253b3f2141b0a3fb07e41c76cd79c2ad743fc27eb79c3f6db",
                "sha256:f12aaf94f4d9679ca475975578739e12cc5b461172e04d66f7a3c39dd14ffc64",
                "sha256:f68016f9a5713684c1507cc37133c28035f29925c75c0df2f9d0f7571e23720a",
                "sha256:f7ea11727feb2861deaa293c7971a4df57ef1c90e42cb53f0da40c3468388000",
                "sha256:f98dbb8fcc6d1c03ae8ec735d3c62110949a3b8bc6e215053aa27096857afb45"
            ],
            "index": "pypi",
            "markers": "python_version >= '2.7' and python_version not in '3.0, 3.1, 3.2, 3.3, 3.4, 3.5'",
            "version": "==1.4.52"
        },
        "typing-extensions": {
            "hashes": [
                "sha256:69b1a937c3a517342112fb4c6df7e72fc39a38e7891a5730ed4985b5214b5475",
                "sha256:b0abd7c89e8fb96f98db18d86106ff1d90ab692004eb746cf6eda2682f91b3cb"
            ],
            "markers": "python_version >= '3.8'",
            "version": "==4.10.0"
        }
    },
    "develop": {}
}

[LucasMahieu]

If you are not happy with version 1.4.52
I would say that you need to restrict the version you want / need for sqlalchemy ?
for exemple : sqlalchemy >= "2.0.0" in your Pipefile.

And in that case, you will see the error that you expect.

But since you said "I need sqlalchemy, in what ever version". pipenv decides to downgrade (you don't express any constraint on that pkg).

From my point of view, it is not an issue at all.

[Kroppeb]

Technically this is indeed expected behavior.

I think it should be quite easy to add a warning whenever a dependency gets downgraded.

But I would also love (even more) a way to specify that I want to install a package with version ~=x.y where x.y.* is the latest non preview release, without having to look up the current version beforehand.

[matteius]

@Kroppeb I believe the version ~=x.y syntax/operator is already supported 🤔

[matteius]

I guess I don't agree with the expectation that "pipenv install dataset should error out, indicating that sqlalchemy is installed and at a later incompatible version." and let me explain why. The resolve gets all the constraints from the Pipfile and seeks to find packages that match them -- since you specified that any package version is acceptable, the pip resolver will find one that matches if one does match at all from the pool of all constraints.

[Kroppeb]

~= is indeed supported. But unless I'm mistaken, I need to lookup the current version on pypi.org (or any other relevant pypi server) to know what to put on the right hand side.

[vvaradarajan]

The issue is the user app is working fine until a new package is added using pipenv. Pipenv then decides to downgrade the sqlalchemy version to be compatible with the new package and now the user app doesn't work anymore. While this works as coded, this behavior is contrary to making development smooth. The '*' in the version should not be interpreted as being ok to downgrade an existing installed package (making the user app not work) Erroring out is a great option. If not a requirement that the user should force downgrade is another., or at least a warning.

…

On Wed, Mar 20, 2024, 3:13 PM Bashintosh ***@***.***> wrote: If you are not happy with version 1.4.52 I would say that you need to restrict the version you want / need for sqlalchemy ? for exemple : sqlalchemy >= "2.0.0" in your Pipefile. And in that case, you will see the error that you expect. But since you said "I need sqlalchemy, in what ever version". pipenv decides to downgrade (you don't express any constraint on that pkg). From my point of view, it is not an issue at all. — Reply to this email directly, view it on GitHub <#6104 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AFMJHPX347T2JO26EEWEZWLYZICXNAVCNFSM6AAAAABEF4MFVSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJQG4ZTOMBSHE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[matteius]

Feel free to work on a PR, but I don't think it should error out entirely unless possibly a flag is passed -- printing warnings should be fine. You might want to look into the upgrade command as well (if you find that is affected) as that is going to be the default behavior of install here in the next major release.

[matteius]

Should be checked against: #6282