Poetry not always find latest version from PyPi

[valbendan]

• I am on the latest Poetry version.
• I have searched the issues of this repo and believe that this is not a duplicate.
• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• OS version and name: macOS 11.1
• Poetry version: 1.1.4

I have no idae with how poetry cache or pypi index works, hence this may be a not bug :)

Issue

Let's say I have two project named p1, p2.

p1 is published on pypi and p2 dependent on p1.

When a new version released for p1(let's say v2.0), I update dependent p1 = ">=2.0,<3.0" to my p2 project.

Then poetry update will complain:

SolverProblemError

    Because p2 depends on p1 (>=2.0,<3.0) which doesn't match any versions, version solving failed.

but poetry run pip install p1==v2.0 works.

If wait some time (about 5 ~30 minutes) after I release p1, poetry update finally will works.

PS: I DO NOT use pypi mirror.

[jeanphilippemercier]

I faced the same issue. The only way I found to get it to work is to clear the PyPI cache:

poetry cache clear pypi --all

then run poetry update.

Resolving dependency can take several 100 seconds.

[BartlomiejSkwira]

I also had to update pip to get the latest Black 21.5b1

python -m pip install --upgrade pip

[mlissner]

I call this art piece, "A portrait of a sad programmer". Note that it includes time stamps of me trying to install version 0.9.13 of a dep I just released. TL;DR is that the answer above works, but arriving there is painful:

eyecite-o10j3129-py3.8) 08:55:44::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

No dependencies to install or update
(eyecite-o10j3129-py3.8) 08:55:54::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

No dependencies to install or update
(eyecite-o10j3129-py3.8) 08:56:03::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

No dependencies to install or update
(eyecite-o10j3129-py3.8) 08:56:06::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

No dependencies to install or update

Perhaps exiting the poetry shell will clear some cache?

(eyecite-o10j3129-py3.8) 08:56:19::mlissner@gabbro::~/Programming/eyecite 
↪ exit
exit
08:56:24::mlissner@gabbro::~/Programming/eyecite 
↪ poetry shell
Spawning shell within /home/mlissner/.cache/pypoetry/virtualenvs/eyecite-o10j3129-py3.8
. /home/mlissner/.cache/pypoetry/virtualenvs/eyecite-o10j3129-py3.8/bin/activate
08:56:26::mlissner@gabbro::~/Programming/eyecite 
↪ . /home/mlissner/.cache/pypoetry/virtualenvs/eyecite-o10j3129-py3.8/bin/activate
(eyecite-o10j3129-py3.8) 08:56:26::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

No dependencies to install or update
(eyecite-o10j3129-py3.8) 08:56:37::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

No dependencies to install or update

OK, I'll give it a few minutes:

(eyecite-o10j3129-py3.8) 09:03:04::mlissner@gabbro::~/Programming/eyecite 
↪ poetry remove courts-db
Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 0 installs, 0 updates, 1 removal

  • Removing courts-db (0.9.11)
(eyecite-o10j3129-py3.8) 09:03:18::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.11 for courts-db

Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing courts-db (0.9.11)

Arg. I guess I better google it. After finding this ticket:

(eyecite-o10j3129-py3.8) 09:03:23::mlissner@gabbro::~/Programming/eyecite 
↪ poetry cache clear pypi --all
Delete 886 entries? (yes/no) [no] yes
(eyecite-o10j3129-py3.8) 09:05:14::mlissner@gabbro::~/Programming/eyecite 
↪ poetry add courts-db@latest
Using version ^0.9.13 for courts-db

Updating dependencies
Resolving dependencies... (40.2s)

Writing lock file

Package operations: 0 installs, 1 update, 0 removals

  • Updating courts-db (0.9.11 -> 0.9.13)

Ahhhh.

[valbendan]

poetry cache clear pypi --all do the trick.

[radusuciu]

Also hit this today and the incantation posted above worked, thank you!

This happened to a package of mine that I had just updated - and published to pypi using poetry.

My poetry version is 1.1.6. I'll try and update and see if this still occurs when I release another update.

Edit: today I encountered the same issue with 1.1.6. Shortly thereafter I upgraded to 1.1.8, and the updated package was picked up.
Edit 2: I just pushed another version and poetry update again failed to find it, this time using 1.1.8. I suspect that in my previous edit the time delay between my attempt with 1.1.6 and then 1.1.8 was involved.

[kornpow]

Also had this problem today... I wish only deleting the cache would solve the problem, that would be understandable.

poetry remove {my-recently-updated-package}
poetry cache clear pypi --all
poetry add {my-recently-updated-package}@latest

[kakyoism]

poetry cache clear pypi --all do the trick.

This does not always work, either.

[kakyoism]

Also had this problem today... I wish only deleting the cache would solve the problem, that would be understandable.

poetry remove {my-recently-updated-package}
poetry cache clear pypi --all
poetry add {my-recently-updated-package}@latest

This seems to be the only way out.

[adriangb]

I've run into this as well. Cache seems to be too aggressive. Maybe failure to find a solution can trigger a cache refresh?

[rr-]

Just switched to poetry and I ran into this on my first day of using it. I agree that the cache should be wiped in case the package is not found.

[thclark]

My story

Yeah, I'm in total agreement that in the event of a solver error, poetry should rub cache and try again... however, I'm wondering if the actual issue is a bit more subtle - perhaps this points to a bug in the caching logic.

At the time of writing (6 May 2022), I'm experiencing this solver error (click to expand) when `sphinx-charts==0.1.2` package has been published since 12 Feb 2022.

 > [6/6] RUN poetry install --extras docs --no-ansi --no-interaction --no-root: 
#10 0.322 Skipping virtualenv creation, as specified in config file.            
#10 0.559 Installing dependencies from lock file                                
#10 0.768 Warning: The lock file is not up to date with the latest changes in pyproject.toml. You may be getting outdated dependencies. Run update to update them.
#10 0.833 
#10 0.833   SolverProblemError
#10 0.833 
#10 0.833   Because power-curve-utilities depends on sphinx-charts (0.1.2) which doesn't match any versions, version solving failed.

More to the point, I'm experiencing it while building a docker container. Why is that important? Because the starting image layer has no cache in the first place.

Here's the dockerfile (click to expand).

# This is basically the Microsoft vscode devcontainer for python, no poetry yet
FROM windpioneers/gdal-python:familiar-catshark-gdal-2.4.1-python-3.9-dev

# Switch to vscode user
USER vscode
WORKDIR /workspaces/power-curve-utilities

# Install poetry
RUN curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
ENV PATH "/home/vscode/.poetry/bin:$PATH"
RUN poetry config virtualenvs.create false

# Install python dependencies. Note that poetry installs any root packages by default,
# But this is not available at this stage of caching dependencies. So we do a dependency-only install here
# to cache the dependencies, then a full poetry install post-create to install any root packages.
COPY pyproject.toml poetry.lock ./
RUN poetry install --extras docs --no-ansi --no-interaction --no-root

It's the last RUN poetry install line that produces the solver error, But its a fresh install of poetry... it shouldn't have a cache at all.

I solved the issue by commenting out that last line, shelling into the container and doing as advised above:

poetry remove sphinx-charts                           
poetry cache clear pypi --all
poetry add sphinx-charts@latest

If I then uncomment the RUN poetry install line and rebuild the container again, it works fine. The significance of this is that the only things that could have changed in between failure and success are:

• any online cache or resources that got refreshed or updated (i.e. not my local cache, but something on a server)
• my pyproject.toml and poetry.lock files (copied in the prior image layer)

Hypothesis

I might be wrong, but I don't think anything online would have changed (e.g. the response from any query to pypi), because that last package was published months ago...

So I'm wondering if to set off this sequence of events, I had made some mistake (or there was some other problem) that put my lockfile out of sync with pyproject.toml... and whether that then gives a SolverError instead of barfing at me and saying Your poetry.lock file might be out of date, run 'poetry update sphinx-charts'.

I note that all of the solutions (eg from @kornpow and @kakyoism ) in this thread - with the exception of the trace provided by @mlissner - would also have refreshed any change in pyproject.lock, as well as refreshing the cache. @mlissner waited a few minutes, but his entire trace spanned <9 minutes and I don't know how long the cache expiry is in poetry but I bet it's at least 10 minutes, which would explain that experience.

(possible) Conclusion

My suspicion is that to solve this:

• if it's been only a few minutes since publishing the dependency, do a cache bust
• if it's been a long time and you get this error, do poetry update <problematic-library>
• perhaps @maintainers could provide a perspective, because maybe that SolverError could include a check that the dependency doesn't need updating? (maybe it does already and I missed it amid the volume of the logstream?)

[damascene]

I've just updated a library on pypi and tried to update it locally, but nothing except for cache clean solved the issue:
The following commands didn't work:
poetry update packagename
poetry update packagename@latest

After a cache clean and update it worked. hopefully this could be solved.

[tony]

• There's no way to wipe only that package's cache
• There's no way to override the cache backend when doing poetry update [package]

❯ poetry self --version
Poetry version 1.1.12

[tony]

Example with python 3.10.1 + poetry 1.1.12

https://pypi.org/project/libvcs/0.13.0a15/

pyproject.toml diff (bumped)

git diff pyproject.toml

diff --git a/pyproject.toml b/pyproject.toml
index eb1b789..117e030 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -57,7 +57,7 @@ vcspull = 'vcspull:cli.cli'
 python = "^3.9"
 click = "~8"
 kaptan = "*"
-libvcs = "~0.13.0a14"
+libvcs = "~0.13.0a15"
 colorama = ">=0.3.9"

 [tool.poetry.dev-dependencies]

poetry update libvcs

❯ poetry update libvcs
Updating dependencies
Resolving dependencies... (0.1s)

  SolverProblemError

  Because vcspull depends on libvcs (~0.13.0a15) which doesn't match any versions, version solving failed.

  at ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages                                                                                                                                                                                                238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

pip install -U libvcs --pre

❯ pip install -U libvcs --pre
Requirement already satisfied: libvcs in ./.venv/lib/python3.10/site-packages (0.13.0a14)
Collecting libvcs
  Downloading libvcs-0.13.0a15-py3-none-any.whl (41 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 41.6/41.6 KB 2.1 MB/s eta 0:00:00
Requirement already satisfied: typing-extensions<5.0.0,>=4.1.1 in ./.venv/lib/python3.10/site-packages (from libvcs) (4.2.0)                                                                                                               Installing collected packages: libvcs
  Attempting uninstall: libvcs
    Found existing installation: libvcs 0.13.0a14
    Uninstalling libvcs-0.13.0a14:
      Successfully uninstalled libvcs-0.13.0a14
Successfully installed libvcs-0.13.0a15

poetry update libvcs -vvv

Using virtualenv: /home/t/work/python/vcspull/.venv
Updating dependencies
Resolving dependencies...
   1: fact: vcspull is 1.13.0a4
   1: derived: vcspull
   1: fact: vcspull depends on click (~8)
   1: fact: vcspull depends on kaptan (*)
   1: fact: vcspull depends on libvcs (~0.13.0a15)
   1: fact: vcspull depends on colorama (>=0.3.9)
   1: fact: vcspull depends on sphinx (*)
   1: fact: vcspull depends on furo (^2022.2.23)
   1: fact: vcspull depends on sphinx-autoapi (*)
   1: fact: vcspull depends on sphinx-autobuild (^2021.3.14)
   1: fact: vcspull depends on sphinx-autodoc-typehints (~1.17.0)
   1: fact: vcspull depends on sphinx-click (*)
   1: fact: vcspull depends on sphinx-issues (^3.0.0)
   1: fact: vcspull depends on sphinx-inline-tabs (*)
   1: fact: vcspull depends on sphinxext-opengraph (*)
   1: fact: vcspull depends on sphinx-copybutton (^0.5.0)
   1: fact: vcspull depends on sphinxext-rediraffe (*)
   1: fact: vcspull depends on myst_parser (~0.17.0)
   1: fact: vcspull depends on pytest (*)
   1: fact: vcspull depends on pytest-rerunfailures (*)
   1: fact: vcspull depends on pytest-watcher (^0.2.3)
   1: fact: vcspull depends on codecov (*)
   1: fact: vcspull depends on coverage (*)
   1: fact: vcspull depends on pytest-cov (*)
   1: fact: vcspull depends on black (*)
   1: fact: vcspull depends on isort (*)
   1: fact: vcspull depends on flake8 (*)
   1: fact: vcspull depends on mypy (*)
   1: fact: vcspull depends on sphinx (*)
   1: fact: vcspull depends on furo (^2022.2.23)
   1: fact: vcspull depends on sphinx-autoapi (*)
   1: fact: vcspull depends on sphinx-autobuild (^2021.3.14)
   1: fact: vcspull depends on sphinx-autodoc-typehints (~1.17.0)
   1: fact: vcspull depends on sphinx-click (*)
   1: fact: vcspull depends on sphinx-issues (^3.0.0)
   1: fact: vcspull depends on sphinx-inline-tabs (*)
   1: fact: vcspull depends on sphinxext-opengraph (*)
   1: fact: vcspull depends on sphinx-copybutton (^0.5.0)
   1: fact: vcspull depends on sphinxext-rediraffe (*)
   1: fact: vcspull depends on myst_parser (~0.17.0)
   1: fact: vcspull depends on pytest (*)
   1: fact: vcspull depends on pytest-rerunfailures (*)
   1: fact: vcspull depends on pytest-watcher (^0.2.3)
   1: fact: vcspull depends on codecov (*)
   1: fact: vcspull depends on coverage (*)
   1: fact: vcspull depends on pytest-cov (*)
   1: fact: vcspull depends on black (*)
   1: fact: vcspull depends on isort (*)
   1: fact: vcspull depends on flake8 (*)
   1: fact: vcspull depends on mypy (*)
   1: selecting vcspull (1.13.0a4)
   1: derived: mypy
   1: derived: flake8
   1: derived: isort
   1: derived: black
   1: derived: pytest-cov
   1: derived: coverage
   1: derived: codecov
   1: derived: pytest-watcher (>=0.2.3,<0.3.0)
   1: derived: pytest-rerunfailures
   1: derived: pytest
   1: derived: myst_parser (>=0.17.0,<0.18.0)
   1: derived: sphinxext-rediraffe
   1: derived: sphinx-copybutton (>=0.5.0,<0.6.0)
   1: derived: sphinxext-opengraph
   1: derived: sphinx-inline-tabs
   1: derived: sphinx-issues (>=3.0.0,<4.0.0)
   1: derived: sphinx-click
   1: derived: sphinx-autodoc-typehints (>=1.17.0,<1.18.0)
   1: derived: sphinx-autobuild (>=2021.3.14,<2022.0.0)
   1: derived: sphinx-autoapi
   1: derived: furo (>=2022.2.23,<2023.0.0)
   1: derived: sphinx
   1: derived: colorama (>=0.3.9)
   1: derived: libvcs (>=0.13.0a15,<0.14.0)
   1: derived: kaptan
   1: derived: click (>=8,<9)
   1: fact: mypy (0.960) depends on typing-extensions (>=3.10)
   1: fact: mypy (0.960) depends on mypy-extensions (>=0.4.3)
   1: fact: mypy (0.960) depends on tomli (>=1.1.0)
   1: selecting mypy (0.960)
   1: derived: tomli (>=1.1.0)
   1: derived: mypy-extensions (>=0.4.3)
   1: derived: typing-extensions (>=3.10)
   1: fact: flake8 (4.0.1) depends on mccabe (>=0.6.0,<0.7.0)
   1: fact: flake8 (4.0.1) depends on pycodestyle (>=2.8.0,<2.9.0)
   1: fact: flake8 (4.0.1) depends on pyflakes (>=2.4.0,<2.5.0)
   1: selecting flake8 (4.0.1)
   1: derived: pyflakes (>=2.4.0,<2.5.0)
   1: derived: pycodestyle (>=2.8.0,<2.9.0)
   1: derived: mccabe (>=0.6.0,<0.7.0)
   1: selecting isort (5.10.1)
   1: fact: black (22.3.0) depends on click (>=8.0.0)
   1: fact: black (22.3.0) depends on platformdirs (>=2)
   1: fact: black (22.3.0) depends on pathspec (>=0.9.0)
   1: fact: black (22.3.0) depends on mypy-extensions (>=0.4.3)
   1: fact: black (22.3.0) depends on typing-extensions (>=3.10.0.0)
   1: fact: black (22.3.0) depends on tomli (>=1.1.0)
   1: selecting black (22.3.0)
   1: derived: pathspec (>=0.9.0)
   1: derived: platformdirs (>=2)
   1: fact: pytest-cov (3.0.0) depends on pytest (>=4.6)
   1: fact: pytest-cov (3.0.0) depends on coverage (>=5.2.1)
   1: selecting pytest-cov (3.0.0)
   1: derived: coverage[toml] (>=5.2.1)
   1: derived: pytest (>=4.6)
PyPI: No release information found for coverage-2.5, skipping
PyPI: No release information found for coverage-2.6, skipping
PyPI: No release information found for coverage-2.75, skipping
PyPI: No release information found for coverage-2.76, skipping
PyPI: No release information found for coverage-2.77, skipping
PyPI: No release information found for coverage-2.78, skipping
PyPI: No release information found for coverage-2.8, skipping
PyPI: No release information found for coverage-2.80, skipping
PyPI: No release information found for coverage-2.85, skipping
PyPI: 17 packages found for coverage >=5.2.1
   1: selecting coverage (6.4)
   1: fact: codecov (2.1.12) depends on requests (>=2.7.9)
   1: fact: codecov (2.1.12) depends on coverage (*)
   1: selecting codecov (2.1.12)
   1: derived: requests (>=2.7.9)
   1: fact: pytest-watcher (0.2.3) depends on watchdog (>=2.0.0)
   1: selecting pytest-watcher (0.2.3)
   1: derived: watchdog (>=2.0.0)
   1: fact: pytest-rerunfailures (10.2) depends on pytest (>=5.3)
   1: selecting pytest-rerunfailures (10.2)
   1: derived: pytest (>=5.3)
   1: fact: pytest (7.1.2) depends on attrs (>=19.2.0)
   1: fact: pytest (7.1.2) depends on iniconfig (*)
   1: fact: pytest (7.1.2) depends on packaging (*)
   1: fact: pytest (7.1.2) depends on pluggy (>=0.12,<2.0)
   1: fact: pytest (7.1.2) depends on py (>=1.8.2)
   1: fact: pytest (7.1.2) depends on tomli (>=1.0.0)
   1: fact: pytest (7.1.2) depends on atomicwrites (>=1.0)
   1: fact: pytest (7.1.2) depends on colorama (*)
   1: selecting pytest (7.1.2)
   1: derived: atomicwrites (>=1.0)
   1: derived: py (>=1.8.2)
   1: derived: pluggy (>=0.12,<2.0)
   1: derived: packaging
   1: derived: iniconfig
   1: derived: attrs (>=19.2.0)
   1: fact: myst-parser (0.17.2) depends on docutils (>=0.15,<0.18)
   1: fact: myst-parser (0.17.2) depends on jinja2 (*)
   1: fact: myst-parser (0.17.2) depends on markdown-it-py (>=1.0.0,<3.0.0)
   1: fact: myst-parser (0.17.2) depends on mdit-py-plugins (>=0.3.0,<0.4.0)
   1: fact: myst-parser (0.17.2) depends on pyyaml (*)
   1: fact: myst-parser (0.17.2) depends on sphinx (>=3.1,<5)
   1: fact: myst-parser (0.17.2) depends on typing-extensions (*)
   1: selecting myst-parser (0.17.2)
   1: derived: sphinx (>=3.1,<5)
   1: derived: pyyaml
   1: derived: mdit-py-plugins (>=0.3.0,<0.4.0)
   1: derived: markdown-it-py (>=1.0.0,<3.0.0)
   1: derived: jinja2
   1: derived: docutils (>=0.15,<0.18)
   1: fact: sphinxext-rediraffe (0.2.7) depends on sphinx (>=2.0)
   1: selecting sphinxext-rediraffe (0.2.7)
   1: fact: sphinx-copybutton (0.5.0) depends on sphinx (>=1.8)
   1: selecting sphinx-copybutton (0.5.0)
   1: fact: sphinxext-opengraph (0.6.3) depends on sphinx (>=2.0)
   1: selecting sphinxext-opengraph (0.6.3)
   1: fact: sphinx-inline-tabs (2022.1.2b11) depends on sphinx (>=3)
   1: selecting sphinx-inline-tabs (2022.1.2b11)
   1: fact: sphinx-issues (3.0.1) depends on sphinx (*)
   1: selecting sphinx-issues (3.0.1)
   1: fact: sphinx-click (4.0.3) depends on sphinx (>=2.0)
   1: fact: sphinx-click (4.0.3) depends on click (>=7.0)
   1: fact: sphinx-click (4.0.3) depends on docutils (*)
   1: selecting sphinx-click (4.0.3)
   1: fact: sphinx-autodoc-typehints (1.17.1) depends on Sphinx (>=4)
   1: selecting sphinx-autodoc-typehints (1.17.1)
   1: derived: Sphinx (>=4)
   1: fact: sphinx-autobuild (2021.3.14) depends on sphinx (*)
   1: fact: sphinx-autobuild (2021.3.14) depends on livereload (*)
   1: fact: sphinx-autobuild (2021.3.14) depends on colorama (*)
   1: selecting sphinx-autobuild (2021.3.14)
   1: derived: livereload
   1: fact: sphinx-autoapi (1.8.4) depends on astroid (>=2.7)
   1: fact: sphinx-autoapi (1.8.4) depends on Jinja2 (*)
   1: fact: sphinx-autoapi (1.8.4) depends on PyYAML (*)
   1: fact: sphinx-autoapi (1.8.4) depends on sphinx (>=3.0)
   1: fact: sphinx-autoapi (1.8.4) depends on unidecode (*)
   1: selecting sphinx-autoapi (1.8.4)
   1: derived: unidecode
   1: derived: astroid (>=2.7)
   1: fact: furo (2022.4.7) depends on beautifulsoup4 (*)
   1: fact: furo (2022.4.7) depends on sphinx (>=4.0,<5.0)
   1: fact: furo (2022.4.7) depends on pygments (>=2.7,<3.0)
   1: selecting furo (2022.4.7)
   1: derived: pygments (>=2.7,<3.0)
   1: derived: beautifulsoup4
   1: fact: sphinx (4.5.0) depends on sphinxcontrib-applehelp (*)
   1: fact: sphinx (4.5.0) depends on sphinxcontrib-devhelp (*)
   1: fact: sphinx (4.5.0) depends on sphinxcontrib-jsmath (*)
   1: fact: sphinx (4.5.0) depends on sphinxcontrib-htmlhelp (>=2.0.0)
   1: fact: sphinx (4.5.0) depends on sphinxcontrib-serializinghtml (>=1.1.5)
   1: fact: sphinx (4.5.0) depends on sphinxcontrib-qthelp (*)
   1: fact: sphinx (4.5.0) depends on Jinja2 (>=2.3)
   1: fact: sphinx (4.5.0) depends on Pygments (>=2.0)
   1: fact: sphinx (4.5.0) depends on docutils (>=0.14,<0.18)
   1: fact: sphinx (4.5.0) depends on snowballstemmer (>=1.1)
   1: fact: sphinx (4.5.0) depends on babel (>=1.3)
   1: fact: sphinx (4.5.0) depends on alabaster (>=0.7,<0.8)
   1: fact: sphinx (4.5.0) depends on imagesize (*)
   1: fact: sphinx (4.5.0) depends on requests (>=2.5.0)
   1: fact: sphinx (4.5.0) depends on packaging (*)
   1: fact: sphinx (4.5.0) depends on importlib-metadata (>=4.4)
   1: fact: sphinx (4.5.0) depends on colorama (>=0.3.5)
   1: selecting sphinx (4.5.0)
   1: derived: importlib-metadata (>=4.4)
   1: derived: imagesize
   1: derived: alabaster (>=0.7,<0.8)
   1: derived: babel (>=1.3)
   1: derived: snowballstemmer (>=1.1)
   1: derived: Jinja2 (>=2.3)
   1: derived: sphinxcontrib-qthelp
   1: derived: sphinxcontrib-serializinghtml (>=1.1.5)
   1: derived: sphinxcontrib-htmlhelp (>=2.0.0)
   1: derived: sphinxcontrib-jsmath
   1: derived: sphinxcontrib-devhelp
   1: derived: sphinxcontrib-applehelp
   1: selecting colorama (0.4.4)
PyPI: No release information found for libvcs-0.1.2, skipping
PyPI: 0 packages found for libvcs >=0.13.0a15,<0.14.0
   1: fact: no versions of libvcs match >=0.13.0a15,<0.14.0
   1: conflict: no versions of libvcs match >=0.13.0a15,<0.14.0
   1: !  libvcs (~0.13.0a15) is satisfied by  libvcs (~0.13.0a15)
   1: ! which is caused by "vcspull depends on libvcs (~0.13.0a15)"
   1: ! thus: version solving failed
   1: Version solving took 0.118 seconds.
   1: Tried 1 solutions.

  Stack trace:

  8  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/clikit/console_application.py:131 in run
      129│             parsed_args = resolved_command.args
      130│ 
    → 131│             status_code = command.handle(parsed_args, io)
      132│         except KeyboardInterrupt:
      133│             status_code = 1

  7  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/clikit/api/command/command.py:120 in handle
      118│     def handle(self, args, io):  # type: (Args, IO) -> int
      119│         try:
    → 120│             status_code = self._do_handle(args, io)
      121│         except KeyboardInterrupt:
      122│             if io.is_debug():

  6  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/clikit/api/command/command.py:171 in _do_handle
      169│         handler_method = self._config.handler_method
      170│ 
    → 171│         return getattr(handler, handler_method)(args, io, self)
      172│ 
      173│     def __repr__(self):  # type: () -> str

  5  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/cleo/commands/command.py:92 in wrap_handle
       90│         self._command = command
       91│ 
    →  92│         return self.handle()
       93│ 
       94│     def handle(self):  # type: () -> Optional[int]

  4  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/poetry/console/commands/update.py:47 in handle
      45│         self._installer.update(True)
      46│ 
    → 47│         return self._installer.run()
      48│ 

  3  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/poetry/installation/installer.py:103 in run
      101│         local_repo = Repository()
      102│ 
    → 103│         return self._do_install(local_repo)
      104│ 
      105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  2  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/poetry/installation/installer.py:235 in _do_install
      233│             )
      234│ 
    → 235│             ops = solver.solve(use_latest=self._whitelist)
      236│         else:
      237│             self._io.write_line("Installing dependencies from lock file")

  1  ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/poetry/puzzle/solver.py:65 in solve
       63│         with self._provider.progress():
       64│             start = time.time()
    →  65│             packages, depths = self._solve(use_latest=use_latest)
       66│             end = time.time()
       67│ 

  SolverProblemError

  Because vcspull depends on libvcs (~0.13.0a15) which doesn't match any versions, version solving failed.

  at ~/.config/.asdf/installs/poetry/1.1.12/venv/lib/python3.7/site-packages/poetry/puzzle/solver.py:241 in _solve
      237│             packages = result.packages
      238│         except OverrideNeeded as e:
      239│             return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
      240│         except SolveFailure as e:
    → 241│             raise SolverProblemError(e)
      242│ 
      243│         results = dict(
      244│             depth_first_search(
      245│                 PackageNode(self._package, packages), aggregate_package_nodes

[pbarker]

This is pretty rough, would be nice if there was a --no-cache flag

[tony]

@pbarker Agreed.

@python-poetry: Is adding a --no-cache flag to 1.2 a viable thing? (I assume this would be for poetry update and poetry add. This would save having to wipe the whole cache just because of a stuck package.

[mkniewallner]

@pbarker Agreed.

@python-poetry: Is adding a --no-cache flag to 1.2 a viable thing? (I assume this would be for poetry update and poetry add. This would save having to wipe the whole cache just because of a stuck package.

AFAIK this was added in #5519 which got released in 1.2.0b2.

[spietras]

I also have stumbled upon this issue. A couple of hours passed since I released a new version of a package on PyPI and still, poetry can't find it. No matter how many reinstalls or cache cleans I do, it just can't be found. pip alone can successfully find the correct version. After smashing my head against the wall several times, I tried to see what poetry does under the hood. After a little investigation, I found that it calls https://pypi.org/pypi/{package}/json to get info about the package, including the latest version. So I tried to call it myself manually, and to my surprise and a slight sigh of relief, it turns out that the PyPI API is returning the old version. I guess I need to wait a little more for PyPI to update its state.

Anyway, the point is: sometimes it's not poetry caching issue, but PyPI or some network cache is at fault and I guess you can't really do anything but wait.

However, pip could find the correct version, so there is probably another way to get that info about the package. Maybe some other API endpoint that for some reason got updated quicker or has not been cached by the network before.

[neersighted]

Poetry requires rich metadata for each package; to get that we use the PyPI JSON API instead of the Simple HTML API (which would require us to download each package to extract metadata) -- what you're running into is the Fastly CDN layer that caches the JSON API.

If it takes more than a few hours for your package to be visible through the JSON API, I'd suggest reaching out to the warehouse team by opening an issue on https://github.com/pypi/warehouse

[jedie]

I have the same troubles, see: #4438 (comment)

And yes, it seems that the real problem is not up-to-date JSON API in PyPi -> pypi/warehouse#11939

But poetry can add a fallback to the old "Simple HTML API" if the version is not found in PyPi JSON until pypi/warehouse#11939 is fixed?!?

[chrisbc]

@neersighted I notice the poetry search command DOES find the correct/updated version , so it looks like poetry might already have some way to identify when the pypi / package repo API is out-of-sync.

to get that we use the PyPI JSON API instead of the Simple HTML API (which would require us to download each package to extract metadata) -- what you're running into is the Fastly CDN layer that caches the JSON API.

But in the meantime the user experience is that pip still works and poetry doesn't ... which is far from ideal. I'm not sure that it's just a CDN issue as the pypi/warehouse#11939 shows us that the JSON API is up to date for the pypi/{NAME}/{VERSION}/json endpoint, but out of date at the /pypi/NAME/json endpoint.

Maybe a hybrid approach could work - using the "old Simple HTML API" to find the up-to-date versions, then the JSON API to get the metadata. Ugly yes, fragile, maybe but still much better downloading the packages and a quick fix for those in need? No word from the pypi/warehouse team about a fix or timeframe AFAIK?

[neersighted]

I'm really not sure how I feel about checking both APIs to see if they agree, or how we would express that to the user. Is the idea that during poetry update or similar, we would print a warning if the PyPI APIs do not match?

Seems like a fairly awful workaround (as in, the user can't do anything and it would be very invasive to make it consistent) for a known Warehouse issue with a (hopefully) finite shelf life...

I'd love to hear more from users if people are getting bit by this frequently.

[neersighted]

Also after doing some more digging, it looks like the Simple API is suffering from this too, but less frequently -- it does look like it's simply a CDN issue, given some of the upstream debugging and pypi/warehouse#11935.

[chrisbc]

@neersighted yeah I agree it's pretty ugly , but i thought I'd mention it anyway in case things get desperate. My project 'came right' after 24 hours or so. I've no idea if this was a simple timed retry thing, or a fix, or even an ops intervention as there's nothing on the ticket.

[di]

This should now be resolved.

[neersighted]

This should now be resolved.

At the server level, or should the stale JSON also be purged from Fastly?

[di]

Entirely resolved. We have issued purges for all affected responses.

[neersighted]

I'm going to close this issue as we chose it as the grab bag for this round of issues. Next time Poetry is reporting inconsistent metadata vs pip we can open a new one. The original issue seems to have been a solver issue, but the OP long since went silent so I'm going to ask them to open a new issue if they encounter it again.

[kornpow]

Idk if we are still waiting for a push from Pypi, but it doesnt seem to be solved for me.

I still can push package1, and then try to update it as a dependancy in package2, and it still says that no updates are available. Confirmed that the pypi simple vs json interfaces are out of sync.

[dimbleby]

Confirmed that the pypi simple vs json interfaces are out of sync.

You'll want to raise that with pypi, not here.

[kornpow]

Will do. The only reason why I bring it up here is @dl said everything is fixed, in this thread, and I confirmed everything is not fixed.

[di]

Looks like we overlooked projects which had old releases but new files added after the bug was introduced. We're looking into purging these as well.

[mstislavk]

I found a workaround.
TL;DR: you need to remove a file from poetry cache with information about a library which you want to update/install.

Solution:

1. grep -lr "<library-name>" ~/.cache/pypoetry/cache/*
2. rm <file-that-was-found-on-2nd-step>

JFYI, these files look like this:

Filename: repositories/pypi/6e/70/20/17/27/f4/90/f4/6e70201727f490f47becc790837f1904ab3c4fdc4df6ae1d34eb6ab3ff836734

Content:
9999999999{"name": "openmetrics-liveness-probe", "version": "0.1.1", "summary": "", "platform": null, "requires_dist": ["pydantic (>=1.9.1,<2.0.0)", "prometheus-client (>=0.14.1,<0.15.0)"], "requires_python": ">=3.6,<4.0", "files": [], "_cache_version": "1.0.0"}

[cbenz]

For me the solution was to list all the caches, then clear them all:

poetry cache list
poetry cache clear pypi --all
poetry cache clear PyPI --all
poetry cache clear _default_cache --all

Yes, there was pypi and PyPI. I deleted the (now empty) directory ~/.cache/pypoetry/cache/repositories/pypi because it seemed to be never filled-in from now.

[neersighted]

There was a change in naming across 1.1 and 1.2. If you want to be aware of data that is not cached you need to clear it. This only presents when a maintainer uploads new wheels after you have first solved for a package.

[2lu3]

poetry update ${package name} --no-cache worked for me.

$ poetry self --version
Poetry (version 1.3.1)

[LinuxIsCool]

poetry update ${package name} --no-cache worked for me.

$ poetry self --version
Poetry (version 1.3.1)

This is the only thing that worked for me. But takes about 5 minutes. Looks like this :

Poetry (version 1.4.0)

[AKuederle]

It seems like this issue still persists... Pip can resolve the package, but poetry can not find it.

Just removing the cache did not work. Only removing the package, cleaning the cache and reinstalling the package allowed poetry to find the correct version.

[lu-pl]

Clearing the cache did not work in my case, but poetry update <package> --no-chache did. Just if anyone runs into the same problem.

[Ersain]

Clearing the cache did not work in my case, but poetry update <package> --no-chache did. Just if anyone runs into the same problem.

+1, also works with poetry add <package> --no-cache

[xLaszlo]

poetry cache clear pypi --all

No cache entries for pypi

poetry cache list

PyPI
_default_cache

poetry cache clear PyPI --all

Delete 874 entries? (yes/no) [yes]

[earonesty]

weird this ticket is closed. the ONLY decent workaround (aside from wiping the entire cache, which is absurd for a large project), is to use a venv. then you can pip install the package, then you can poetry update it... and everything works. the --no-cache operation can take 20 minutes for me, so it's a non-starter.

[davide445]

Due I also got this problem and read this issue, I followed poetry suggestion and for me worked

poetry add <package>@latest

[sassanh]

is to use a venv

Thanks! I sometimes needed to wait 10-20 minutes to be able to lock.
Good news is you don't need a venv, you can run poetry run pip install <package>==<version> and then poetry lock and poetry update will work without needing to wait.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.