Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

publish to devpi #726

Closed
3 tasks done
xsteadfastx opened this issue Dec 11, 2018 · 14 comments
Closed
3 tasks done

publish to devpi #726

xsteadfastx opened this issue Dec 11, 2018 · 14 comments
Labels
area/build-system Related to PEP 517 packaging (see poetry-core) kind/bug Something isn't working as expected

Comments

@xsteadfastx
Copy link

  • I am on the latest Poetry version.
  • I have searched the issues of this repo and believe that this is not a duplicate.
  • If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).
  • OS version and name: Ubuntu Bionic Beaver
  • Poetry version: 0.12.10

Issue

i try to publish to my local devpi server. i added it like this:

poetry config repositories.foo http://localhost:3141/root/dev/

when i try to publish i get his:

poetry publish -r foo -u root -p mypassword --build -vvv                                                                                         12.2s  Di 11 Dez 2018 10:00:55 CET

There are 2 files ready for publishing. Build anyway? (yes/no) [no] yes

Virtualenv my-python-project-py3.7 already exists.
Using virtualenv: /home/preuss/.cache/pypoetry/virtualenvs/my-python-project-py3.7
Building my_python_project (0.0.4)
 - Building sdist
 - Adding: src/my_python_project/__init__.py
 - Adding: src/my_python_project/cli.py
 - Adding: src/my_python_project/core.py
 - Adding: pyproject.toml
 - Adding: README.md
 - Built my_python_project-0.0.4.tar.gz

 - Building wheel
 - Adding: /tmp/tmpcnft47s2/my_python_project-0.0.4/src/my_python_project/__init__.py
 - Adding: /tmp/tmpcnft47s2/my_python_project-0.0.4/src/my_python_project/cli.py
 - Adding: /tmp/tmpcnft47s2/my_python_project-0.0.4/src/my_python_project/core.py
 - Built my_python_project-0.0.4-py3-none-any.whl

Publishing my_python_project (0.0.4) to foo

[RuntimeError]
"my-python-project-0.0.4.tar.gz" does not exist.

Exception trace:
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/console/commands/command.py in run() at line 77
   return super(BaseCommand, self).run(i, o)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/console/commands/publish.py in handle() at line 54
   self.option("repository"), self.option("username"), self.option("password")
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/publisher.py in publish() at line 84
   return self._uploader.upload(url)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/uploader.py in upload() at line 97
   self._upload(session, url)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/uploader.py in _upload() at line 193
   self._register(session, url)
 /home/preuss/.pyenv/versions/3.7.1/lib/python3.7/site-packages/poetry/masonry/publishing/uploader.py in _register() at line 258
   raise RuntimeError('"{0}" does not exist.'.format(file.name))

publish [-r|--repository REPOSITORY] [-u|--username USERNAME] [-p|--password PASSWORD] [--build]

the devpi logs this:

2018-12-11 09:01:01,089 INFO  [req44] POST /root/dev/
2018-12-11 09:01:01,193 ERROR [req44] [Wtx6] while handling http://localhost:3141/root/dev/:
no project named 'my-python-project' was ever registered

@xsteadfastx
Copy link
Author

i investigated a little more. it turns out there is a problem with underscores. if my project is named without underscores, everything gets published well.

1 similar comment
@xsteadfastx
Copy link
Author

i investigated a little more. it turns out there is a problem with underscores. if my project is named without underscores, everything gets published well.

@sdispater sdispater added kind/bug Something isn't working as expected area/build-system Related to PEP 517 packaging (see poetry-core) labels Dec 13, 2018
@vlcinsky
Copy link
Contributor

I have very similar problem with publishing package named ttr.run.

Full test of publishing a package to devpi-server

Before experimenting, login to devpi-server:

$ devpi login javl
...

Make sure the package ttr.run is not present in the devpi index

$ devpi remove ttr.run
*redirected: http://127.0.0.1:3141/javl/dev/ttr-run
GET http://127.0.0.1:3141/javl/dev/ttr-run
404 Not Found: no project 'ttr-run'

This is fine, it is not there.

Build and try to publish by poetry

Build the package by poetry:

$ poetry build

and publish it by poetry:

$ poetry publish -r local -u javl -p verysecreteone

Publishing ttr.run (0.3.0) to local
 - Uploading ttr.run-0.3.0-py3-none-any.whl 100%
 - Uploading ttr.run-0.3.0.tar.gz 100%

According to responses, one would call it a success

Check the result (of poetry publishing)

devpi-server logs

Let's check devpi-server logs

$ devpi-server --log
......

show records (following deletion):

2019-04-12 10:57:23,110 INFO  [req114] GET /javl/dev/ttr.run/
2019-04-12 10:57:23,115 INFO  [req115] GET /javl/dev/ttr-run
2019-04-12 10:57:34,283 INFO  [req116] POST /javl/dev/+simple/
2019-04-12 10:57:34,284 INFO  [req116] starting +simple
2019-04-12 10:57:34,679 INFO  [req117] POST /javl/dev/+simple/
2019-04-12 10:57:34,680 INFO  [req117] starting +simple
2019-04-12 10:58:33,919 INFO  [req118] GET /javl/dev/ttr.run/
2019-04-12 10:58:33,929 INFO  [req119] GET /javl/dev/ttr-run
2019-04-12 10:58:50,417 INFO  [req120] POST /javl/dev/+simple/
2019-04-12 10:58:50,418 INFO  [req120] starting +simple
2019-04-12 10:58:50,803 INFO  [req121] POST /javl/dev/+simple/
2019-04-12 10:58:50,804 INFO  [req121] starting +simple

All looks fine.

devpi client listing packages (package missing)

Now use devpi client to check, if the packages are present in the index:

$ devpi list ttr.run
*redirected: http://127.0.0.1:3141/javl/dev/ttr-run
GET http://127.0.0.1:3141/javl/dev/ttr-run
404 Not Found: no project 'ttr-run'

Upload packages by devpi client (works)

$ cd dist
$ devpi upload ttr.run-0.3.0*
register ttr.run-0.3.0 to http://127.0.0.1:3141/javl/dev/
file_upload of ttr.run-0.3.0-py3-none-any.whl to http://127.0.0.1:3141/javl/dev/
register ttr.run-0.3.0 to http://127.0.0.1:3141/javl/dev/
file_upload of ttr.run-0.3.0.tar.gz to http://127.0.0.1:3141/javl/dev/

and check, if it is present

$ devpi list ttr.run
*redirected: http://127.0.0.1:3141/javl/dev/ttr-run
http://127.0.0.1:3141/javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl
http://127.0.0.1:3141/javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz

Success in devpi-server logs

How does the success looks like in devpi-server logs:

Upload records:

2019-04-12 11:13:18,473 INFO  [req127] POST /javl/dev/
2019-04-12 11:13:18,475 INFO  [req127] [Wtx2564] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,476 INFO  [req127] javl/dev: got submit release info 'ttr.run'
2019-04-12 11:13:18,483 INFO  [req127] [Wtx2564] fswriter2565: committed: keys: 'javl/dev/ttr-run/.versions','javl/dev/ttr-run/0.3.0/.config','javl/dev/.projects'
2019-04-12 11:13:18,499 INFO  [req128] POST /javl/dev/
2019-04-12 11:13:18,573 INFO  [req128] [Wtx2565] added 'releasefile' link javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl
2019-04-12 11:13:18,574 INFO  [req128] [Wtx2565] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,590 INFO  [NOTI] [Rtx2565] Committing 6 new documents to search index.
2019-04-12 11:13:18,598 INFO  [req128] [Wtx2565] fswriter2566: committed: keys: 'javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl','javl/dev/ttr-run/.simple','javl/dev/ttr-run/0.3.0/.config', files_commit: +files/javl/dev/+f/64f/72bb608ab8fbd/ttr.run-0.3.0-py3-none-any.whl
2019-04-12 11:13:18,603 INFO  [req129] POST /javl/dev/
2019-04-12 11:13:18,606 INFO  [req129] [Wtx2566] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,606 INFO  [req129] javl/dev: got submit release info 'ttr.run'
2019-04-12 11:13:18,614 INFO  [req129] [Wtx2566] fswriter2567: committed: keys: 'javl/dev/ttr-run/0.3.0/.config'
2019-04-12 11:13:18,619 INFO  [req130] POST /javl/dev/
2019-04-12 11:13:18,626 INFO  [req130] [Wtx2567] added 'releasefile' link javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz
2019-04-12 11:13:18,626 INFO  [req130] [Wtx2567] set_metadata ttr-run-0.3.0
2019-04-12 11:13:18,638 INFO  [req130] [Wtx2567] fswriter2568: committed: keys: 'javl/dev/ttr-run/.simple','javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz','javl/dev/ttr-run/0.3.0/.config', files_commit: +files/javl/dev/+f/c69/5743f17f1b724/ttr.run-0.3.0.tar.gz
2019-04-12 11:13:18,685 INFO  [NOTI] [Rtx2565] Finished committing 6 documents to search index.
2019-04-12 11:13:18,796 INFO  [NOTI] [Rtx2566] Committing 6 new documents to search index.
2019-04-12 11:13:18,849 INFO  [NOTI] [Rtx2566] Finished committing 6 documents to search index.
2019-04-12 11:13:18,937 INFO  [NOTI] [Rtx2567] Committing 5 new documents to search index.
2019-04-12 11:13:19,159 INFO  [NOTI] [Rtx2567] Finished committing 5 documents to search index.
2019-04-12 11:13:19,243 INFO  [NOTI] [Rtx2568] Committing 5 new documents to search index.
2019-04-12 11:13:19,260 INFO  [NOTI] [Rtx2568] Finished committing 5 documents to search index.

and traces of listing:

2019-04-12 11:13:24,743 INFO  [req131] GET /javl/dev/ttr.run/
2019-04-12 11:13:24,747 INFO  [req132] GET /javl/dev/ttr-run

Versions

Regarding version:

$ devpi --version
devpi-client 4.3.0

current devpi server: http://127.0.0.1:3141/
    devpi-server 4.8.1
    devpi-web 3.5.0
$ poetry --version
Poetry 0.12.11

Conclusions

  • devpi-server logs differ for publishing via poetry publish (much simpler) and by devpi client.
  • apparently, there is something related to naming packages, see ttr.run being redirected to ttr-run in devpi-server logs
  • naming issues do not relate only to underscores, but also to dots in package names.
  • the issue could be with poetry or with devpi-server, next action on resolving this issue would be to investigate this.
  • there is an workaround: build package by poetry and then use devpi client to upload it

@hoefling
Copy link
Contributor

hoefling commented Sep 6, 2019

Since version 5.0, devpi doesn't require package registration anymore, so poetry publish should work once you update your devpi-server installation.

@stinovlas
Copy link

It seems that I'm encountering the same problem with devpi-server==5.1.0. And my package doesn't even contain any dashes or underscores in its name.

@hoefling
Copy link
Contributor

@stinovlas I can't confirm that with devpi-server==5.1.0. What's your server log on poetry publish? Here's a recent example from one of mine instances, publishing .tar.gz, .whl and docs:

2019-09-23 16:38:16,273 INFO  [req25421] POST /company/dev/
2019-09-23 16:38:16,393 INFO  [req25421] [Wtx4429] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:16,393 INFO  [req25421] company/dev: got submit release info 'samplepkg-sphinxcontrib'
2019-09-23 16:38:16,395 INFO  [req25421] [Wtx4429] added 'releasefile' link company/dev/+f/ec1/f13e044d8339a/samplepkg-sphinxcontrib-0.1.0.tar.gz
2019-09-23 16:38:16,395 INFO  [req25421] [Wtx4429] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:16,518 INFO  [req25421] [Wtx4429] fswriter4430: committed: keys: 'company/dev/samplepkg-sphinxcontrib/.simple','company/dev/samplepkg-sphinxcontrib/0.1.0/.config','company/dev/.projects','company/dev/samplepkg-sphinxcontrib/.versions','company/dev/+f/ec1/f13e044d8339a/samplepkg-sphinxcontrib-0.1.0.tar.gz', files_commit: +files/company/dev/+f/ec1/f13e044d8339a/samplepkg-sphinxcontrib-0.1.0.tar.gz
2019-09-23 16:38:16,537 INFO  [req25422] POST /company/dev/
2019-09-23 16:38:17,089 INFO  [req25422] [Wtx4430] not re-registering same metadata for samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:17,096 INFO  [req25422] company/dev: got submit release info 'samplepkg-sphinxcontrib'
2019-09-23 16:38:17,103 INFO  [NOTI] [Rtx4430] Committing 5 new documents to search index.
2019-09-23 16:38:17,105 INFO  [req25422] [Wtx4430] added 'releasefile' link company/dev/+f/c7b/0307e35270867/samplepkg_sphinxcontrib-0.1.0-py35-none-any.whl
2019-09-23 16:38:17,106 INFO  [req25422] [Wtx4430] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:17,190 INFO  [NOTI] [Rtx4430] Finished committing 5 documents to search index.
2019-09-23 16:38:17,212 INFO  [req25422] [Wtx4430] fswriter4431: committed: keys: 'company/dev/samplepkg-sphinxcontrib/.simple','company/dev/samplepkg-sphinxcontrib/0.1.0/.config','company/dev/+f/c7b/0307e35270867/samplepkg_sphinxcontrib-0.1.0-py35-none-any.whl', files_commit: +files/company/dev/+f/c7b/0307e35270867/samplepkg_sphinxcontrib-0.1.0-py35-none-any.whl
2019-09-23 16:38:17,395 INFO  [NOTI] [Rtx4431] Committing 5 new documents to search index.
2019-09-23 16:38:17,469 INFO  [NOTI] [Rtx4431] Finished committing 5 documents to search index.
2019-09-23 16:38:19,088 INFO  [req25423] GET /company/dev/+api
2019-09-23 16:38:19,320 INFO  [req25424] POST /+login
2019-09-23 16:38:19,771 INFO  [req25425] POST /company/dev/
2019-09-23 16:38:19,779 INFO  [req25425] [Wtx4431] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:19,779 INFO  [req25425] company/dev: got submit release info 'samplepkg-sphinxcontrib'
2019-09-23 16:38:19,781 INFO  [req25425] [Wtx4431] added 'doczip' link company/dev/+f/495/b4f4b48ca5f76/samplepkg-sphinxcontrib-0.1.0.doc.zip
2019-09-23 16:38:19,781 INFO  [req25425] [Wtx4431] set_metadata samplepkg-sphinxcontrib-0.1.0
2019-09-23 16:38:19,785 INFO  [req25425] [Wtx4431] fswriter4432: committed: keys: 'company/dev/+f/495/b4f4b48ca5f76/samplepkg-sphinxcontrib-0.1.0.doc.zip','company/dev/samplepkg-sphinxcontrib/0.1.0/.config', files_commit: +files/company/dev/+f/495/b4f4b48ca5f76/samplepkg-sphinxcontrib-0.1.0.doc.zip
2019-09-23 16:38:19,992 INFO  [NOTI] [Rtx4432] Committing 11 new documents to search index.
2019-09-23 16:38:20,124 INFO  [NOTI] [Rtx4432] Finished committing 11 documents to search index.
2019-09-23 16:38:20,308 INFO  [NOTI] [Rtx4432] Committing 11 new documents to search index.
2019-09-23 16:38:20,696 INFO  [NOTI] [Rtx4432] Finished committing 11 documents to search index.
2019-09-23 16:38:53,859 INFO  [req25426] GET /company/dev

@stinovlas
Copy link

Yesterday I got exactly the same results as vlcinsky did, but I wasn't able to recreate it with local devpi and minimal working example. However, I have run into a different error. Now, the upload does not start in a first place. Meaning that there is no devpi-server log and poetry is stuck in this state and eventualy crashes with ConnectionError:

Publishing my_package (0.2.0) to test
 - Uploading my_package-0.2.0-py3-none-any.whl 0%                                                
[ConnectionError]                
('Connection aborted.', OSError(0, 'Error'))  
                                                
Exception trace:
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/application.py in run() at line 94
   status_code = self.do_run(input_, output_)
 /home/jmusilek/.poetry/lib/poetry/console/application.py in do_run() at line 88
   return super(Application, self).do_run(i, o)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/application.py in do_run() at line 197
   status_code = command.run(input_, output_)
 /home/jmusilek/.poetry/lib/poetry/console/commands/command.py in run() at line 77
   return super(BaseCommand, self).run(i, o)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/commands/base_command.py in run() at line 146
   status_code = self.execute(input_, output_)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/cleo/commands/command.py in execute() at line 107
   return self.handle()
 /home/jmusilek/.poetry/lib/poetry/console/commands/publish.py in handle() at line 54
   self.option("repository"), self.option("username"), self.option("password")
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/publisher.py in publish() at line 84
   return self._uploader.upload(url)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in upload() at line 101
   self._upload(session, url)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in _upload() at line 185
   self._do_upload(session, url)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in _do_upload() at line 202
   resp = self._upload_file(session, url, file)
 /home/jmusilek/.poetry/lib/poetry/masonry/publishing/uploader.py in _upload_file() at line 237
   headers={"Content-Type": monitor.content_type},
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/sessions.py in post() at line 581
   return self.request('POST', url, data=data, json=json, **kwargs)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/sessions.py in request() at line 533
   resp = self.send(prep, **send_kwargs)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/sessions.py in send() at line 646
   r = adapter.send(request, **kwargs)
 /home/jmusilek/.poetry/lib/poetry/_vendor/py3.7/requests/adapters.py in send() at line 498
   raise ConnectionError(err, request=request)

publish [-r|--repository REPOSITORY] [-u|--username USERNAME] [-p|--password PASSWORD] [--build]

Uploading package through devpi upload still works fine.

Output of devpi --version:

devpi-client 5.0.0

current devpi server: http://localhost:3141/
    devpi-server 5.1.0

I tried both poetry 0.12.17 and 1.0.0b1.

@stale
Copy link

stale bot commented Nov 23, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the stale label Nov 23, 2019
@stale
Copy link

stale bot commented Nov 30, 2019

Closing this issue automatically because it has not had any activity since it has been marked as stale. If you think it is still relevant and should be addressed, feel free to open a new one.

@stale stale bot closed this as completed Nov 30, 2019
@kierun
Copy link

kierun commented Dec 2, 2019

Does this really need closing? Has it been fixed?

@VikashKothary
Copy link

VikashKothary commented Mar 12, 2020

Yes I believe it has been fixed. I managed to get it working thanks to a few links below.

What I did:

  1. Connect to DevPi Server: devpi use http://localhost:3141
  2. Log in: devpi login root
  3. Create non-root user: devpi user -c packages email=packaging@company.com password=packages
  4. Create new package index: devpi index -c packages/stable bases=root/pypi volatile=False
  5. Add the Devpi repository to poetry: poetry config repositories.stable https://localhost:3141/packages/stable
  6. Publish to Devpi: poetry publish -r stable -u ${username} -p {password}

Then it should work. If you setup you've set the devpi repo as the default source, you can then just run poetry add <package> and it should work.

Links:

  1. Upload to TestPyPI silently failed. #742 (comment)
  2. https://stefan.sofa-rockers.org/2017/11/09/getting-started-with-devpi/
  3. https://python-poetry.org/docs/repositories/#using-a-private-repository

@agatti
Copy link

agatti commented Feb 4, 2021

It seems this is still occurring on Python 3.8.7 installed via MacPorts with:

❯ devpi --version
devpi-client 5.2.1

current devpi server: http://localhost:3141/
    devpi-lockdown 1.0.1
    devpi-semantic-ui 0.2.2
    devpi-server 5.5.0
    devpi-web 4.0.5

❯ poetry -V
Poetry version 1.1.4

with Poetry being installed from the installer script to work around #3362

@kthr
Copy link

kthr commented Jan 14, 2022

I can also reproduce the error. Poetry seems to exit with exit code 0. Although I get the following error:

UploadError
HTTP Error 403: Invalid or non-existent authentication information. See https://pypi.org/help/#invalid-auth for more information.

Upload/publish with twine on the same index with same credentials works fine.

> devpi --version
devpi-client 5.2.1

current devpi server: https://devpi.*.com
    devpi-lockdown 2.0.0
    devpi-semantic-ui 0.2.2
    devpi-server 6.2.0
    devpi-web 4.0.8

> poetry -V
Poetry version 1.1.12

Copy link

github-actions bot commented Mar 2, 2024

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Mar 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area/build-system Related to PEP 517 packaging (see poetry-core) kind/bug Something isn't working as expected
Projects
None yet
Development

No branches or pull requests

9 participants