Can not read files from ssh repos

[KantorSerhiy]

Issue description

Hey,
I have some problem with installing .zip files from ssh-repos.Maybe you can help me.

When I try to install from the public repository - everything works fine ->
pipenv install https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

but when the repository is private - I get the following ->
pipenv.patched.pip._internal.exceptions.NetworkConnectionError: 404 Client Error: Not Found for url: https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

I tried to do something like this ->
test = {url = "https://$GITHUB_TOKEN@github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip"},
but it doesn't work for me=(

Expected result

Everything will be installed and will work.

Actual result

pipenv.patched.pip._internal.exceptions.NetworkConnectionError: 404 Client Error: Not Found for url: https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

---

$ pipenv --support

Pipenv version: '2023.7.23'

Pipenv location: '/Users/skantor/.local/share/virtualenvs/work-E2tBrjiz/lib/python3.9/site-packages/pipenv'

Python location: '/Users/skantor/.local/share/virtualenvs/work-E2tBrjiz/bin/python'

OS Name: 'posix'

User pip version: '23.1.2'

user Python installations found:

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.9.17',
 'os_name': 'posix',
 'platform_machine': 'arm64',
 'platform_python_implementation': 'CPython',
 'platform_release': '22.5.0',
 'platform_system': 'Darwin',
 'platform_version': 'Darwin Kernel Version 22.5.0: Thu Jun  8 22:21:34 PDT '
                     '2023; root:xnu-8796.121.3~7/RELEASE_ARM64_T8112',
 'python_full_version': '3.9.17',
 'python_version': '3.9',
 'sys_platform': 'darwin'}

System environment variables:

• PATH
• __CFBundleIdentifier
• SHELL
• TERM
• USER
• FIG_JETBRAINS_SHELL_INTEGRATION
• TMPDIR
• COMMAND_MODE
• TERMINAL_EMULATOR
• SSH_AUTH_SOCK
• XPC_FLAGS
• TERM_SESSION_ID
• __CF_USER_TEXT_ENCODING
• LOGNAME
• LC_CTYPE
• XPC_SERVICE_NAME
• HOME
• SHLVL
• PWD
• OLDPWD
• VIRTUAL_ENV
• PS1
• _
• PYTHONDONTWRITEBYTECODE
• PIP_DISABLE_PIP_VERSION_CHECK
• PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

• PATH: /Users/skantor/.local/share/virtualenvs/work-E2tBrjiz/bin:/opt/homebrew/bin:/Library/Frameworks/Python.framework/Versions/3.11/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin
• SHELL: /bin/zsh
• PWD: /Users/skantor/work
• VIRTUAL_ENV: /Users/skantor/.local/share/virtualenvs/work

### Tasks

[matteius]

@KantorSerhiy I just checked this against #5793 and I believe that branch fixes the issue you describe. However if you could check as well that would be helpful.

[KantorSerhiy]

@matteius i just tried,I installed the latest from your draft-no-reqlib branch and performed
pipenv install https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip
and I got the same error:

Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
Installing https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip...
Resolving https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip...
⠹ Installing...CRITICAL:pipenv.patched.pip._internal.network.download:HTTP error 404 while getting https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/bin/pipenv", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/cli/options.py", line 58, in main
    return super().main(*args, **kwargs, windows_expand_args=False)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/cli/command.py", line 233, in install
    do_install(
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/routines/install.py", line 221, in do_install
    pkg_requirement = Requirement.from_line(pkg_line)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 2467, in from_line
    parsed_line.set_setup_info(parsed_line.get_setup_info())
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/requirements.py", line 791, in get_setup_info
    setup_info = SetupInfo.from_ireq(self.ireq, subdir=self.subdirectory)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1697, in from_ireq
    unpack_url(
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1837, in unpack_url
    file = get_http_url(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/vendor/requirementslib/models/setup_info.py", line 1802, in get_http_url
    from_path, content_type = download(link, temp_dir.path)
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/patched/pip/_internal/network/download.py", line 134, in __call__
    resp = _http_get_download(self._session, link)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/patched/pip/_internal/network/download.py", line 118, in _http_get_download
    raise_for_status(resp)
  File "/opt/homebrew/Cellar/pipenv/2023.7.23/libexec/lib/python3.11/site-packages/pipenv/patched/pip/_internal/network/utils.py", line 54, in raise_for_status
    raise NetworkConnectionError(http_error_msg, response=resp)
pipenv.patched.pip._internal.exceptions.NetworkConnectionError: 404 Client Error: Not Found for url: https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip

unfortunately it doesn't work for me =(
maybe you still have guesses about this?

(if I manually click on the link( https://github.com/KantorSerhiy/test/releases/download/1.0.0/test.zip) - the download process begins)

[matteius]

@KantorSerhiy I think there is some confusion on the expectations here vcs+ssh:// dependencies work (at least in my branch I referenced) but they have to be the actual repository and not a link to the zip file -- a link to the zip file url would just be an installable remote file URI, but it should not require (and will not work) with the git+ssh:// prefix. But you should be able to do pipenv install git+ssh://${GITHUB_TOKEN}@github.com/KantorSerhiy/test.git@1.0.0 (at leas ton the draft-no-reqslib branch) and it should manage it as installable vcs dependency. I hope this helps!

[KantorSerhiy]

this one is a bit not what I expect..

I will reveal a little more details:
I have a private repository that has a certain project structure. With the help of github actions, I form the structure I need(package) and immediately make a release. In the release files, I have a file with the extension .zip that I created during actions.
The main problem is that I can't install this file (.zip) into my project as a package (again, if it's a private repository. If it's public, then everything is fine).
I don't want to create a separate branch for this, I don't see the need, I have everything already packed in .zip

[matteius]

@KantorSerhiy Is there a regular pip install line format that works for this example of a private github repo zip file?

[KantorSerhiy]

no, that's why I wrote to you)

[matteius]

@KantorSerhiy based on our architecture that heavily utilizes pip, if it cannot be installed that way through pip it won't be installable that way through pipenv. I think the git+ssh:// repository install would fit your needs though -- it doesn't have to be a branch, it can be a specific commit hash you tag to.

More details on the forward looking architecture (which is mostly inline with the existing): #5818

[KantorSerhiy]

thank you for the answers.

I am attaching a screenshot.
the first is a private repository, the second I changed to public.

I don't understand why this is so and how it works.
what is the reason that the dependency is installed from the public repository without problems, but not from the private one.

P.S
same for
pipenv install https://github.com/KantorSerhiy/logica-test/releases/download/1.0.0/logica.zip
after the command above :
pipfile -> logica = {file = "https://github.com/KantorSerhiy/logica-test/releases/download/1.0.0/logica.zip"}

pipfile.lock ->

        "logica": {
            "editable": true,
            "file": "https://github.com/KantorSerhiy/logica-test/releases/download/1.0.0/logica.zip",
            "hashes": [
                "sha256:9e0ade2b10d4462401cc576c158eb5e6f6232ee8bf72281f49ab7a27ade95c1b"
            ],
            "markers": "python_version >= '3.0'"

[matteius]

what is the reason that the dependency is installed from the public repository without problems, but not from the private one.

There is no mechanism to try and authenticate the https url from within pip for the file, but there is a mechanism for working with private ssh vcs repositories. It will require you unlock the ssh key and then will be able to clone the repository from a certain commit or branch. I am not aware of an analogous way to install from a private git repository using just a file. You would have to publish a package to a private antifactory that supports keyring to do something like that, which is really something different from that.