Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error: Version X with arch x64 not found #162

Closed
3 of 7 tasks
kldavis4 opened this issue Nov 13, 2020 · 73 comments · Fixed by OML-Team/open-metric-learning#237
Closed
3 of 7 tasks

Error: Version X with arch x64 not found #162

kldavis4 opened this issue Nov 13, 2020 · 73 comments · Fixed by OML-Team/open-metric-learning#237
Assignees
Labels
bug Something isn't working

Comments

@kldavis4
Copy link

Describe the bug
I am attempting to use setup-python with a self-hosted runner running on an ec2 instance. I am using this ami ubuntu/images/hvm-ssd/ubuntu-bionic-18.04-amd64-server-20201112. I have followed the instructions here. I am getting the following output when the setup-python action is run:

Run actions/setup-python@v2
  with:
    python-version: 3.7.7
    token: ***
  env:
    AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
Version 3.7.7 was not found in the local cache
Version 3.7.7 is available for downloading
Download from "https://github.com/actions/python-versions/releases/download/3.7.7-20200609.4/python-3.7.7-linux-18.04-x64.tar.gz"
Extract downloaded archive
/bin/tar xz --warning=no-unknown-keyword -C /actions-runner/_work/_temp/f7f341f3-b2b9-4eff-bff0-5b55acb3a0f1 -f /actions-runner/_work/_temp/402c2129-71c1-46e4-9263-3b04a4b9772a
Execute installation script
Check if Python hostedtoolcache folder exist...
Create Python 3.7.7 folder
Copy Python binaries to hostedtoolcache folder
Create additional symlinks (Required for the UsePythonVersion Azure Pipelines task and the setup-python GitHub Action)
Upgrading PIP...
Looking in links: /tmp/tmpryj40n3d
Requirement already satisfied: setuptools in /opt/hostedtoolcache/Python/3.7.7/x64/lib/python3.7/site-packages (41.2.0)
Requirement already satisfied: pip in /opt/hostedtoolcache/Python/3.7.7/x64/lib/python3.7/site-packages (19.2.3)
Collecting pip
Using cached https://files.pythonhosted.org/packages/cb/28/91f26bd088ce8e22169032100d4260614fc3da435025ff389ef1d396a433/pip-20.2.4-py2.py3-none-any.whl
Installing collected packages: pip
Successfully installed pip-20.2.4
Create complete file
Error: Version 3.7.7 with arch x64 not found
The list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json

It's unclear what is actually failing as it seems to be finding the 3.7.7 python version and installs it and then it fails after installing pip.

Which version of the action are you using?

  • v1
  • v2
  • Some other tag (such as v2.0.1 or master)

Environment

  • self-hosted
  • Linux
  • Windows
  • Mac

If applicable, please specify if you're using a container

Python Versions
Please list all of the effected versions of Python (3.8.2, etc.)
Tried 3.7.7, 3.10.0-alpha.2

To Reproduce
Steps to reproduce the behavior:

  1. Go to '...'
  2. Click on '....'
  3. Scroll down to '....'
  4. See error

Run/Repo Url
If applicable, and if your repo/run is public, please include a URL so it is easier for us to investigate.
NA

Screenshots
Screen Shot 2020-11-13 at 5 00 32 PM

Additional context
Here's the script that launches the runner:

#!/bin/bash -xe
apt-get update -y
apt-get upgrade -y
${local.apt-get-all-packages}
sudo usermod -a -G docker ubuntu
sudo systemctl start docker
sudo systemctl enable docker
export RUNNER_ALLOW_RUNASROOT=true
export AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
mkdir actions-runner
mkdir /opt/hostedtoolcache
cd actions-runner
curl -O -L https://github.com/actions/runner/releases/download/v${var.runner-version}/actions-runner-linux-x64-${var.runner-version}.tar.gz
tar xzf ./actions-runner-linux-x64-${var.runner-version}.tar.gz
./config.sh --url https://github.com/${var.owner}/${var.repository} --token ${var.github-registration-token} --name "actions-runner-$(hostname)" --unattended
sudo ./svc.sh install
sudo ./svc.sh start
@kldavis4 kldavis4 added the bug Something isn't working label Nov 13, 2020
@jolo-dev
Copy link

For me the same.
I use https://github.com/nektos/act for testing Github actions locally.

I got

[Deploy Powerlevel10k Docs/Python 3.x sample-2]   ❗  ::error::Version 3.x with arch x64 not found%0AThe list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
[Deploy Powerlevel10k Docs/Python 3.x sample-2]   ❌  Failure - Setup python
Error: exit with `FAILURE`: 1

and even tried the one from the README

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ['2.x', '3.x']
    name: Python ${{ matrix.python-version }} sample
    steps:
      - uses: actions/checkout@v2
      - name: Setup python
        uses: actions/setup-python@v2
        with:
          python-version: ${{ matrix.python-version }}
          architecture: x64
      - run: python my_script.py

@kldavis4
Copy link
Author

kldavis4 commented Dec 2, 2020

Found these related issues: #89 and #92

@kldavis4
Copy link
Author

kldavis4 commented Dec 2, 2020

The root of this issue seems to be that the tool-cache library function for fetching the cache directory (https://github.com/actions/toolkit/blob/master/packages/tool-cache/src/tool-cache.ts#L693) is using RUNNER_TOOL_CACHE but this is set to /actions-runner/_work/_tool instead of /opt/hostedtoolcache. Setting RUNNER_TOOL_CACHE (within the self-hosted runner or the actions yml) seems to have no effect and I haven't been able to determine yet how the default is getting set.

@kldavis4
Copy link
Author

kldavis4 commented Dec 3, 2020

I was able to figure out a workaround that seems ok. Not fully tested yet for possible side effects.

If I modify the setup script to create a symlink from /opt/hostedtoolcache to the directory it expects for RUNNER_TOOL_CACHE, everything installs correctly. So my modified script is now:

#!/bin/bash -xe
apt-get update -y
apt-get upgrade -y
${local.apt-get-all-packages}
sudo usermod -a -G docker ubuntu
sudo systemctl start docker
sudo systemctl enable docker
export RUNNER_ALLOW_RUNASROOT=true
export AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
mkdir actions-runner
mkdir /opt/hostedtoolcache
cd actions-runner

# Make /actions-runner/_work
mkdir _work

# Link /opt/hostedtoolcache as /actions-runner/_work/_tool
ln -s /opt/hostedtoolcache _work/_tool

curl -O -L https://github.com/actions/runner/releases/download/v${var.runner-version}/actions-runner-linux-x64-${var.runner-version}.tar.gz
tar xzf ./actions-runner-linux-x64-${var.runner-version}.tar.gz
./config.sh --url https://github.com/${var.owner}/${var.repository} --token ${var.github-registration-token} --name "actions-runner-$(hostname)" --unattended
sudo ./svc.sh install
sudo ./svc.sh start

@meiswjn
Copy link

meiswjn commented Dec 21, 2020

Unrelated to the problem of the author, but I initially thought I had the same problem:
I used debian instead of ubuntu, thus it did not work.

Mabye helpful for other stumbling over this

@higebu
Copy link

higebu commented Dec 23, 2020

@meiswjn You can use Debian with creating /etc/lsb-release like Ubuntu. If you use Debian 10, /etc/lsb-release should be same as Ubuntu 18.04 like this.

DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04 LTS"

@Segfaultd
Copy link

Any chance this issue get attention?

@Segfaultd
Copy link

Still not dead, and still wanting this issue to get attention. Someone alive?

@thetaiter
Copy link

poke

@IdoPort
Copy link

IdoPort commented Apr 19, 2021

another poke in the wall...

@tianqiliu-RBLX
Copy link

+1, following on this

@cep21
Copy link

cep21 commented May 3, 2021

@kldavis4 How did your workaround end up long term? Still working well?

@janjur
Copy link

janjur commented May 22, 2021

...PPoke

@epwalsh
Copy link

epwalsh commented Jun 12, 2021

+1

@codecholeric
Copy link

+1

1 similar comment
@shreeraviadhikari
Copy link

+1

@nitrocode
Copy link

nitrocode commented Jun 21, 2021

@kldavis4 I tried your workaround on amazon linux 2 but still no luck.

action

      # pre-commit setup
      - uses: actions/setup-python@v2
        with:
          python-version: 3.9.5
        env:
          AGENT_TOOLSDIRECTORY: /opt/hostedtoolcache
          RUNNER_TOOL_CACHE: /opt/hostedtoolcache

userdata

export USER=root
export RUNNER_ALLOW_RUNASROOT=true
export RUNNER_TOOL_CACHE=/opt/hostedtoolcache
export AGENT_TOOLSDIRECTORY=/opt/hostedtoolcache
mkdir -p $AGENT_TOOLSDIRECTORY
mkdir -p /actions-runner/_work
ln -s $AGENT_TOOLSDIRECTORY /actions-runner/_work/_tool
curl -s https://raw.githubusercontent.com/actions/runner/v2.278.0/scripts/create-latest-svc.sh | bash -s myorg

check

# ls -la /actions-runner/_work/
total 0
drwxr-xr-x 2 root root 19 Jun 21 20:45 .
drwxr-xr-x 3 root root 19 Jun 21 20:45 ..
lrwxrwxrwx 1 root root 20 Jun 21 20:45 _tool -> /opt/hostedtoolcache

In the action, I see

Error: Version 3.9.5 with arch x64 not found

If I look in the manifest, I see it

      {
        "filename": "python-3.9.5-linux-16.04-x64.tar.gz",
        "arch": "x64",
        "platform": "linux",
        "platform_version": "16.04",
        "download_url": "https://github.com/actions/python-versions/releases/download/3.9.5-109482/python-3.9.5-linux-16.04-x64.tar.gz"
      },

It looks like @actions/tool-cache may be the culprit.

import * as tc from '@actions/tool-cache';

We're using the default or some non pypy one so it's hitting useCpythonVersion

switch (version.toUpperCase()) {
case 'PYPY2':
return usePyPy('2', architecture);
case 'PYPY3':
// keep pypy3 pointing to 3.6 for backward compatibility
return usePyPy('3.6', architecture);
default:
return await useCpythonVersion(version, architecture);

which calls tc.find which fails to find it so it throws the error.

let installDir: string | null = tc.find(
'Python',
semanticVersionSpec,
architecture
);
if (!installDir) {
core.info(
`Version ${semanticVersionSpec} was not found in the local cache`
);
const foundRelease = await installer.findReleaseFromManifest(
semanticVersionSpec,
architecture
);
if (foundRelease && foundRelease.files && foundRelease.files.length > 0) {
core.info(`Version ${semanticVersionSpec} is available for downloading`);
await installer.installCpythonFromRelease(foundRelease);
installDir = tc.find('Python', semanticVersionSpec, architecture);
}
}
if (!installDir) {
throw new Error(
[
`Version ${version} with arch ${architecture} not found`,
`The list of all available versions can be found here: ${installer.MANIFEST_URL}`
].join(os.EOL)
);
}

@kldavis4
Copy link
Author

@kldavis4 How did your workaround end up long term? Still working well?

@cep21 yes. we still have the workaround in place and haven't run into any issues so far

@kldavis4
Copy link
Author

@nitrocode couple things to check out...

  1. I am not specifying any environment variables (AGENT_TOOLSDIRECTORY & RUNNER_TOOL_CACHE). I don't think that should cause a problem but it is different than our current workflows.
  2. I'm not totally sure on what directory the userdata runs in. My userdata creates the actions-runner folder in the "current" directory and I'm not confident that it will be the filesystem root directory and not the root user's home
  3. what version of ubuntu is your AMI? I'm using ubuntu/images/hvm-ssd/ubuntu-focal-20.04-amd64*

@nitrocode
Copy link

nitrocode commented Jun 21, 2021

@kldavis4 I'm using Amazon Linux 2 AMI. That's probably the issue. The platform_version in the manifest seems to be exclusive to Ubuntu with no way to override it.

@juanjgarcia
Copy link

I've been down the rabbit hole and I can't find an elegant solution. It looks like the best bet is to add support here: https://github.com/actions/python-versions/ and then have those new files added to the manifest?

Either way, this is blocking us, and I think its a pretty common situation given the prowess of AWS.

@neilferreira
Copy link

@nitrocode @kldavis4 Do either of you have a solution for Amazon Linux 2 yet?

@bumbummen99
Copy link

On my AWS Linux 2 runner i settled with just installing python myself in a setup script, does work well enough.

@Baekalfen
Copy link

Just for reference, I'm seeing this issues on self-hosted macOS as well.

@nitrocode
Copy link

nitrocode commented Jul 20, 2021

@bumbummen99 could you share that ? I'd be interested in using that in the interim if it's easy enough to choose a specific python version to install.

Edit: This is our workaround

      # pre-commit setup
      # This has issues running on self-hosted. See https://github.com/actions/setup-python/issues/162.
      # - uses: actions/setup-python@v2
      # The self-hosted runner has python 2.7.18 and 3.7.10 installed.
      - name: Ensure pip is linked to pip3
        if: false # disabled on eks self-hosted runners, enabled on asg self-hosted runners
        shell: bash
        run: |
          # The priority resembles python version (3000) so that if we add an alternative for an
          # older version of python with the same strategy, the newest version will win. Ideally,
          # we'd wrap the remaining steps in pre/post inside a virtualenv, which is probably what
          # the setup-python action does.
          update-alternatives --install /usr/bin/pip pip /usr/bin/pip3 3000
          # when pip is run as root (in later steps) it defaults to --prefix=/usr/local
          echo "/usr/local/bin" >> $GITHUB_PATH

      - uses: actions/setup-node@v2
        with:
          node-version: '14'
      # end pre-commit setup (workaround)

@Baekalfen
Copy link

I went down the rabbit hole as well, and like @kldavis4 points out, I link the problem might be related to _getCacheDirectory.

And the same workaround "solves" the issue on macOS. Although I haven't thoroughly tested, if it causes any issues down the road.

rm -rf actions-runner/_work/_tool && ln -s /Users/runner/hostedtoolcache actions-runner/_work/_tool

Defining RUNNER_TOOL_CACHE didn't affect the process.

@vincerubinetti
Copy link

vincerubinetti commented Jan 6, 2023

In my case I was doing something stupid. Maybe this will help someone.

- name: Set up Python
  uses: actions/setup-python@v4
  with:
    python-version: 3.10

3.10 is just a number in yaml and gets interpreted as version 3.1. Changed it to "3.10" and it worked (with ubuntu-latest).

@PavanMudigondaTR
Copy link

In my case I was doing something stupid. Maybe this will help someone.

- name: Set up Python
  uses: actions/setup-python@v4
  with:
    python-version: 3.10

3.10 is just a number in yaml and gets interpreted as version 3.1. Changed it to "3.10" and it worked (with ubuntu-latest).

good catch !

DinoBektesevic added a commit to dirac-institute/kbmod that referenced this issue Jan 10, 2023
Better isolation of environment compared to relying on
setup python actions and activating various base environments on
shell login. See github.com/actions/setup-python/issues/162
@fan9704
Copy link

fan9704 commented Jan 16, 2023

I Use Python 3.9.1
Still Cannot Use
image

@LucaMarconato
Copy link

Same problem here since last week, I get

 ::error::The version '3.9' with architecture 'arm64' was not found for Ubuntu 22.04.%0AThe list of all available versions can be found here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json

@abarragan-adtriba
Copy link

@fan9704 which platform and version are you using?
@LucaMarconato there's no 3.9 version to choose from

@LucaMarconato
Copy link

LucaMarconato commented Jan 20, 2023

@abarragan-adtriba true, I have changed it to "3.9.16" and the problem persists:

[Test/test-2        ]   ❗  ::error::Version 3.10.9 with arch arm64 not found%0AThe list of all available versions can be found 
here: https://raw.githubusercontent.com/actions/python-versions/main/versions-manifest.json
[Test/test-2        ]   ❌  Failure - Main Set up Python 3.10.9
[Test/test-2        ] exitcode '1': failure
[Test/test-2        ] 🏁  Job failed

I checked the link above and I can't find any reference to arm64 (I am having the problem since a switched to a Apple Silicon Mac).

EDIT: additional details.
I am using act for running the Github Actions, so I think the problem is related to this: nektos/act#1509.

@abarragan-adtriba
Copy link

abarragan-adtriba commented Jan 20, 2023

@LucaMarconato how is this a problem if there's no arch arm64 available at all? you are just querying for a non-existing arch, or maybe im missing smth. I mean it may be a problem on your setup but it's not related to this issue, but more on the looks of opening a request for adding versions for arm64

@fan9704
Copy link

fan9704 commented Jan 20, 2023

@abarragan-adtriba ubuntu-latest on GitHub Action

@abarragan-adtriba
Copy link

abarragan-adtriba commented Jan 23, 2023

@abarragan-adtriba ubuntu-latest on GitHub Action

@fan9704 , your issue is already explained in the thread, in particular: #162 (comment)

@fan9704
Copy link

fan9704 commented Jan 23, 2023

Got it thanks your reply

@LucaMarconato
Copy link

LucaMarconato commented Jan 23, 2023

@abarragan-adtriba ok I understand. So this seems like a limitation of act, I will report there. With act one is supposed to be able to obtain a docker image of the same OS that is used in Github for running the actions. From your message I understand that the Apple Silicon chips can't be used to run those os images.

@0x78f1935
Copy link

0x78f1935 commented Feb 14, 2023

Our Github actions runner is hosted in a containerized docker container.
I had the same issues when using the option

runs-on: [self-hosted]

Changing this too

runs-on: ubuntu-latest

did the trick for me.

@abarragan-adtriba
Copy link

abarragan-adtriba commented Feb 14, 2023

@0x78f1935 ubuntu-latest is a bad practice as you could have read on the thread, this was the reason people (myself included) got into this problem because that's a dynamic tag.

@0x78f1935
Copy link

0x78f1935 commented Feb 14, 2023

ubuntu-latest is a bad practice as you could have read on the thread,

Didn't see someone mentioning it was bad practice. Perhaps I read past it, but at the end of the day I can see why. Thanks for mentioning this!

In addition the comment made by abarragan-adtriba was very helpful.

In my case, wanting Python 3.11.1, I should use ubuntu-22.04 instead.
Again, thanks for pointing this out!

@Alotab
Copy link

Alotab commented Feb 17, 2023

In my case I was doing something stupid. Maybe this will help someone.

- name: Set up Python
  uses: actions/setup-python@v4
  with:
    python-version: 3.10

3.10 is just a number in yaml and gets interpreted as version 3.1. Changed it to "3.10" and it worked (with ubuntu-latest).

This was the missing piece. It's working now.

hyoklee added a commit to HDFGroup/hermes that referenced this issue Mar 20, 2023
This patch is for testing locally with `act`.
See actions/setup-python#162 .
AStar9999 added a commit to AStar9999/aspose-barcode-cloud-python that referenced this issue Jun 2, 2023
* Add MaxiCodeMode class

*Add :param mode: :type: MaxiCodeMode to MaxiCodeParams

*Delete :param encode_mode: :type: int MaxiCodeParams

* Try to temp fix build issue
actions/setup-python#162
myungjoo added a commit to myungjoo/nnstreamer that referenced this issue Apr 29, 2024
Github-action macos build started spitting:
```
Run actions/setup-python@v1
  with:
    python-version: 3.x
    architecture: x64
  env:
    changed_file_list: /var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmp.b0uhtmFUzp
    rebuild:        1
Error: Version 3.x with arch x64 not found
Available versions:
```
Similar with actions/setup-python#162

Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
myungjoo added a commit to myungjoo/nnstreamer that referenced this issue Apr 29, 2024
Github-action macos build started spitting:
```
Run actions/setup-python@v1
  with:
    python-version: 3.x
    architecture: x64
  env:
    changed_file_list: /var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmp.b0uhtmFUzp
    rebuild:        1
Error: Version 3.x with arch x64 not found
Available versions:
```
Similar with actions/setup-python#162

Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
jaeyun-jung pushed a commit to nnstreamer/nnstreamer that referenced this issue Apr 29, 2024
Github-action macos build started spitting:
```
Run actions/setup-python@v1
  with:
    python-version: 3.x
    architecture: x64
  env:
    changed_file_list: /var/folders/3m/p59k4qdj0f17st0gn2cmj3640000gn/T/tmp.b0uhtmFUzp
    rebuild:        1
Error: Version 3.x with arch x64 not found
Available versions:
```
Similar with actions/setup-python#162

Signed-off-by: MyungJoo Ham <myungjoo.ham@samsung.com>
@PhoenixxZ2023
Copy link

Run pytest
pytest
shell: /usr/bin/bash -e {0}
env:
pythonLocation: /opt/hostedtoolcache/Python/3.11.9/x64
LD_LIBRARY_PATH: /opt/hostedtoolcache/Python/3.11.9/x64/lib
============================= test session starts ==============================
platform linux -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
rootdir: /home/runner/work/PLUS/PLUS
collected 0 items

============================ no tests ran in 0.02s =============================
Error: Process completed with exit code 5.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.