Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minikube_1.17.0-0_amd64.deb: package architecture (arm64) does not match system (amd64) #10224

Closed
mgabeler-lee-6rs opened this issue Jan 22, 2021 · 21 comments
Assignees
Labels
area/build-release kind/bug Categorizes issue or PR as related to a bug. os/linux packaging/apt Debian (apt) packaging issues priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.

Comments

@mgabeler-lee-6rs
Copy link

Steps to reproduce the issue:

This is #9995 again, because no fix was merged for it

#10004 was closed without any changes being merged it looks like, and #9998 is approved but not merged.

The broken binary from #9995 got addressed by @tstromberg doing some manual build & upload operations, seems 1.17.0 needs this again, unless #9998 can be merged and a 1.17.1 release sent out to replace it?

@medyagh
Copy link
Member

medyagh commented Jan 22, 2021

@mgabeler-lee-6rs thank you very much for the quick issue you opened ! we totally screwed u ! and missed to merge the PRs
I will make a patch for this early next week ! thank you for the reporting this

@medyagh medyagh added kind/bug Categorizes issue or PR as related to a bug. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Jan 22, 2021
@mgabeler-lee-6rs
Copy link
Author

Thanks for the quick followup :)

we totally screwed u !

No need to feel too bad, I just noticed this because I have some scripts to pull new minikube releases into my workflow when I get the new release announcement from GH, and they tripped on this. My actual workflow continues happily using the prior release :)

@tstromberg tstromberg added priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now. and removed priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. labels Jan 23, 2021
@tstromberg
Copy link
Contributor

tstromberg commented Jan 23, 2021

Any updates? We should definitely do a post-mortem for this event -- I should have done so the first time around.

Assigning to @medyagh as he's on duty this week, though I understand @ilya-zuyev was also looking into this.

@tstromberg
Copy link
Contributor

@ilya-zuyev has removed the errant files from the release and will be posting new ones.

@ilya-zuyev
Copy link
Contributor

Uploaded correct versions

ilyaz@skeletron --- /tmp/mk » wget https://github.com/kubernetes/minikube/releases/download/v1.17.0/minikube_1.17.0-0_amd64.deb                                                                                                                         127 ↵
--2021-01-22 17:37:37--  https://github.com/kubernetes/minikube/releases/download/v1.17.0/minikube_1.17.0-0_amd64.deb
Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/56353740/17fac800-5cd6-11eb-8066-db1c000cdd6c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210123%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210123T013737Z&X-Amz-Expires=300&X-Amz-Signature=0cfb6f525469e51e51b9fbe7a83e5584dd381d1adab806ebeeef131edf8af98d&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=56353740&response-content-disposition=attachment%3B%20filename%3Dminikube_1.17.0-0_amd64.deb&response-content-type=application%2Foctet-stream [following]
--2021-01-22 17:37:37--  https://github-production-release-asset-2e65be.s3.amazonaws.com/56353740/17fac800-5cd6-11eb-8066-db1c000cdd6c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210123%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210123T013737Z&X-Amz-Expires=300&X-Amz-Signature=0cfb6f525469e51e51b9fbe7a83e5584dd381d1adab806ebeeef131edf8af98d&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=56353740&response-content-disposition=attachment%3B%20filename%3Dminikube_1.17.0-0_amd64.deb&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.217.94.76
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.217.94.76|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 21845176 (21M) [application/octet-stream]
Saving to: ‘minikube_1.17.0-0_amd64.deb’

minikube_1.17.0-0_amd64.deb                                     100%[=====================================================================================================================================================>]  20.83M  8.88MB/s    in 2.3s    

2021-01-22 17:37:40 (8.88 MB/s) - ‘minikube_1.17.0-0_amd64.deb’ saved [21845176/21845176]

ilyaz@skeletron --- /tmp/mk » dpkg --info minikube_1.17.0-0_amd64.deb
 new Debian package, version 2.0.
 size 21845176 bytes: control archive=488 bytes.
     404 bytes,    12 lines      control              
 Package: minikube
 Version: 1.17.0
 Section: base
 Priority: optional
 Architecture: amd64
 Recommends: virtualbox
 Maintainer: Thomas Strömberg <t+minikube@stromberg.org>
 Description: Minikube
  minikube is a tool that makes it easy to run Kubernetes locally.
  minikube runs a single-node Kubernetes cluster inside a VM on your
  laptop for users looking to try out Kubernetes or develop with it 
  day-to-day.
ilyaz@skeletron --- /tmp/mk » 
ilyaz@skeletron --- /tmp/mk » 
ilyaz@skeletron --- /tmp/mk » sudo dpkg -i minikube_1.17.0-0_amd64.deb
Selecting previously unselected package minikube.
(Reading database ... 585808 files and directories currently installed.)
Preparing to unpack minikube_1.17.0-0_amd64.deb ...
Unpacking minikube (1.17.0) ...
Setting up minikube (1.17.0) ...
ilyaz@skeletron --- /tmp/mk » minikube version
minikube version: v1.17.0
commit: 7e8b5a89575945ba8f8246bfe547178c1a995198
ilyaz@skeletron --- /tmp/mk » 
ilyaz@skeletron --- /tmp/mk » uname -a
Linux skeletron 5.8.0-40-generic #45-Ubuntu SMP Fri Jan 15 11:05:36 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
ilyaz@skeletron --- /tmp/mk » 

@ilya-zuyev
Copy link
Contributor

for ARM Ubuntu:

ilyaz@pi64 --- /tmp/mk » wget https://github.com/kubernetes/minikube/releases/download/v1.17.0/minikube_1.17.0-1_arm64.deb                                                                                                                                1 ↵
--2021-01-23 01:40:14--  https://github.com/kubernetes/minikube/releases/download/v1.17.0/minikube_1.17.0-1_arm64.deb
Resolving github.com (github.com)... 192.30.255.113
Connecting to github.com (github.com)|192.30.255.113|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/56353740/2f39b580-5cd6-11eb-9bdd-35d95565b04b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210123%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210123T014000Z&X-Amz-Expires=300&X-Amz-Signature=c933c590c3c200b2a4be120b11e21724b9f1d8b006c69c0cc53ab6d36ae18913&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=56353740&response-content-disposition=attachment%3B%20filename%3Dminikube_1.17.0-1_arm64.deb&response-content-type=application%2Foctet-stream [following]
--2021-01-23 01:40:15--  https://github-production-release-asset-2e65be.s3.amazonaws.com/56353740/2f39b580-5cd6-11eb-9bdd-35d95565b04b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20210123%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20210123T014000Z&X-Amz-Expires=300&X-Amz-Signature=c933c590c3c200b2a4be120b11e21724b9f1d8b006c69c0cc53ab6d36ae18913&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=56353740&response-content-disposition=attachment%3B%20filename%3Dminikube_1.17.0-1_arm64.deb&response-content-type=application%2Foctet-stream
Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.137.172
Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.137.172|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 20070068 (19M) [application/octet-stream]
Saving to: ‘minikube_1.17.0-1_arm64.deb’

minikube_1.17.0-1_arm64.deb                                     100%[=====================================================================================================================================================>]  19.14M  8.46MB/s    in 2.3s    

2021-01-23 01:40:17 (8.46 MB/s) - ‘minikube_1.17.0-1_arm64.deb’ saved [20070068/20070068]

ilyaz@pi64 --- /tmp/mk » 
ilyaz@pi64 --- /tmp/mk » dpkg --info minikube_1.17.0-1_arm64.deb 
 new Debian package, version 2.0.
 size 20070068 bytes: control archive=472 bytes.
     381 bytes,    11 lines      control              
 Package: minikube
 Version: 1.17.0
 Section: base
 Priority: optional
 Architecture: arm64
 Maintainer: Thomas Strömberg <t+minikube@stromberg.org>
 Description: Minikube
  minikube is a tool that makes it easy to run Kubernetes locally.
  minikube runs a single-node Kubernetes cluster inside a VM on your
  laptop for users looking to try out Kubernetes or develop with it 
  day-to-day.
ilyaz@pi64 --- /tmp/mk » sudo dpkg -i minikube_1.17.0-1_arm64.deb 
Selecting previously unselected package minikube.
(Reading database ... 137104 files and directories currently installed.)
Preparing to unpack minikube_1.17.0-1_arm64.deb ...
Unpacking minikube (1.17.0) ...
Setting up minikube (1.17.0) ...
ilyaz@pi64 --- /tmp/mk » minikube version
minikube version: v1.17.0
commit: 7e8b5a89575945ba8f8246bfe547178c1a995198
ilyaz@pi64 --- /tmp/mk » uname -a
Linux pi64 5.4.0-1025-raspi #28-Ubuntu SMP PREEMPT Wed Dec 9 17:10:53 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
ilyaz@pi64 --- /tmp/mk » 

@ilya-zuyev
Copy link
Contributor

our Makefile creates deb files using template minikube_${VERSION}-${IDX}_${ARCH}.deb
Not sure, what's the use of ${IDX}. I created two versions of each deb file, just to avoid confusion:

minikube_1.17.0-0_amd64.deb 
minikube_1.17.0-0_arm64.deb
minikube_1.17.0-1_amd64.deb
minikube_1.17.0-1_arm64.deb

@afbjorklund
Copy link
Collaborator

Not sure, what's the use of ${IDX}.

Normally, this field is increased each time that you update the package (for the same upstream software version):

https://www.debian.org/doc/debian-policy/ch-controlfields.html#version

Since we don't have any "real" system packages, but just a vendor package, we always kept this revision as 0

The absence of a debian_revision is equivalent to a debian_revision of 0.

Then when we had to issue a rebuild, for the same version, we bumped it to 1. But it should go back to zero...

If the minikube package would have been in the Debian distribution proper, it would have a more "normal" revision.

@afbjorklund
Copy link
Collaborator

afbjorklund commented Jan 23, 2021

@ilya-zuyev : unfortunately, it doesn't seem to work:

minikube: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by minikube)

It was supposed to be built on an older platform (16.04)

This was minikube_1.17.0-1_amd64.deb, on 20.04

@apupier
Copy link

apupier commented Jan 25, 2021

the "latest" from https://storage.googleapis.com/minikube/releases/latest/minikube_latest_amd64.deb is still the arm one

apupier added a commit to apupier/vscode-camelk that referenced this issue Jan 25, 2021
a specific version

see kubernetes/minikube#10224

Signed-off-by: Aurélien Pupier <apupier@redhat.com>
apupier added a commit to apupier/vscode-camelk that referenced this issue Jan 25, 2021
a specific version

see kubernetes/minikube#10224

Signed-off-by: Aurélien Pupier <apupier@redhat.com>
apupier added a commit to apupier/vscode-camelk that referenced this issue Jan 25, 2021
a specific version

see kubernetes/minikube#10224
using 1.16 instead of 1.17 as even pointing to updated .deb is nto
working due to another problem:
kubernetes/minikube#10224 (comment)

Signed-off-by: Aurélien Pupier <apupier@redhat.com>
@tstromberg
Copy link
Contributor

Removed the bad .deb files from GCS that I was aware of.

gsutil rm gs://minikube/releases/latest/minikube_1.17.0-0_amd64.deb
gsutil rm gs://minikube/releases/latest/minikube_latest_amd64.deb
gsutil rm gs://minikube/releases/v1.17.0/minikube_latest_amd64.deb
gsutil rm gs://minikube/releases/v1.17.0/minikube_1.17.0-0_amd64.deb

@tstromberg tstromberg assigned priyawadhwa and unassigned medyagh and ilya-zuyev Jan 25, 2021
@tstromberg
Copy link
Contributor

tstromberg commented Jan 25, 2021

@priyawadhwa is working on publishing updated debian release packages, containing the correct minikube amd64 binary.

The package revision will be 2 (minikube_1.17.0-2_amd64.deb)

@priyawadhwa
Copy link

Following #9995 (comment), I ran the following the manually upload correct binaries:

Build:

 curl -L https://github.com/kubernetes/minikube/releases/download/v1.17.0/minikube-linux-amd64 > out/minikube 
make -e DEB_VERSION=1.17.0 -e DEB_REVISION=2 out/minikube_1.17.0-2_amd64.deb

Propagate:

gsutil cp out/minikube_1.17.0-2_amd64.deb gs://minikube/releases/latest
gsutil cp out/minikube_1.17.0-2_amd64.deb gs://minikube/releases/latest/minikube_latest_amd64.deb 
gsutil cp out/minikube_1.17.0-2_amd64.deb gs://minikube/releases/v1.17.0/

Then uploaded manually to GitHub releases:

curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube_1.17.0-2_amd64.deb
<drag file to GitHub release>

@priyawadhwa
Copy link

I've also removed minikube_1.17.0-0_amd64.deb and minikube_1.17.0-1_amd64.deb from the github release. The first was republished with changed checksums (against Debian policy), the second can't be installed on older distro's due to being linked against too new of a glibc

priyawadhwa pushed a commit to priyawadhwa/minikube that referenced this issue Jan 25, 2021
@tstromberg
Copy link
Contributor

Confirmed that the new packages install cleanly. Will leave this open for the next hour in case folks find additional issues, and then close.

@medyagh
Copy link
Member

medyagh commented Jan 25, 2021

@mgabeler-lee-6rs do u mind verifying on your end if the issue is fixed now ?

@tstromberg
Copy link
Contributor

No new reports, so I'm going to close this issue. Please comment with /reopen if you see anything.

@mgabeler-lee-6rs
Copy link
Author

mgabeler-lee-6rs commented Jan 25, 2021

@medyagh amd64 fix is good on my end.

I do notice that the minikube_1.17.0-0_arm64.deb was removed from GitHub without replacement

Edit: yes, I see that the _latest file is still there, but the versioned name is not.

@mgabeler-lee-6rs
Copy link
Author

A minor note: the rebuilt minikube_1.17.0-2_amd64.deb actually contains 1.17.0, not 1.17.0-0 or 1.17.0-2.

The same seems to be true of the arm64 packages, and I think of prior releases too. I think this is a separate issue however.

@tstromberg
Copy link
Contributor

tstromberg commented Jan 25, 2021

A minor note: the rebuilt minikube_1.17.0-2_amd64.deb actually contains 1.17.0, not 1.17.0-0 or 1.17.0-2.

The same seems to be true of the arm64 packages, and I think of prior releases too. I think this is a separate issue however.

Are you mentioning the versions reported by apt/dpkg, or the one reported by minikube version?

If the latter, this is intentional. The minikube binary installed by the Debian package is identical (same checksum) to the one we distributed for other platforms. The only thing we regenerated was the Debian packag, to contain the correct binary (ARM vs x86-64). The final version field used (2) is the Debian package revision: https://www.debian.org/doc/debian-policy/ch-controlfields.html#version

@mgabeler-lee-6rs
Copy link
Author

Are you mentioning the versions reported by apt/dpkg, or the one reported by minikube version?

The former -- the version info in the .deb package metadata.

A quick check identifies the issue here: https://github.com/kubernetes/minikube/blob/master/Makefile#L478

That should be sed -E -i 's/--VERSION--/'$(DEB_VERSION)-$(DEB_REVISION)'/g' ...

apupier added a commit to camel-tooling/vscode-camelk that referenced this issue Jan 26, 2021
a specific version

see kubernetes/minikube#10224
using 1.16 instead of 1.17 as even pointing to updated .deb is nto
working due to another problem:
kubernetes/minikube#10224 (comment)

Signed-off-by: Aurélien Pupier <apupier@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/build-release kind/bug Categorizes issue or PR as related to a bug. os/linux packaging/apt Debian (apt) packaging issues priority/critical-urgent Highest priority. Must be actively worked on as someone's top priority right now.
Projects
None yet
Development

No branches or pull requests

7 participants