Fedora 31 #12619

croissanne · 2019-08-22T08:51:18Z

image-refresh fedora-31

rebuild image again to get grubby installed
fix custom cockpit keytab: PR session: Fix custom cockpit kerberos keytab with krb5 > 1.17 #12732

Issues:

atomic package isn't available on f31
docker doesn't support cgroupsv2: fedora 31 docker support #12670

marusak

Thank you, a few things though:

Before this PR should come PR that would declare fedora-31 into testmap.py so it can be triggered.
This needs to be added into bots/image-trigger
Bunch of naughties needs to be copied over, but I agree with with firstly running tests and copying only those, that are needed
Adding master as default target for fedora-31 (see point 0.)
Needs to be mentioned in README
Needs to build image in this PR
There are a few tests that behave differently on fedora-30, they also needs to check for fedora-31

marusak · 2019-08-22T08:57:59Z

bots/images/scripts/fedora-31.bootstrap

+# 02110-1301 USA.
+
+BASE=$(dirname $0)
+$BASE/virt-install-fedora "$1" x86_64 "http://dl.fedoraproject.org/pub/fedora/linux/releases/31/Server/x86_64/os/"


This URl does not exists (yet), please use https://dl.fedoraproject.org/pub/fedora/linux/development/31/Everything/x86_64/os/

bots/image-refresh

marusak · 2019-08-22T09:06:50Z

Needs to build image in this PR

I see you added it. Don't forget to add bot label when ready so bots really pick it up

cockpituous · 2019-08-22T09:26:37Z

image-refresh in progress on 3-ci-srv-06.
Log: https://209.132.184.41:8493/logs/image-refresh-12619-20190822-092626/

bots/images/scripts/fedora-31.bootstrap

cockpituous · 2019-08-22T09:59:50Z

Task failed: https://209.132.184.41:8493/logs/image-refresh-12619-20190822-092626/

cockpituous · 2019-08-22T12:26:53Z

image-refresh in progress on 1-ci-srv-03.
Log: https://209.132.184.41:8493/logs/image-refresh-12619-20190822-122626/

cockpituous · 2019-08-22T13:33:44Z

Task failed: https://209.132.184.41:8493/logs/image-refresh-12619-20190822-122626/

cockpituous · 2019-08-23T07:46:54Z

image-refresh in progress on centosci-tasks-w4dn3.
Log: https://logs-https-cockpit.apps.ci.centos.org/logs/image-refresh-12619-20190823-074646/

cockpituous · 2019-08-23T07:57:21Z

Task failed: https://logs-https-cockpit.apps.ci.centos.org/logs/image-refresh-12619-20190823-074646/

cockpituous · 2019-08-23T08:15:52Z

image-refresh in progress on 3-cockpit-8.
Log: https://209.132.184.41:8493/logs/image-refresh-12619-20190823-081515/

cockpituous · 2019-08-23T08:38:16Z

Task failed: https://209.132.184.41:8493/logs/image-refresh-12619-20190823-081515/

martinpitt · 2019-09-06T14:25:20Z

Notes about the curl check-realms failure: My initial thought was that F31 may have switched from the kernel keyring to the user-land KCM (keyring cache daemon) by default. But /etc/krb5.conf.d/kcm_default_ccache already exists in F30, and so does sssd-kcm.service. On F31 this service is active, and in klist I do see the ticket:

# klist
Ticket cache: KCM:0
Default principal: admin@COCKPIT.LAN

Valid starting       Expires              Service principal
06.09.2019 10:02:54  07.09.2019 10:02:53  host/x0.cockpit.lan@COCKPIT.LAN
06.09.2019 10:02:53  07.09.2019 10:02:53  krbtgt/COCKPIT.LAN@COCKPIT.LAN
06.09.2019 10:03:01  07.09.2019 10:02:53  HTTP/x0.cockpit.lan@COCKPIT.LAN

But with curl

curl --negotiate --delegation always -u : -D- http://x0.cockpit.lan:9090/cockpit/login

you get

cockpit-session: gssapi auth failed: Request ticket server HTTP/x0.cockpit.lan@COCKPIT.LAN not found in keytab (ticket kvno 1)

But it's cleary there:

# klist -k /etc/cockpit/krb5.keytab 
Keytab name: FILE:/etc/cockpit/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   1 HTTP/x0.cockpit.lan@COCKPIT.LAN
   1 HTTP/x0.cockpit.lan@COCKPIT.LAN

This is pretty much exactly how it looks like on Fedora 30.

FTR, this is not about the user ticket, that works fine (with KCM and kernel keyring, and you get a ticket either way). This is about kerberos not being able to authenticate the cockpit service.

Interestingly, when we move the cockpit server keytab to the default location, it works:

mv /etc/krb5.keytab{,.disabled}
mv /etc/cockpit/krb5.keytab /etc/krb5.keytab

after that, curl gets a valid login/session. So it seems whatever we do to support a custom keytab in the server stopped working.

I downloaded the older 1.17-4.fc30 version of libkadm5, krb-workstation, and krb5-libs from
Fedora 30, downgraded (rpm -U --force --nodeps), and it immediately started working again. That gives some chance of bisecting between -4 and -40, possibly the $KRB5_KTNAME support got broken? Fortunately it's still the same upstream version.

When looking through the diff, the secure_getenv() patch (backported from upstream) caught my attention, as cockpit-session is suid root. And indeed when I downgrade to 1.17-15 (the version immediately before that), things work. When I upgrade to 1.17-16 (the version that introduced secure_getenv), it fails.

There is unfortunately no other way to set the keytab than through the environment, as changing the global configuration file is out of the question for cockpit-session. So we either need to convince secure_getenv() into accepting the variable, or I'll file an upstream and downstream bug and ask if there's another trick or this could be partially reverted.

I asked on the original upstream PR for advice.

cockpituous · 2019-09-06T16:24:26Z

image-refresh in progress on 2-cockpit-9.
Log: https://209.132.184.41:8493/logs/image-refresh-12619-20190906-162424/

cockpituous · 2019-09-06T17:08:51Z

image-refresh fedora-31 done: https://github.com/cockpituous/cockpit/commits/image-refresh-fedora-31-20190906-170808

martinpitt · 2019-09-07T06:42:15Z

I sent a fix for custom krb keytab in PR #12732. Adding "blocked" for that. Feel free to rope in the commit in the meantime if you wan to run tests on CI.

martinpitt · 2019-09-09T07:19:37Z

Unblocking. PR #12732 landed, so kerberos (check-realms) should work now.

martinpitt

Thanks, great work! Please do some squashing here, this should be by and large just two commits ("add fedora-31", and the image refresh), plus possibly the two unrelated changes (dashboard as "admin" user, and firewall test generalizations) unless you split them out.

bots/images/scripts/fedora.setup

martinpitt · 2019-09-10T07:07:32Z

bots/images/scripts/fedora.setup

+# https://github.com/cockpit-project/cockpit/issues/12670
+if [ "$IMAGE" != fedora-31 ]; then
+    COCKPIT_DEPS="$COCKPIT_DEPS atomic docker"
+fi


If that's a temporary hack, fine. If it's planned (or at least likely) that docker just won't be a thing any more in F31, then let's swap this around: Only add docker to fedora-30 (legacy mode).

martinpitt · 2019-09-10T07:08:10Z

bots/images/scripts/fedora.setup

-    docker build -t cockpit/base /var/tmp/cockpit-base
+# HACK docker not available on f31
+# https://github.com/cockpit-project/cockpit/issues/12670
+if [ "$IMAGE" != fedora-31 ]; then


Let's perhaps make this if type docker >/dev/null 2>&1, as long as it's not yet clear which relelases docker will end up on in the long run. Or something similar to $HAVE_KUBERNETES?

bots/images/scripts/fedora.setup

martinpitt · 2019-09-10T07:10:05Z

bots/images/scripts/lib/fedora.install

@@ -91,6 +92,9 @@ else
    echo '====== skipping rpmlint check, not installed ====='
 fi
 EOF
+    chmod +x /tmp/rpmlint
+    su builder -c "/usr/bin/mock --offline --copyin /tmp/rpmlint /var/tmp/rpmlint"
+    su builder -c "/usr/bin/mock --offline --shell /var/tmp/rpmlint"


Can you please split this out into a separate commit (or even PR) with a more detailled explanation?

martinpitt · 2019-09-10T07:12:44Z

tools/cockpit.spec

@@ -634,7 +634,7 @@ bastion hosts, and a basic dashboard.
 %endif

 %ifarch x86_64 %{arm} aarch64 ppc64le i686 s390x
-%if 0%{?fedora}
+%if 0%{?fedora} && 0%{?fedora} <= 30


Not sure why this is in the "docker: Skip docker tests on fedora-31" and the previous "add fedora-31" commit. (github web UI bug?). Either way, this commit should be squashed into the "add fedora-31 image" commit.

bots/naughty/fedora-31/10038-abrtd-crash-fn-2

martinpitt · 2019-09-10T07:14:02Z

test/verify/check-networking-firewall

@@ -34,6 +34,12 @@ def wait_unit_state(machine, unit, state):

    wait(lambda: active_state(unit) == state, delay=0.2)

+def get_active_rules(machine):


Most of this commit looks like it could go into a separate PR and be landed first, as it's some generalization? Then the "add fedora-31" main commit can just add the "fedora-31" special cases to the lists.

martinpitt · 2019-09-10T07:15:49Z

bots/naughty/fedora-31/12730-no-custom-keytab

@@ -0,0 +1,3 @@
+File "test/verify/check-realms", line *, in *
+  self.assertIn("HTTP/1.1 200 OK", output)
+AssertionError: 'HTTP/1.1 200 OK' not found in 'HTTP/1.1 401*


This should be fixed properly now in master, and can be dropped.

martinpitt · 2019-09-10T07:16:31Z

test/verify/check-dashboard

@@ -94,7 +94,7 @@ class TestBasicDashboard(MachineCase, DashBoardHelpers):

        # Start second browser and check that it is in sync
        b2 = self.new_browser()
-        b2.default_user = "root"
+        b2.default_user = "admin"


This could also be split out and landed separately. (If it helps, then you don't need to run the entire set of tests for this PR)

martinpitt · 2019-09-10T11:55:59Z

This now conflicts after PR #12735, sorry.

martinpitt

Some nitpicks still, but looking really good now!

bots/images/scripts/fedora.setup

It's currently unclear whether or not docker will make it to fedora-31, so we can't run docker tests for now.

martinpitt

Thanks! Still needs a Closes #, although it's also fine to just squash the two commits.

changed

croissanne added the no-test For doc/workflow changes, or experiments which don't need a full CI run, label Aug 22, 2019

cockpituous added the test-external label Aug 22, 2019

marusak previously requested changes Aug 22, 2019

View reviewed changes

marusak reviewed Aug 22, 2019

View reviewed changes

bots/image-refresh Show resolved Hide resolved

croissanne changed the title ~~Fedora 31~~ WIP Fedora 31 Aug 22, 2019

croissanne force-pushed the fedora-31 branch from cb57aef to fb44266 Compare August 22, 2019 09:14

croissanne added the bot label Aug 22, 2019

croissanne changed the title ~~WIP Fedora 31~~ Fedora 31 Aug 22, 2019

croissanne added the needswork label Aug 22, 2019

cockpituous changed the title ~~Fedora 31~~ WIP: 3-ci-srv-06: [no-test] Fedora 31 Aug 22, 2019

marusak reviewed Aug 22, 2019

View reviewed changes

bots/images/scripts/fedora-31.bootstrap Show resolved Hide resolved

croissanne force-pushed the fedora-31 branch from fb44266 to 85b591b Compare August 22, 2019 09:28

cockpituous changed the title ~~WIP: 3-ci-srv-06: [no-test] Fedora 31~~ Fedora 31 Aug 22, 2019

croissanne force-pushed the fedora-31 branch from 8055d87 to e4c7a49 Compare August 22, 2019 12:25

cockpituous changed the title ~~Fedora 31~~ WIP: 1-ci-srv-03: [no-test] Fedora 31 Aug 22, 2019

cockpituous changed the title ~~WIP: 1-ci-srv-03: [no-test] Fedora 31~~ Fedora 31 Aug 22, 2019

croissanne force-pushed the fedora-31 branch from e4c7a49 to f89b031 Compare August 23, 2019 07:42

cockpituous changed the title ~~Fedora 31~~ WIP: centosci-tasks-w4dn3: [no-test] Fedora 31 Aug 23, 2019

cockpituous changed the title ~~WIP: centosci-tasks-w4dn3: [no-test] Fedora 31~~ Fedora 31 Aug 23, 2019

croissanne force-pushed the fedora-31 branch from f89b031 to 9a96889 Compare August 23, 2019 08:15

cockpituous changed the title ~~Fedora 31~~ WIP: 3-cockpit-8: [no-test] Fedora 31 Aug 23, 2019

croissanne mentioned this pull request Sep 6, 2019

kerberos auth: custom keytab stopped working on fedora-31 #12730

Closed

croissanne force-pushed the fedora-31 branch from 67b99f7 to d82206a Compare September 6, 2019 16:23

cockpituous changed the title ~~Fedora 31~~ WIP: 2-cockpit-9: [no-test] Fedora 31 Sep 6, 2019

cockpituous changed the title ~~WIP: 2-cockpit-9: [no-test] Fedora 31~~ Fedora 31 Sep 6, 2019

martinpitt added the blocked Don't land until something else happens first (see task list) label Sep 7, 2019

martinpitt removed the blocked Don't land until something else happens first (see task list) label Sep 9, 2019

croissanne force-pushed the fedora-31 branch 2 times, most recently from bcf6494 to 8ce48a6 Compare September 9, 2019 12:25

martinpitt requested changes Sep 10, 2019

View reviewed changes

croissanne force-pushed the fedora-31 branch from ed0e575 to 396fff7 Compare September 10, 2019 10:30

martinpitt added the needs-rebase label Sep 10, 2019

croissanne force-pushed the fedora-31 branch from 396fff7 to f2e1c08 Compare September 11, 2019 10:13

croissanne removed needs-rebase needswork no-test For doc/workflow changes, or experiments which don't need a full CI run, labels Sep 11, 2019

martinpitt requested changes Sep 11, 2019

View reviewed changes

bots/images/scripts/fedora.setup Show resolved Hide resolved

bots/images/scripts/fedora.setup Show resolved Hide resolved

bots/images/scripts/fedora.setup Outdated Show resolved Hide resolved

croissanne and others added 2 commits September 11, 2019 12:59

bots: Add fedora-31

f1e26c2

It's currently unclear whether or not docker will make it to fedora-31, so we can't run docker tests for now.

images: Update fedora-31 image

8836df8

croissanne force-pushed the fedora-31 branch from e464220 to 8836df8 Compare September 11, 2019 11:06

croissanne requested a review from martinpitt September 11, 2019 11:45

martinpitt approved these changes Sep 11, 2019

View reviewed changes

croissanne merged commit 9eb7ce7 into cockpit-project:master Sep 11, 2019

croissanne deleted the fedora-31 branch September 23, 2019 19:20

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fedora 31 #12619

Fedora 31 #12619

croissanne commented Aug 22, 2019 •

edited

Loading

marusak left a comment

marusak Aug 22, 2019

marusak commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 23, 2019

cockpituous commented Aug 23, 2019

cockpituous commented Aug 23, 2019

cockpituous commented Aug 23, 2019

martinpitt commented Sep 6, 2019 •

edited

Loading

cockpituous commented Sep 6, 2019

cockpituous commented Sep 6, 2019

martinpitt commented Sep 7, 2019

martinpitt commented Sep 9, 2019

martinpitt left a comment

martinpitt Sep 10, 2019

martinpitt Sep 10, 2019

martinpitt Sep 10, 2019

martinpitt Sep 10, 2019

martinpitt Sep 10, 2019

martinpitt Sep 10, 2019

martinpitt Sep 10, 2019

martinpitt commented Sep 10, 2019

martinpitt left a comment

martinpitt left a comment

		@@ -34,6 +34,12 @@ def wait_unit_state(machine, unit, state):

		wait(lambda: active_state(unit) == state, delay=0.2)

		def get_active_rules(machine):

Fedora 31 #12619

Fedora 31 #12619

Conversation

croissanne commented Aug 22, 2019 • edited Loading

marusak left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marusak commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 22, 2019

cockpituous commented Aug 23, 2019

cockpituous commented Aug 23, 2019

cockpituous commented Aug 23, 2019

cockpituous commented Aug 23, 2019

martinpitt commented Sep 6, 2019 • edited Loading

cockpituous commented Sep 6, 2019

cockpituous commented Sep 6, 2019

martinpitt commented Sep 7, 2019

martinpitt commented Sep 9, 2019

martinpitt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

martinpitt commented Sep 10, 2019

martinpitt left a comment

Choose a reason for hiding this comment

martinpitt left a comment

Choose a reason for hiding this comment

croissanne commented Aug 22, 2019 •

edited

Loading

martinpitt commented Sep 6, 2019 •

edited

Loading