tests: tests fixes for sru validation #15093

sergiocazzolato · 2025-02-17T21:35:41Z

These fixes are needed to fix sru validation where the snap snap is not used and we use the snapd deb from proposed instead.

The change include fixes for tests:

tests/main/apparmor-batch-reload/task.yaml
tests/main/apparmor-prompting-flag-restart/task.yaml
tests/main/apparmor-prompting-integration-tests/task.yaml
tests/main/apparmor-prompting-snapd-startup/task.yaml
tests/main/interfaces-requests-activates-handlers/task.yaml
tests/main/interfaces-snap-interfaces-requests-control/task.yaml
tests/main/remote-home/task.yaml
tests/main/store-state/task.yaml

github-actions · 2025-02-17T21:38:17Z

Tue Feb 18 20:32:44 UTC 2025
The following results are from: https://github.com/canonical/snapd/actions/runs/13398332102

Failures:

Preparing:

openstack:opensuse-tumbleweed-64
openstack:opensuse-tumbleweed-64

Restoring:

openstack:opensuse-tumbleweed-64
openstack:opensuse-tumbleweed-64

tests/main/apparmor-batch-reload/task.yaml

tests/main/apparmor-prompting-flag-restart/task.yaml

bboozzoo · 2025-02-18T12:16:54Z

tests/main/apparmor-prompting-flag-restart/task.yaml

@@ -14,12 +14,18 @@ systems:
  - ubuntu-core-*

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


I think this highlights the problem we discussed during one of the standups, the kernel supports prompting, but the userspace does not, unless reexec is enabled in which case we're using the apparmor userspace vendored in snapd snap. This should likely be considered a bug.

I don't think it's a bug that can be solved on the snapd side, unless snapd were somehow able to probe whether the system apparmor parser supports prompting.

One solution JJ suggested would be to simply backport a newer apparmor parser to 22.04, which would keep AA support for prompting in alignment between the kernel and parser.

The problem is that in general we can't/don't want to assume that kind of alignment. We can guarantee it on Ubuntu versions, but not other distributions. At the moment, AA kernel patches to support prompting are only in ubuntu kernels, so this would be a successful temporary fix, but I think the only way to know for sure if we can use prompting would be to check the system AA parser for support.

Even then, support could theoretically change after a .deb/.rpm update of the AA parser or kernel (likely due to a bug, but still), so I'm not sure we can have any real guarantees.

bboozzoo · 2025-02-18T12:17:20Z

tests/main/remote-home/task.yaml

-        # We are testing on Ubuntu where we know that reexec is active and we
-        # use an internal apparmor userspace stack.
-        tests.cleanup defer test ! -e /var/lib/snapd/apparmor/snap-confine.internal/nfs-support
+        if tests.info is-reexec-in-use; then


olivercalder

The problem with 22.04 without re-exec is that the system parser doesn't support prompting, not the kernel; the kernel does support it.

There are existing checks for expected prompting support in the execute sections of the prompting-related tests, and these validate that if we don't expect support, prompting is indeed not supported, so I'd rather integrate this new check into those existing cases, so it's more consistent and we know if/when that support changes (e.g. because of AppArmor parser being backported).

tests/main/apparmor-prompting-flag-restart/task.yaml

olivercalder · 2025-02-18T14:51:23Z

tests/main/apparmor-prompting-flag-restart/task.yaml

@@ -14,12 +14,18 @@ systems:
  - ubuntu-core-*

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


I don't think it's a bug that can be solved on the snapd side, unless snapd were somehow able to probe whether the system apparmor parser supports prompting.

One solution JJ suggested would be to simply backport a newer apparmor parser to 22.04, which would keep AA support for prompting in alignment between the kernel and parser.

The problem is that in general we can't/don't want to assume that kind of alignment. We can guarantee it on Ubuntu versions, but not other distributions. At the moment, AA kernel patches to support prompting are only in ubuntu kernels, so this would be a successful temporary fix, but I think the only way to know for sure if we can use prompting would be to check the system AA parser for support.

Even then, support could theoretically change after a .deb/.rpm update of the AA parser or kernel (likely due to a bug, but still), so I'm not sure we can have any real guarantees.

olivercalder · 2025-02-18T14:52:30Z

tests/main/apparmor-prompting-integration-tests/task.yaml

+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0
+    fi
+


I would rather integrate this into the check in execute: so we can verify that it is unsupported when we expect it to be, and supported otherwise.

olivercalder · 2025-02-18T15:37:41Z

tests/main/apparmor-prompting-integration-tests/task.yaml

    TEST_UID="$(id -u test)"
    echo "outstanding prompts:"
    snap debug api "/v2/interfaces/requests/prompts?user-id=$TEST_UID" || true
    echo "rules:"
    snap debug api "/v2/interfaces/requests/rules?user-id=$TEST_UID" || true

 execute: |
+    tests.exec is-skipped && exit 0
+
    echo "Precondition check that snapd is active"
    systemctl is-active snapd.service snapd.socket



Below,

if ! os.query is-ubuntu || os.query is-core || [ ! -f /sys/kernel/security/apparmor/features/policy/permstable32 ] || NOMATCH 'prompt' < /sys/kernel/security/apparmor/features/policy/permstable32 ; then

becomes

if ! os.query is-ubuntu || os.query is-core || [ ! -f /sys/kernel/security/apparmor/features/policy/permstable32 ] || NOMATCH 'prompt' < /sys/kernel/security/apparmor/features/policy/permstable32 || tests.info is-reexec-in-use && os.query is-ubuntu 22.04 ; then

so, all the checks to validate that apparmor prompting works in the supported systems and it doesn't in the unsupported will me moved to a new test with just one variant to avoid rechecks.
I'll push a following pr once this is merged.

olivercalder · 2025-02-18T15:38:29Z

tests/main/apparmor-prompting-flag-restart/task.yaml

@@ -14,12 +14,18 @@ systems:
  - ubuntu-core-*

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


Suggested change

tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0

tests.exec skip-test "Ubuntu 22.04 AppArmor parser doesn't support prompting" && exit 0

I'd like the check to occur on line 114 though so we can verify that prompting is not supported as expected.

tests/main/apparmor-prompting-integration-tests/task.yaml

olivercalder · 2025-02-18T15:39:20Z

tests/main/apparmor-prompting-snapd-startup/task.yaml

@@ -11,12 +11,18 @@ systems:
  - ubuntu-2*

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


Suggested change

tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0

tests.exec skip-test "Ubuntu 22.04 AppArmor parser doesn't support prompting" && exit 0

But I'd also prefer to add this to existing check on line 113 rather than skip the test for this edge case.

olivercalder · 2025-02-18T15:41:25Z

tests/main/interfaces-requests-activates-handlers/task.yaml

@@ -8,16 +8,24 @@ systems:
  - ubuntu-2*

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


Suggested change

tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0

tests.exec skip-test "Ubuntu 22.04 AppArmor parser doesn't support prompting" && exit 0

Same here though, I'd rather this check be part of the existing one on line 36.

olivercalder · 2025-02-18T15:43:32Z

tests/main/interfaces-snap-interfaces-requests-control/task.yaml

@@ -18,10 +18,16 @@ environment:
    PYTHONIOENCODING: utf-8

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


Suggested change

tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0

tests.exec skip-test "Ubuntu 22.04 AppArmor parser doesn't support prompting" && exit 0

But I would rather do this on line 67 instead

Only other remaining blocker: #15093 (comment)

olivercalder

Looks good, pending the final comment fix s/kernel/AppArmor parser/

And good idea to consolidate all the prompting support checks into a dedicated spread test, so all of the prompting behavior checks can just run on the systems for which support is expected and not deal with duplicate checks in each one.

olivercalder · 2025-02-18T19:18:58Z

tests/main/apparmor-prompting-integration-tests/task.yaml

@@ -74,7 +85,7 @@ execute: |

        exit 0
    fi
-
+    


Suggested change

olivercalder · 2025-02-18T19:19:17Z

tests/main/interfaces-snap-interfaces-requests-control/task.yaml

@@ -18,10 +18,16 @@ environment:
    PYTHONIOENCODING: utf-8

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


Only other remaining blocker: #15093 (comment)

codecov · 2025-02-18T19:33:28Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 78.11%. Comparing base (a272aac) to head (39a3d52).
Report is 24 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #15093      +/-   ##
==========================================
+ Coverage   78.07%   78.11%   +0.04%     
==========================================
  Files        1182     1174       -8     
  Lines      157743   157682      -61     
==========================================
+ Hits       123154   123179      +25     
+ Misses      26943    26859      -84     
+ Partials     7646     7644       -2

Flag	Coverage Δ
unittests	`78.11% <ø> (+0.04%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

ernestl

Thanks, approving conditionally to blocker comment about kernel not supporting being corrected.

olivercalder · 2025-02-19T14:26:46Z

tests/main/interfaces-snap-interfaces-requests-control/task.yaml

@@ -18,10 +18,16 @@ environment:
    PYTHONIOENCODING: utf-8

 prepare: |
+    if not tests.info is-reexec-in-use && os.query is-ubuntu 22.04; then
+        tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0


Suggested change

tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0

tests.exec skip-test "Ubuntu 22.04 AppArmor parser doesn't support prompting" && exit 0

olivercalder · 2025-02-19T14:26:58Z

tests/main/apparmor-prompting-integration-tests/task.yaml

@@ -74,7 +85,7 @@ execute: |

        exit 0
    fi
-
+    


Suggested change

sergiocazzolato requested review from olivercalder, maykathm and ernestl February 17, 2025 21:35

updates needed to be applied on master for sru

d9c9940

sergiocazzolato force-pushed the tests-fixes-sru-validation branch from 7f4b7c3 to d9c9940 Compare February 17, 2025 21:37

sergiocazzolato changed the title ~~updates needed to be applied on master for sru~~ tests: tests fixes for sru validation Feb 17, 2025

maykathm reviewed Feb 18, 2025

View reviewed changes

tests/main/apparmor-batch-reload/task.yaml Outdated Show resolved Hide resolved

maykathm reviewed Feb 18, 2025

View reviewed changes

tests/main/apparmor-prompting-flag-restart/task.yaml Outdated Show resolved Hide resolved

bboozzoo reviewed Feb 18, 2025

View reviewed changes

olivercalder requested changes Feb 18, 2025

View reviewed changes

sergiocazzolato added 5 commits February 18, 2025 15:43

remove extra SNAPD_NO_MEMORY_LIMIT

3ea339d

Update exit message for prompting tests

ce93ec0

updated the check for ubuntu 22.04

7cc28a4

move exit in apparmor-prompting-integration-tests

52b459d

extra chacks for apparmor promtping are gonna be done in a following pr

39a3d52

olivercalder approved these changes Feb 18, 2025

View reviewed changes

sergiocazzolato requested review from bboozzoo and maykathm February 18, 2025 19:27

ernestl approved these changes Feb 19, 2025

View reviewed changes

maykathm approved these changes Feb 19, 2025

View reviewed changes

olivercalder reviewed Feb 19, 2025

View reviewed changes

sergiocazzolato merged commit d5ee313 into canonical:master Feb 19, 2025
76 of 77 checks passed

sergiocazzolato deleted the tests-fixes-sru-validation branch February 19, 2025 17:12

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tests: tests fixes for sru validation #15093

tests: tests fixes for sru validation #15093

sergiocazzolato commented Feb 17, 2025

github-actions bot commented Feb 17, 2025 •

edited

Loading

bboozzoo Feb 18, 2025

olivercalder Feb 18, 2025

bboozzoo Feb 18, 2025

olivercalder left a comment

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

sergiocazzolato Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025 •

edited

Loading

olivercalder left a comment

olivercalder Feb 18, 2025

olivercalder Feb 18, 2025 •

edited

Loading

codecov bot commented Feb 18, 2025 •

edited

Loading

ernestl left a comment

olivercalder Feb 19, 2025

olivercalder Feb 19, 2025

	tests.exec skip-test "Ubuntu 22.04 kernel doesn't support prompting" && exit 0
	tests.exec skip-test "Ubuntu 22.04 AppArmor parser doesn't support prompting" && exit 0

tests: tests fixes for sru validation #15093

tests: tests fixes for sru validation #15093

Conversation

sergiocazzolato commented Feb 17, 2025

github-actions bot commented Feb 17, 2025 • edited Loading

Failures:

Preparing:

Restoring:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

olivercalder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

olivercalder Feb 18, 2025 • edited Loading

Choose a reason for hiding this comment

olivercalder left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

olivercalder Feb 18, 2025 • edited Loading

Choose a reason for hiding this comment

codecov bot commented Feb 18, 2025 • edited Loading

Codecov Report

ernestl left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Feb 17, 2025 •

edited

Loading

olivercalder Feb 18, 2025 •

edited

Loading

olivercalder Feb 18, 2025 •

edited

Loading

codecov bot commented Feb 18, 2025 •

edited

Loading