Skip to content

Turn unwarranted ABORT cases into FAIL for scs-0117, scs-0123 #1063

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mbuechse wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Turn unwarranted ABORT cases into FAIL for scs-0117, scs-0123 #1063

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3204469
Turn unwarranted ABORT cases into FAIL for scs-0117, scs-0123
mbuechse Jan 24, 2026
325a385
Revert accidental change that (rightly) trips up flake8
mbuechse Jan 24, 2026
cc99232
Skip scs-0123-swift-s3 if object-store not present
mbuechse Feb 19, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Tests/iaas/openstack_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,7 +150,7 @@ def make_container(cloud):
c.add_function('scs_0123_service_placement', lambda c: compute_scs_0123_service_presence(c.services_lookup, 'placement'))
c.add_function('scs_0123_service_object_store', lambda c: compute_scs_0123_service_presence(c.services_lookup, 'object-store'))
c.add_function('scs_0123_storage_apis', lambda c: compute_scs_0123_service_presence(c.services_lookup, 'volume', 'volumev3', 'block-storage'))
c.add_function('scs_0123_swift_s3', lambda c: compute_scs_0123_swift_s3(c.conn))
c.add_function('scs_0123_swift_s3', lambda c: compute_scs_0123_swift_s3(c.services_lookup, c.conn))
return c


Expand Down
16 changes: 10 additions & 6 deletions Tests/iaas/scs_0117_volume_backup/volume_backup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ def cleanup(conn: openstack.connection.Connection, prefix=DEFAULT_PREFIX) -> boo
logging.info(f"↳ deleting volume backup '{backup.id}' ...")
conn.block_storage.delete_backup(backup.id, ignore_missing=False)
except Exception as e:
if isinstance(e, openstack.exceptions.ResourceNotFound):
if isinstance(e, (openstack.exceptions.ResourceNotFound, openstack.exceptions.NotFoundException)):
# if the resource has vanished on its own in the meantime ignore it
# however, ResourceNotFound will also be thrown if the service 'cinder-backup' is missing
if 'cinder-backup' in str(e):
Expand Down Expand Up @@ -209,15 +209,19 @@ def compute_scs_0117_test_backup(conn, prefix=DEFAULT_PREFIX):
the restored volume is correct (for the sake of simplicity, it only uses empty volumes
and does not look at data).
"""
if not cleanup(conn, prefix=prefix):
raise RuntimeError("Initial cleanup failed")
try:
test_backup(conn, prefix=prefix)
if not cleanup(conn, prefix=prefix):
# what we're usually seeing here is either a problem with cinder-backup or with volumes
# -- either way, consider this a FAIL, not an ABORT (these things have to work!)
logging.error("Initial cleanup failed")
return False
try:
test_backup(conn, prefix=prefix)
finally:
cleanup(conn, prefix=prefix)
except BaseException:
logging.error('Backup test failed.')
logging.debug('exception details', exc_info=True)
return False
else:
return True
finally:
cleanup(conn, prefix=prefix)
7 changes: 4 additions & 3 deletions Tests/iaas/scs_0123_mandatory_services/mandatory_services.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,12 +114,13 @@ def s3_from_ostack(usable_credentials, conn, rgx=re.compile(r"^(https*://[^/]*)/
return s3_creds


def compute_scs_0123_swift_s3(conn: openstack.connection.Connection):
def compute_scs_0123_swift_s3(services_lookup, conn: openstack.connection.Connection):
"""
This test ensures that S3 can be used to access object storage using EC2 credentials from the identity API.
It will abort with an exception if no service of type object-storage is present. As of now, we deem
this behavior adequate.
"""
if 'object-store' not in services_lookup:
Copy link
Contributor

@depressiveRobot depressiveRobot Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do I misunderstand something here or should this be s3 here instead? s3 is listed in the Mandatory IaaS APIs, whereas object-store is listed optional.

Copy link
Contributor Author

@mbuechse mbuechse Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know what you mean, but this test requires the catalog entry to derive the S3 host. We can improve this test if and when a test subject arrives that satisfies our standard without offering object-store.

Copy link
Contributor Author

@mbuechse mbuechse Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One alternative would be to have the test succeed, and I'm open to that.

Copy link
Contributor

@depressiveRobot depressiveRobot Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As this seems to be related to the issue #1003, let's discuss this in the next SIG Std/Cert.

Copy link
Contributor Author

@mbuechse mbuechse Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The thing is that we also have the testcase scs-0123-service-object-store listed as mandatory, which would then be wrong as well, and I have a hunch that this was a conscious decision to make testing possible. We would have to check the supplement.

Copy link
Contributor Author

@mbuechse mbuechse Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The supplement doesn't help here. The origin can be found here:
https://github.com/SovereignCloudStack/standards/blame/b61054b7ca8a7de6bd627351964bf63b28642feb/Tests/iaas/mandatory-services/mandatory-iaas-services.py#L27

Copy link
Contributor Author

@mbuechse mbuechse Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apart from that, the script used to have this capability:

If the deployment uses s3 only and does not have the object store endpoint specified in the service catalog, the "--s3-endpoint" flag may be used to specify the s3 endpoint. In that case the "--s3-access" and "--s3-access-secret" flags must also be set, to give all necessery credentials to the test suite

This capability has since been removed (I think) because it's hard to automate. However, it can be added in again should the need arise.

That doesn't affect the matter regarding object-store.

Copy link
Contributor Author

@mbuechse mbuechse Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I have to retract the last sentence:

    if s3_credentials:
        mandatory_services.remove("object-store")

this was in the original code and has since been removed 👀

Copy link
Contributor Author

@mbuechse mbuechse Feb 19, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The testcase is just skipped (but marked PASS) if object-store isn't present. I think this change is acceptable for the time being (until we get the whole s3 situation sorted out, see #1004).

logger.info('skipping scs-0123-swift-s3 because object-store not present')
return True
# we assume s3 is accessible via the service catalog, and Swift might exist too
usable_credentials = []
s3_buckets = []
Expand Down