WIP: Add 'sti' execute plugin #229

thrix · 2020-04-30T23:48:31Z

Support running sti tests via 'tmt'. Add an template for easily
create the test plan.

Example of the workflow

$ fedpkg clone python3
$ cd python3
$ tmt init --template sti

$ tmt run -a provision -h virtual.testcloud
or
$ tmt run -a provision -h container

Relates to https://pagure.io/fedora-ci/general/issue/4

Still needs some touches:

execution in container works
ansible failure is considered as test failure

Signed-off-by: Miroslav Vadkerti mvadkert@redhat.com

lgtm-com · 2020-04-30T23:57:58Z

This pull request introduces 1 alert when merging 7bb7d1c into 59bf14c - view on LGTM.com

new alerts:

1 for Unused local variable

lgtm-com · 2020-05-01T00:23:06Z

This pull request introduces 1 alert when merging f42b6f7 into 59bf14c - view on LGTM.com

new alerts:

1 for Unused local variable

psss

Let's add the new sti plugin as a dynamic one. In other words, let's finish #227 first and rebase on it. Essential part of an execute plugin is to create the results.yaml file with results which then can be consumed by the report step. Please include generating this file in the implementation as well.

thrix · 2020-05-02T11:15:19Z

@psss ah that was the one ;) definitely ... will do ... thanks

psss · 2020-05-04T07:41:31Z

Thanks. One more thought: Should we also do sti discover to explore available playbooks so that we can later report which one of them failed/passed?

thrix · 2020-05-06T07:35:45Z

@psss

Thanks. One more thought: Should we also do sti discover to explore available playbooks so that we can later report which one of them failed/passed?

I was thinking about this, are 2 things to note:

with tests.yml, I was not sure if we want a separate tests.yml for STI plugins, as the current structure is not something applicable here.
we only will support 1 playbook, because if we would more, basically it would mean we have to run each them in clean environment, and that we cannot do and I would like to avoid it.

So I would say we will need to live without discover 'sti', but report should ideally work anyway?

thrix · 2020-05-12T07:49:12Z

@psss

Thanks. One more thought: Should we also do sti discover to explore available playbooks so that we can later report which one of them failed/passed?

I was thinking about this, are 2 things to note:

with tests.yml, I was not sure if we want a separate tests.yml for STI plugins, as the current structure is not something applicable here.

we only will support 1 playbook, because if we would more, basically it would mean we have to run each them in clean environment, and that we cannot do and I would like to avoid it.

So I would say we will need to live without discover 'sti', but report should ideally work anyway?

Let's discuss today this topic also @psss

lgtm-com · 2020-05-28T20:16:57Z

This pull request introduces 1 alert when merging 510b3ff into bb67195 - view on LGTM.com

new alerts:

1 for Unused local variable

thrix · 2020-05-28T20:29:23Z

Ok, the code is now on dynamic plugins, seems to work. Now I need to parse out the results from results.yml if around and make sure the result is interpreted fine in case there is no results.yml from STI and we are done \o/

lgtm-com · 2020-05-28T20:38:11Z

This pull request introduces 1 alert when merging 31eaa6a into bb67195 - view on LGTM.com

new alerts:

1 for Unused local variable

psss

Thanks for working on this, @thrix! A couple of comments added.

spec/steps/execute.fmf

psss · 2020-05-29T10:00:28Z

spec/steps/execute.fmf

+        summary: Execute default playbook
+        example: |
+            execute:
+                how: sti


Child stories should contain at least a short description. Otherwise it's inherited from parent. You can use make docs to quickly check the final doc output.

psss · 2020-05-29T10:07:30Z

tmt/steps/execute/sti.py

+                self._playbook(),
+                inventory=self._inventory(guest),
+                options=f'-e artifacts={self.step.workdir}'
+            )


I tried to run it in this way:

tmt run -avvvddd provision -h virtual execute -h sti

In the middle of test execution a password prompt popped up:

root@192.168.122.54's password:

Not sure where the problem could be.

psss · 2020-05-29T10:10:58Z

tmt/steps/provision/podman.py

        """ Prepare container using ansible playbook """
        playbook = self._ansible_playbook_path(playbook)
+        inventory = inventory or f'{self.container},'
+        options = f'{options} ' or ''
+        local_temp = os.path.expanduser('~/.ansible/tmp')


What is this used for? Probably add a comment?

seems like a leftover.

tmt/steps/execute/sti.py

psss · 2020-05-29T10:15:04Z

tmt/steps/execute/sti.py

+class ExecutorSTI(tmt.steps.execute.ExecutePlugin):
+    """ Run tests using ansible according to Standard Test Interface spec """
+    _doc = """
+    Execute STI tests


Use the full STI name here, this is shown in the --help message.

psss · 2020-05-29T10:18:56Z

tmt/steps/execute/sti.py

+"""
+
+
+class ExecutorSTI(tmt.steps.execute.ExecutePlugin):


Unit tests are failing. It would be also nice to include at least a simple tier: 3 test.

will add some, keeping open for now

psss · 2020-05-29T10:20:54Z

tmt/steps/execute/sti.py

+        """ Show discover details """
+        super().show(['playbook'])
+
+    def _playbook(self):


I believe this would be more easily handled by the default() method. See other plugins for inspiration.

psss · 2020-05-29T10:23:20Z

tmt/steps/execute/sti.py

+        hostname = guest.guest or guest.container
+
+        inventory_file = os.path.join(
+            self.step.workdir, f'inventory-{hostname}')


You can use self.step.write(f'inventory-{hostname}', content) to create a file in the step workdir.

psss · 2020-05-29T10:25:52Z

tmt/steps/execute/sti.py

+
+    def wake(self):
+        """ Wake up the plugin (override data with command line) """
+


Easier way to achieve the same: super().wake(['playbook'])

lgtm-com · 2020-06-02T12:08:51Z

This pull request introduces 1 alert when merging ba3691a into 8060946 - view on LGTM.com

new alerts:

1 for Unused local variable

Support running Standard Test Interface tests via 'tmt'. Example of the workflow $ fedpkg clone python3 $ cd python3 $ tmt run -a provision -h virtual.testcloud execute -h sti Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

lgtm-com · 2021-02-10T20:28:56Z

This pull request introduces 1 alert when merging 3f3301f into 1cbb71e - view on LGTM.com

new alerts:

1 for Unused import

thrix · 2021-03-04T19:06:36Z

@psss pls ignore the rerequest for review :) I did not want to do that :)

thrix · 2021-04-29T10:36:39Z

sorry, infra again :( need to move

lukaszachy · 2022-07-07T09:41:02Z

@thrix Hi, Is this still relevant?

psss · 2022-07-07T09:53:51Z

I'd vote for closing the issue and reopenning only if there is a real need for this.

thrix · 2022-07-22T18:04:15Z

I agree with closing. Let's not complicate our lives with STI. There is more pressing stuff to do. There are no real users for now, I would rather force people to migrate to STI, which we have clear path for.

psss · 2022-07-22T20:00:55Z

would rather force people to migrate to STI

... to migrate "from STI"? ;-)

thrix requested a review from psss April 30, 2020 23:48

thrix assigned lukaszachy and thrix and unassigned lukaszachy Apr 30, 2020

thrix requested review from t184256, lukaszachy and pvalena April 30, 2020 23:49

thrix force-pushed the rfe-sti-executor branch from 7bb7d1c to f42b6f7 Compare May 1, 2020 00:12

psss requested changes May 2, 2020

View reviewed changes

thrix changed the title ~~Add 'sti' execute plugin~~ WIP: Add 'sti' execute plugin May 6, 2020

thrix force-pushed the rfe-sti-executor branch from f42b6f7 to 510b3ff Compare May 28, 2020 20:00

thrix force-pushed the rfe-sti-executor branch from 510b3ff to 31eaa6a Compare May 28, 2020 20:28

thrix requested a review from psss May 28, 2020 20:28

psss added enhancement step | execute Stuff related to the execute step labels May 29, 2020

psss requested changes May 29, 2020

View reviewed changes

thrix force-pushed the rfe-sti-executor branch from 31eaa6a to ba3691a Compare June 2, 2020 11:58

psss marked this pull request as draft June 9, 2020 15:15

psss mentioned this pull request Dec 10, 2020

Support executing ansible playbooks as tests #490

Open

Add 'sti' execute plugin

3f3301f

Support running Standard Test Interface tests via 'tmt'. Example of the workflow $ fedpkg clone python3 $ cd python3 $ tmt run -a provision -h virtual.testcloud execute -h sti Signed-off-by: Miroslav Vadkerti <mvadkert@redhat.com>

thrix force-pushed the rfe-sti-executor branch from ba3691a to 3f3301f Compare February 10, 2021 20:14

thrix requested review from psss and removed request for t184256 March 4, 2021 19:06

psss added this to the 1.5 milestone Mar 9, 2021

thrix modified the milestones: 1.5, 1.6 Apr 29, 2021

psss removed this from the 1.6 milestone May 31, 2021

thrix removed their assignment Jul 22, 2022

thrix closed this Jul 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

WIP: Add 'sti' execute plugin #229

WIP: Add 'sti' execute plugin #229

thrix commented Apr 30, 2020 •

edited

Loading

lgtm-com bot commented Apr 30, 2020

lgtm-com bot commented May 1, 2020

psss left a comment

thrix commented May 2, 2020

psss commented May 4, 2020

thrix commented May 6, 2020 •

edited

Loading

thrix commented May 12, 2020

lgtm-com bot commented May 28, 2020

thrix commented May 28, 2020

lgtm-com bot commented May 28, 2020

psss left a comment

psss May 29, 2020

psss May 29, 2020

psss May 29, 2020

thrix Feb 2, 2021

psss May 29, 2020

psss May 29, 2020

thrix Feb 2, 2021

psss May 29, 2020

psss May 29, 2020

psss May 29, 2020

lgtm-com bot commented Jun 2, 2020

lgtm-com bot commented Feb 10, 2021

thrix commented Mar 4, 2021

thrix commented Apr 29, 2021

lukaszachy commented Jul 7, 2022

psss commented Jul 7, 2022

thrix commented Jul 22, 2022

psss commented Jul 22, 2022


		def wake(self):
		""" Wake up the plugin (override data with command line) """

WIP: Add 'sti' execute plugin #229

WIP: Add 'sti' execute plugin #229

Conversation

thrix commented Apr 30, 2020 • edited Loading

lgtm-com bot commented Apr 30, 2020

lgtm-com bot commented May 1, 2020

psss left a comment

Choose a reason for hiding this comment

thrix commented May 2, 2020

psss commented May 4, 2020

thrix commented May 6, 2020 • edited Loading

thrix commented May 12, 2020

lgtm-com bot commented May 28, 2020

thrix commented May 28, 2020

lgtm-com bot commented May 28, 2020

psss left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lgtm-com bot commented Jun 2, 2020

lgtm-com bot commented Feb 10, 2021

thrix commented Mar 4, 2021

thrix commented Apr 29, 2021

lukaszachy commented Jul 7, 2022

psss commented Jul 7, 2022

thrix commented Jul 22, 2022

psss commented Jul 22, 2022

thrix commented Apr 30, 2020 •

edited

Loading

thrix commented May 6, 2020 •

edited

Loading