A plan installs deps from all the tests

[TomasTomecek]

http://artifacts.dev.testing-farm.io/c089aa49-7fb6-4c72-acca-c19c3af59f00/
http://artifacts.dev.testing-farm.io/49368cf2-f543-4163-a3ca-d72b8928376a/

the rpmlint plan is failing above, this is the plan

summary:
  Execute rpmlint on the spec file
prepare:
  - name: packages
    how: install
    package:
    - rpmlint
execute:
  script:
  - rpmlint packit.spec

we have a few tests with require: it seems the plan inherits all of those requirements which is incorrect for us

if I want to run rpmlint on a spec in our upstream repo, what is the best way to do it? plan or a test?

[psss]

Looking at the plan I see the local test discover using fmf is enabled:

> tmt plan show rpmlint
/plans/rpmlint
     summary Execute rpmlint on the spec file
    discover 
         how fmf
     prepare 
        name packages
         how install
     package rpmlint
     exclude 
     missing fail
     execute 
         how tmt
      script rpmlint packit.spec
     enabled true

This is because it's inherited from main.fmf:

execute:
    how: tmt
discover:
    how: fmf

The solution is to use discover --how shell, the config can look like this:

summary:
    Execute rpmlint on the spec file
discover:
    tests:
      - name: rpmlint
        test: rpmlint packit.spec
prepare:
    how: install
    package: rpmlint

See also the discover --how shell docs for some more info.

[FrNecas]

Hah, Petr beat me to it, was just about to respond, never mind :)

Also as an alternative you can keep your plan the way it is and just add

discover:
    how: shell

[TomasTomecek]

Thank you, gentlemen! Trying it now here: packit/packit#1496

I'm wondering what would help me to understand the cause so I wouldn't need to create this issue. I guess that I was confused that the package list is not displayed here and it is in the TF runs above.

$ tmt plan show -v
...
/plans/rpmlint                                                                                                        
     summary Execute rpmlint on the spec file                                                                         
    discover                                                                                                          
         how fmf                                                                                                      
   provision                                                                                                          
         how virtual                                                                                                                                                                                                                        
       image fedora
        user root
      memory 2048
        disk 10
  connection session
     prepare 
        name packages
         how install
     package rpmlint
     exclude 
     missing fail
     execute 
         how tmt
      script rpmlint packit.spec
      report 
         how display
      finish 
         how shell
      script 
     enabled true
     sources /home/tt/g/packit/packit/plans/main.fmf
             /home/tt/g/packit/packit/plans/rpmlint.fmf

If you feel like this is properly documented and there is nothing to do in code, please close and I'll try to read the docs better next time (meaning, I will ask @FrNecas :)

[psss]

I don't think we can do much here, as discover can fetch tests from other repos and only then we can check for possible requires. Modifying tmt plan show to fetch all remote repositories is not an option, could be very slow. What could help you understand that something weird is happening is tmt run discover -v which gives an overview of tests which would be run.

[thrix]

what about providing a hint to the output with -v? Something like:

$ tmt plan show -v
...
/plans/rpmlint                                                                                                        
     summary Execute rpmlint on the spec file
        hint Run "tmt run discover -v" to discover tests
    discover                                                                                                          
         how fmf
   provision                                                                                                          
         how virtual                                                                                                                                                                                          
       image fedora
        user root
      memory 2048
        disk 10
  connection session
     prepare 
        name packages
         how install
     package rpmlint
     exclude 
     missing fail
     execute 
         how tmt
      script rpmlint packit.spec
      report 
         how display
      finish 
         how shell
      script 
     enabled true
     sources /home/tt/g/packit/packit/plans/main.fmf
             /home/tt/g/packit/packit/plans/rpmlint.fmf

[FrNecas]

And also possibly a hint to non-verbose discover output?

[TomasTomecek]

hint++

that would be very helpful since I had no idea discover is so powerful (which again can be mitigated if I read the docs more carefully)

[psss]

Hmmm, I'm not completely convinced that it would be good to pollute the default output with many hints. At least until we have a config implemented where user could specify level of hinting or disable the feature completely. Showing hints in verbose mode sounds better. What definitely makes sense to me is to cover the As a tester I want to see which tests would be run use case in the guide.