Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

allure commandline: support building report when results and categories.json are in different folders #1236

Open
vlsi opened this issue Apr 20, 2021 · 1 comment
Labels
type:new feature Change that add something new for end users

Comments

@vlsi
Copy link
Contributor

vlsi commented Apr 20, 2021

Is your feature request related to a problem? Please describe.

Currently bin/allure requires that both simple-result.json and categories.json are in the same folder.
If I put the files into different folders and pass both of them to bin/allure serve, then the report does not pick up the categories.

The problem of having different data in the same folder looks like "overlapping outputs" to the build systems.
Overlapping outputs incur extra burden on builds systems (e.g. https://docs.gradle.org/current/userguide/build_cache_concepts.html#concepts_overlapping_outputs)

Describe the solution you'd like

I would like to provide an explicit pair of "results" and "categories", so allure could build the report accordingly.

For instance: ./allure serve --results results1 --categories path/to/categories.json --results results2 --categories another/categories.json
An alternative option is to prepare the description in a standalone JSON file and pass it to ./allure serve --input inputs.json

Describe alternatives you've considered

An alternative would be producing files into different folders and create yet another folder that combines all the files right before calling allure. However, it would require extra file copy operation.

@vlsi vlsi added the type:new feature Change that add something new for end users label Apr 20, 2021
@vlsi
Copy link
Contributor Author

vlsi commented Apr 21, 2021

Yet more ideas: what if categories.json file had "applicability" information associated with the categories?
In other words:

  1. executor.json lists the metadata on the executor side. For instance, it could be test type, build system type, etc, etc
  2. categories.json could specify that "if test_type=e2e, then the following categories apply: ..."

The benefits would be:
a) There would be no reason to copy categories.json into every folder with test data
b) categories.json files could be combined, and they would still behave well. For instance, if a certain categories classifies only a subset of executors, then it is fine. The key plus here is that we no longer need to maintain relationship between "test data results" and "categories files". The command might be way simpler: ./allure serve --categories path/to/categories.json results1 results2
c) There might be globally-shared categories that might even be published as artifacts (e.g. io.qameta.allure.categories:allure-selenide:1.2). It would be so much fun if Allure could automatically classify "Selenium known issues" :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:new feature Change that add something new for end users
Projects
None yet
Development

No branches or pull requests

1 participant