[Cross version tests] Run tests for dev versions when the `enable-dev…

…-tests` label is applied (mlflow#4533)

* detect enable-dev-tests label

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* Fix syntax error

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* use github-script to check label is applied

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* fix parameters

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* fix unrecognized arguments

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* trigger sklearn tests

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* update README

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* update README

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* add if and name

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

* revert dummy change

Signed-off-by: harupy <17039389+harupy@users.noreply.github.com>

.github/workflows/README.md

@@ -52,3 +52,9 @@ python dev/set_matrix.py --ref-versions-yaml $REF_VERSIONS_YAML --changed-files
 ```sh
 pytest dev/set_matrix.py --doctest-modules
 ```
+
+## How to run tests for dev versions on a pull request:
+
+1. Click `Labels` in the right sidebar.
+2. Select the `enable-dev-tests` label.
+3. Push a commit or re-run the `Cross version-tests` workflow.

.github/workflows/cross-version-tests.yml

@@ -23,6 +23,19 @@ jobs:
       - name: Install dependencies
         run: |
           pip install packaging pyyaml pytest
+      - name: Check labels
+        uses: actions/github-script@v4
+        id: enable-dev-tests
+        if: ${{ github.event_name == 'pull_request' }}
+        with:
+          result-encoding: string
+          script: |
+            const labels = await github.issues.listLabelsOnIssue({
+              owner: context.repo.owner,
+              repo: context.repo.repo,
+              issue_number: context.issue.number,
+            });
+            return labels.data.some(({ name }) => name === "enable-dev-tests");
       - name: Test set_matrix.py
         run: |
           pytest dev/set_matrix.py --doctest-modules
@@ -36,7 +49,9 @@ jobs:
           if [ "${{ github.event_name }}" = "pull_request" ]; then
             REF_VERSIONS_YAML="https://raw.githubusercontent.com/mlflow/mlflow/master/mlflow/ml-package-versions.yml"
             CHANGED_FILES="$(git diff --name-only master..HEAD)"
-            python dev/set_matrix.py --ref-versions-yaml "$REF_VERSIONS_YAML" --changed-files "$CHANGED_FILES" --exclude-dev-versions
+            ENABLE_DEV_TESTS="${{ steps.enable-dev-tests.outputs.result }}"
+            EXCLUDE_DEV_VERSIONS_FLAG=$([ "$ENABLE_DEV_TESTS" == "true" ] && echo "" || echo "--exclude-dev-versions")
+            python dev/set_matrix.py --ref-versions-yaml "$REF_VERSIONS_YAML" --changed-files "$CHANGED_FILES" $EXCLUDE_DEV_VERSIONS_FLAG
           else
             python dev/set_matrix.py
           fi