Add jenkins job for Windows OVS containerization

Add new jobs to run e2e, conformance and networkpolicy tests on testbed
with Containerd runtime and OVS running inside container.

Signed-off-by: Naman Agarwal <naman.agarwal75@gmail.com>
Signed-off-by: Pulkit Jain <pulkit.jain@broadcom.com>

ci/jenkins/README.md

@@ -85,70 +85,34 @@ DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
 ./ci/jenkins/test.sh --testcase networkpolicy --registry ${DOCKER_REGISTRY}
 ```
 
-* windows e2e: e2e tests in a Windows setup with Docker runtime.
+* Windows e2e: e2e tests in a Windows setup.
 
 ```shell
 #!/bin/bash
 DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
 ./ci/jenkins/test.sh --testcase windows-e2e --registry ${DOCKER_REGISTRY}
 ```
 
-* windows conformance: community tests on Windows cluster with Docker runtime, focusing on "Conformance|sig-windows" and
+* Windows conformance: community tests on Windows cluster
+  with OVS daemons running inside a container, focusing on "conformance|sig-windows" and
   "sig-network", and skipping "LinuxOnly", "Slow", "Serial", "Disruptive", "Flaky", "Feature", "sig-cli", "sig-storage",
-  "sig-auth", "sig-api-machinery", "sig-apps", "sig-node", "Privileged", "should be able to change the type from",
-  "[sig-network] Services should be able to create a functioning NodePort service [Conformance]", "Service endpoints
-  latency should not be very high".
+  "sig-auth", "sig-api-machinery", "sig-apps", "sig-node", "Privileged".
 
 ```shell
 #!/bin/bash
 DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
 ./ci/jenkins/test.sh --testcase windows-conformance --registry ${DOCKER_REGISTRY}
 ```
 
-* windows network policy: community tests on Windows cluster with Docker runtime, focusing on "Feature:NetworkPolicy".
+* Windows network policy: community tests on Windows cluster,
+  with OVS daemons running inside a container focusing on "Feature:NetworkPolicy".
 
 ```shell
 #!/bin/bash
 DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
 ./ci/jenkins/test.sh --testcase windows-networkpolicy --registry ${DOCKER_REGISTRY}
 ```
 
-* windows e2e with proxyAll enabled: e2e tests in a Windows setup with proxyAll enabled.
-
-```shell
-#!/bin/bash
-DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-./ci/jenkins/test.sh --testcase windows-e2e --registry ${DOCKER_REGISTRY} --proxyall
-```
-
-* windows containerd e2e: e2e tests in a Windows setup with containerd runtime.
-
-```shell
-#!/bin/bash
-DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-./ci/jenkins/test.sh --testcase windows-e2e --registry ${DOCKER_REGISTRY} --win-image-node {antrea_win_image_node_name}
-```
-
-* windows containerd conformance: community tests on Windows cluster with containerd runtime, focusing on "Conformance|sig-windows" and
-  "sig-network", and skipping "LinuxOnly", "Slow", "Serial", "Disruptive", "Flaky", "Feature", "sig-cli", "sig-storage",
-  "sig-auth", "sig-api-machinery", "sig-apps", "sig-node", "Privileged", "should be able to change the type from",
-  "[sig-network] Services should be able to create a functioning NodePort service [Conformance]", "Service endpoints
-  latency should not be very high".
-
-```shell
-#!/bin/bash
-DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-./ci/jenkins/test.sh --testcase windows-conformance --registry ${DOCKER_REGISTRY} --win-image-node {antrea_win_image_node_name}
-```
-
-* windows containerd network policy: community tests on Windows cluster with containerd runtime, focusing on "Feature:NetworkPolicy".
-
-```shell
-#!/bin/bash
-DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-./ci/jenkins/test.sh --testcase windows-networkpolicy --registry ${DOCKER_REGISTRY} --win-image-node {antrea_win_image_node_name}
-```
-
 * Multicast e2e: e2e tests in a multicast cluster
 
 ```shell
@@ -333,5 +297,5 @@ updated with new code.
 
 ## Tips for Developer
 
-* [macro.yaml](jobs/macros.yaml): Use "{{}}" instead of "{}" to escape the "{" in "builder-list-tests", "builder-conformance" and "builder-*-win-containerd" because the macro has parameters.
+* [macro.yaml](jobs/macros.yaml): Use "{{}}" instead of "{}" to escape the "{" in "builder-list-tests", "builder-conformance" and "builder-*-win" because the macro has parameters.
 * While setting up the Rancher testbed, delete the cattle-cluster-agent deployment and use cattle-node-agent because cluster-agent adds extra watchers for all the resources. Antrea Controller counts connected Antrea Agent from watcher connections. Extra watchers lead to wrong Antrea Agent number in AntreaControllerInfo CR.

ci/jenkins/jobs/macros.yaml

@@ -122,31 +122,22 @@
           ./ci/jenkins/test.sh --testcase windows-install-ovs
 
 - builder:
-    name: builder-e2e-proxyall-win
+    name: builder-e2e-win
     builders:
       - shell: |-
           #!/bin/bash
           set -ex
           DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-          ./ci/jenkins/test.sh --testcase windows-e2e --registry ${DOCKER_REGISTRY} --proxyall
+          ./ci/jenkins/test.sh --testcase '{e2e_type}' --registry ${{DOCKER_REGISTRY}}
 
 - builder:
-      name: builder-e2e-win
-      builders:
-          - shell: |-
-                #!/bin/bash
-                set -ex
-                DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-                ./ci/jenkins/test.sh --testcase windows-containerd-e2e --registry ${{DOCKER_REGISTRY}} --win-image-node '{win_image_node}'
-
-- builder:
-      name: builder-conformance-win
-      builders:
-          - shell: |-
-                #!/bin/bash
-                set -ex
-                DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
-                ./ci/jenkins/test.sh --testcase '{conformance_type}' --registry ${{DOCKER_REGISTRY}} --win-image-node '{win_image_node}'
+    name: builder-conformance-win
+    builders:
+      - shell: |-
+          #!/bin/bash
+          set -ex
+          DOCKER_REGISTRY="$(head -n1 ci/docker-registry)"
+          ./ci/jenkins/test.sh --testcase '{conformance_type}' --registry ${{DOCKER_REGISTRY}}
 
 - builder:
     name: builder-flexible-ipam-e2e

ci/jenkins/jobs/projects-lab.yaml

@@ -1,6 +1,6 @@
 - project:
-    # ghpr_auth, antrea_admin_list, antrea_org_list, antrea_white_list and
-    # antrea_win_image_node_name should be defined as a global variable somewhere else
+    # ghpr_auth, antrea_admin_list, antrea_org_list, antrea_white_list
+    # should be defined as global variables somewhere else
     name: antrea
     git_credentials_id: ANTREA_GIT_CREDENTIAL
     org_repo: antrea-io/antrea
@@ -575,7 +575,7 @@
           wrappers: []
           publishers: []
       - '{name}-{test_name}-no-scm':
-          test_name: windows-e2e-pending-label
+          test_name: windows-e2e-with-ovs-pending-label
           node: null
           description: 'This is for marking PR as pending for e2e test.'
           builders:
@@ -590,25 +590,24 @@
           trigger_permit_all: true
           throttle_concurrent_builds_category:
           throttle_concurrent_builds_enabled: 'false'
-          status_context: jenkins-windows-e2e
+          status_context: jenkins-windows-e2e-with-ovs
           status_url: --none--
-          success_status: Pending test. Mark as failure. Add comment /test-windows-e2e to trigger.
-          failure_status: Pending test. Mark as failure. Add comment /test-windows-e2e to trigger.
-          error_status: Pending test. Mark as failure. Add comment /test-windows-e2e to trigger.
+          success_status: Pending test. Mark as failure. Add comment /test-windows-e2e-with-ovs to trigger.
+          failure_status: Pending test. Mark as failure. Add comment /test-windows-e2e-with-ovs to trigger.
+          error_status: Pending test. Mark as failure. Add comment /test-windows-e2e-with-ovs to trigger.
           triggered_status: null
           started_status: null
           wrappers: []
           publishers: []
       - '{name}-{test_name}-for-pull-request':
             test_name: windows-conformance
-            node: 'antrea-windows-containerd-testbed'
+            node: 'antrea-windows-testbed'
             description: 'This is the {test_name} test for {name}.'
             branches:
                 - ${{sha1}}
             builders:
                 - builder-conformance-win:
-                      conformance_type: 'windows-containerd-conformance'
-                      win_image_node: '{antrea_win_image_node_name}'
+                      conformance_type: 'windows-conformance'
             trigger_phrase: ^(?!Thanks for your PR).*/test-windows-(conformance|all).*
             white_list_target_branches: [ ]
             allow_whitelist_orgs_as_admins: true
@@ -658,14 +657,13 @@
                       only-if-success: false
       - '{name}-{test_name}-for-pull-request':
             test_name: windows-networkpolicy
-            node: 'antrea-windows-containerd-testbed'
+            node: 'antrea-windows-testbed'
             description: 'This is the {test_name} test for {name}.'
             branches:
                 - ${{sha1}}
             builders:
                 - builder-conformance-win:
-                      conformance_type: 'windows-containerd-networkpolicy'
-                      win_image_node: '{antrea_win_image_node_name}'
+                      conformance_type: 'windows-networkpolicy'
             trigger_phrase: ^(?!Thanks for your PR).*/test-windows-(networkpolicy|all).*
             white_list_target_branches: [ ]
             allow_whitelist_orgs_as_admins: true
@@ -713,16 +711,88 @@
                       default-excludes: true
                       fingerprint: false
                       only-if-success: false
+      - '{name}-{test_name}-no-scm':
+          test_name: windows-e2e-skip
+          node: null
+          description: 'This is for marking PR as passed.'
+          branches:
+          - ${{sha1}}
+          builders: []
+          allow_whitelist_orgs_as_admins: true
+          admin_list: '{antrea_admin_list}'
+          org_list: '{antrea_org_list}'
+          white_list: '{antrea_white_list}'
+          only_trigger_phrase: true
+          trigger_permit_all: false
+          trigger_phrase: ^(?!Thanks for your PR).*/skip-(windows-e2e-with-ovs|all).*
+          white_list_target_branches: []
+          status_context: jenkins-windows-e2e-with-ovs
+          status_url: --none--
+          success_status: Skipped test. Mark as succeeded.
+          failure_status: Skipped test. Mark as succeeded.
+          error_status: Skipped test. Mark as succeeded.
+          triggered_status: null
+          started_status: null
+          wrappers: []
+          publishers: []
+      - '{name}-{test_name}-no-scm':
+          test_name: windows-conformance-skip
+          node: null
+          description: 'This is for marking PR as passed.'
+          branches:
+          - ${{sha1}}
+          builders: []
+          allow_whitelist_orgs_as_admins: true
+          admin_list: '{antrea_admin_list}'
+          org_list: '{antrea_org_list}'
+          white_list: '{antrea_white_list}'
+          only_trigger_phrase: true
+          trigger_permit_all: false
+          trigger_phrase: ^(?!Thanks for your PR).*/skip-(windows-conformance|all).*
+          white_list_target_branches: []
+          status_context: jenkins-windows-conformance
+          status_url: --none--
+          success_status: Skipped test. Mark as succeeded.
+          failure_status: Skipped test. Mark as succeeded.
+          error_status: Skipped test. Mark as succeeded.
+          triggered_status: null
+          started_status: null
+          wrappers: []
+          publishers: []
+      - '{name}-{test_name}-no-scm':
+          test_name: windows-networkpolicy-skip
+          node: null
+          description: 'This is for marking PR as passed.'
+          branches:
+          - ${{sha1}}
+          builders: []
+          allow_whitelist_orgs_as_admins: true
+          admin_list: '{antrea_admin_list}'
+          org_list: '{antrea_org_list}'
+          white_list: '{antrea_white_list}'
+          only_trigger_phrase: true
+          trigger_permit_all: false
+          trigger_phrase: ^(?!Thanks for your PR).*/skip-(windows-networkpolicy|all).*
+          white_list_target_branches: []
+          status_context: jenkins-windows-networkpolicy
+          status_url: --none--
+          success_status: Skipped test. Mark as succeeded.
+          failure_status: Skipped test. Mark as succeeded.
+          error_status: Skipped test. Mark as succeeded.
+          triggered_status: null
+          started_status: null
+          wrappers: []
+          publishers: []         
       - '{name}-{test_name}-for-pull-request':
-            test_name: windows-e2e
-            node: 'antrea-windows-containerd-testbed'
+            test_name: windows-e2e-with-ovs
+            node: 'antrea-windows-testbed'
             description: 'This is the {test_name} test for {name}.'
             branches:
                 - ${{sha1}}
             builders:
                 - builder-e2e-win:
-                      win_image_node: '{antrea_win_image_node_name}'
-            trigger_phrase: ^(?!Thanks for your PR).*/test-windows-(e2e|all).*
+                      e2e_type: 'windows-e2e-with-ovs'    
+            trigger_phrase: ^(?!Thanks for your PR).*/test-windows-(e2e-with-ovs|all).*
             white_list_target_branches: [ ]
             allow_whitelist_orgs_as_admins: true
             admin_list: '{antrea_admin_list}'
@@ -733,11 +803,11 @@
             throttle_concurrent_builds_category:
                 - e2e-lock-per-testbed
             throttle_concurrent_builds_enabled: 'true'
-            status_context: jenkins-windows-e2e
+            status_context: jenkins-windows-e2e-with-ovs
             status_url: --none--
             success_status: Build finished.
-            failure_status: Failed. Add comment /test-windows-e2e to re-trigger.
-            error_status: Failed. Add comment /test-windows-e2e to re-trigger.
+            failure_status: Failed. Add comment /test-windows-e2e-with-ovs to re-trigger.
+            error_status: Failed. Add comment /test-windows-e2e-with-ovs to re-trigger.
             triggered_status: null
             started_status: null
             wrappers:
@@ -769,6 +839,49 @@
                       default-excludes: true
                       fingerprint: false
                       only-if-success: false
+      - '{name}-{test_name}-for-period':
+          test_name: windows-e2e
+          node: 'antrea-windows-testbed'
+          description: 'This is for running e2e tests with OVS daemons running as Windows services.'
+          builders:
+            - builder-e2e-win:
+                e2e_type: 'windows-e2e'
+          branches:
+          - '*/main'
+          included_regions: []
+          cron: 'H H * * *'
+          ignore_post_commit_hooks: false
+          publishers:
+          - email:
+              recipients: projectantrea-dev@googlegroups.com
+          - archive:
+              allow-empty: true
+              artifacts: 'antrea-test-logs.tar.gz'
+              case-sensitive: true
+              default-excludes: true
+              fingerprint: false
+              only-if-success: false
+          wrappers:
+          - timeout:
+              fail: true
+              timeout: 135
+              type: absolute
+          - credentials-binding:
+            - text:
+                credential-id: GOVC_URL
+                variable: GOVC_URL
+            - text:
+                credential-id: GOVC_USERNAME
+                variable: GOVC_USERNAME
+            - text:
+                credential-id: GOVC_PASSWORD
+                variable: GOVC_PASSWORD
+            - text:
+                credential-id: GOVC_DATACENTER
+                variable: GOVC_DATACENTER
+            - text:
+                credential-id: GOVC_DATASTORE
+                variable: GOVC_DATASTORE
       - '{name}-{test_name}-for-pull-request':
           test_name: kind-conformance
           node: 'antrea-kind-testbed'