V23.4 external release (#67)

* v23.4 external release
intel · Dec 22, 2023 · 6bf7b07 · 6bf7b07
1 parent c6388bb
commit 6bf7b07
Show file tree

Hide file tree

Showing 452 changed files with 314,706 additions and 2,407 deletions.
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -123,6 +123,7 @@ if(BACKEND)
     include(${BACKEND} OPTIONAL)
 endif()
 set(IMAGEARCH "linux/amd64")
+set(IMAGESUFFIX "")
 include(multi-arch OPTIONAL)
 
 foreach(subdir image stack workload dist)

diff --git a/doc/user-guide/preparing-infrastructure/setup-containerd.md b/doc/user-guide/preparing-infrastructure/setup-containerd.md
@@ -45,6 +45,18 @@ EOF
 sudo systemctl restart containerd
 ```
 
+On-Premises workload validation based on RKE2 requires to use a docker registry. If you need to setup any insecure registries with `containerd`, modify the `rke2` configuration as follows, assuming your private registry is `foo.com:5000`:   
+
+```shell
+cat | sudo tee /etc/rancher/rke2/registries.yaml <<EOF
+mirrors:
+  "foo.com:5000":
+    endpoint:
+      - "http://foo.com:5000"
+EOF 
+sudo systemctl restart rke2-server
+```
+
 ## Setup Data Storage
 
 Optionally, if you need to move the containerd storage location to, for example, `/mnt/storage/containerd`:

diff --git a/doc/user-guide/preparing-infrastructure/setup-dlb.md b/doc/user-guide/preparing-infrastructure/setup-dlb.md
@@ -0,0 +1,24 @@
+
+# DLB Setup
+
+DLB is supported in certain SPR SKUs. Please make sure your CPU sku (QDF) supports DLB.
+
+And DLB is broken with latest BKC kernel `5.15.0-spr.bkc.pc.2.10.0.x86_64`.
+Before running the workload, please make sure your SPR has DLB device by running the following command:
+
+```shell
+lspci | grep 2710
+```
+
+If there are devices listed, then please download the DLB driver from this link: https://www.intel.com/content/www/us/en/download/686372/intel-dynamic-load-balancer.html
+
+Execute the following commands:
+
+```shell
+tar -xf dlb_linux_src_release_<dlb_driver_version>.txz
+cd dlb/driver/dlb2/
+make
+sudo insmod dlb2.ko
+```
+
+Then you can run the workload on this machine.
diff --git a/doc/user-guide/preparing-infrastructure/setup-nginx-cache.md b/doc/user-guide/preparing-infrastructure/setup-nginx-cache.md
@@ -0,0 +1,65 @@
+# Nginx Cache Server Setup
+
+This document is a guide for setting up Nginx-DLB benchmark environment, including Hardware platform and Software configuration on storage.
+
+## HW Prerequisites
+
+- Setup 3 servers: one node (client) for client deployment; 2 node for cache server (worker-0) and content server (worker-1) deployment.
+- Cache server (worker-0) requires 5 NVMe disks.
+
+## K8S Labels configuaration
+
+Please finish the section, [Storage configuration](setup-nginx-cache.md#storage-configuration), then label the corresponding nodes.
+
+Command examples:
+
+- Label:
+  ```shell
+  kubectl label node <node name> HAS-SETUP-NVMECACHE=yes
+  ```
+- Unlabel:
+  ```shell
+  kubectl label node <node name> HAS-SETUP-NVMECACHE-
+  ```
+
+Nginx cache server worker-0:*
+
+- `HAS-SETUP-NGINX-CACHE=yes`
+
+## Storage configuration
+
+This should be done on worker-0.
+
+- Prepare 5 nvme disk for nginx cache server pod. *nvme?* means multiple nvme disk.
+
+  - Check NVMe drives and Partition drives
+    ```shell command
+    ls /dev/nvme*
+    ```
+
+    ```output
+    /dev/nvme1    /dev/nvme2    /dev/nvme3    /dev/nvme4    /dev/nvme5
+    ```
+
+  - Format drives as ext4 (or xfs):
+    ```shell command
+    mkfs.ext4 /dev/nvme1n1
+    mkfs.ext4 /dev/nvme2n1
+    mkfs.ext4 /dev/nvme3n1
+    mkfs.ext4 /dev/nvme4n1
+    mkfs.ext4 /dev/nvme5n1
+    ```
+
+  - Create cache mountpoints and mount to four pairs
+    ```shell command
+    mkdir /nginx/cache1
+    mount -o rw,noatime,seclabel,discard /dev/nvme1n1 /nginx/cache1
+    mkdir /nginx/cache2
+    mount -o rw,noatime,seclabel,discard /dev/nvme2n1 /nginx/cache2
+    mkdir /nginx/cache3
+    mount -o rw,noatime,seclabel,discard /dev/nvme3n1 /nginx/cache3
+    mkdir /nginx/cache4
+    mount -o rw,noatime,seclabel,discard /dev/nvme4n1 /nginx/cache4
+    mkdir /nginx/cache5
+    mount -o rw,noatime,seclabel,discard /dev/nvme5n1 /nginx/cache5
+    ```
diff --git a/doc/user-guide/preparing-infrastructure/setup-terraform.md b/doc/user-guide/preparing-infrastructure/setup-terraform.md
@@ -180,5 +180,4 @@ $ exit
 [WSF Cloud Setup]: setup-wsf.md#cloud-development-setup
 [WSF On-Premises Setup]: setup-wsf.md#on-premises-development-setup
 [terraform-config.static.tf]: ../../../script/terraform/terraform-config.static.tf
-[WSF KVM Setup]: setup-wsf.md#kvm-development-setup
 [Trace Module]: ../executing-workload/terraform-options.md#trace-module-parameters
diff --git a/doc/user-guide/preparing-infrastructure/setup-wsf.md b/doc/user-guide/preparing-infrastructure/setup-wsf.md
@@ -19,7 +19,7 @@ flowchart TD;
   net <--> sut3;;
 ```
 
-where the SUT machines can be physical hosts in the On-Premesis case, or virtualized VMs in the Cloud or KVM execution. If Kubernetes is used, it is assumed that the SUT hosts form a Kubernetes cluster, where one of the SUTs be the Kubernetes controller.  
+where the SUT machines can be physical hosts in the On-Premesis case, or virtualized VMs in the Cloud. If Kubernetes is used, it is assumed that the SUT hosts form a Kubernetes cluster, where one of the SUTs be the Kubernetes controller.  
 
 There can be many variations of the above diagram:
 - Combine dev and the Kubernetes controller on the same machine.  
@@ -167,18 +167,16 @@ Use the following setup steps:
 
 - Use `cmake -DREGISTRY=<registry_url> ..` to set the private registry URL.
 - Set `k8s_enable_registry: false` in `script/terraform/terraform-config.static.tf`. This is the default.
-- If the host has enough CPU cores and memory, suggest to use the KVM development setup for more flexibility.
 
 ---
 
 ## Setup Scripts
-> Note: All scripts mentioned below, such as setup-dev.sh, setup-sut-kvm.sh, etc, are found inside ```script/setup/``` and must be executed from there.
+> Note: All scripts mentioned below, such as setup-dev.sh, etc, are found inside ```script/setup/``` and must be executed from there.
 - **[`setup-dev.sh`][setup-dev.sh-self]**: Setup the dev host.   
 - **[`setup-reg.sh`][setup-reg.sh-self]**: Setup a private docker registry.   
 - **[`setup-sut-native.sh`][setup-sut-native.sh-self]**: Setup the SUT host for native workload execution.  
 - **[`setup-sut-docker.sh`][setup-sut-docker.sh-self]**: Setup the SUT host for docker/docker-compose workload execution.  
 - **[`setup-sut-k8s.sh`][setup-sut-k8s.sh-self]**: Setup the SUT host for Kubernetes workload execution.  
-- **[`setup-sut-kvm.sh`][setup-sut-kvm.sh-self]**: Setup the KVM host.  
 
 ### setup-dev.sh
 
@@ -266,7 +264,6 @@ where `options` are:
 
 [Instructions of Cloud Setup]: #cloud-setup
 [Instructions of On-Premises Setup]: #on-premises-setup
-[Instructions of KVM Setup]: #kvm-setup
 [Manual of Setup Scripts]: #setup-scripts
 [Prerequisites]: #prerequisites
 [RFC-1178]: http://www.faqs.org/rfcs/rfc1178.html

diff --git a/script/benchmark/ctest.sh b/script/benchmark/ctest.sh
@@ -121,7 +121,7 @@ if [ -n "$stop" ]; then
     if [ "$($cmd | wc -l)" -ne 2 ] && [ "$stop" != "--stop=all" ]; then
         echo "None or multiple ctest instances detected:"
         echo ""
-        $cmd --format '{{.Names}}\t\t{{.ID}}\t{{.Status}}'
+        $cmd --format '{{.Names}}\t{{.Status}}'
         echo ""
         echo "Please identify the instance with: ./ctest.sh --stop=<prefix> or ./ctest.sh --stop=all"
         exit 3
@@ -255,7 +255,7 @@ for var in "$@"; do
     --check-docker-image)
         export CTESTSH_OPTIONS="$CTESTSH_OPTIONS --check-docker-image"
         ;;
-    --push-docker-image=)
+    --push-docker-image=*)
         export CTESTSH_OPTIONS="$CTESTSH_OPTIONS --push-docker-image=${var#--push-docker-image=}"
         ;;
     --push-docker-image)

diff --git a/script/benchmark/kpi-list.awk b/script/benchmark/kpi-list.awk
@@ -5,6 +5,9 @@
 # SPDX-License-Identifier: Apache-2.0
 #
 
+BEGIN {
+    status="failed"
+}
 /^#svrinfo[:-] / {
     if (!svrinfo) next
 }
@@ -14,7 +17,10 @@
 {   
     print $0
 }
-/^[*].*: *([0-9.-][0-9.e+-]*) *#*.*$/ {
+/^# status: (passed|failed)/ {
+    status=$3
+}
+/^[*].*: *([0-9.-][0-9.e+-]*) *#*.*$/ && status=="passed" {
     k=gensub(/^(.*): *[0-9.-][0-9.-]*.*$/,"\\1",1,$0)
     v=gensub(/^.*: *([0-9.-][0-9.-]*).*$/,"\\1",1,$0)
     n[k]=n[k]+1

diff --git a/script/benchmark/kpi-xls-ai.awk b/script/benchmark/kpi-xls-ai.awk
@@ -26,6 +26,11 @@ function get_value() {
 /\/itr-[0-9]*:$/{
     name=gensub(/^.*logs-([^/]*)[/].*$/,"\\1",1)
     itr=gensub(/^.*[/]itr-([0-9]+):$/,"\\1",1)
+    status="failed"
+}
+
+/^# status: (passed|failed)/ {
+    status=$3
 }
 
 index($0,var1)==1 || ($1=="#" && index($2,var1)==1) {
@@ -36,14 +41,14 @@ index($0,var2)==1 || ($1=="#" && index($2,var2)==1) {
     var2v=gensub(/.*"(.*)".*/,"\\1",1,$NF)
 }
 
-index($0,var3)==1 {
+index($0,var3)==1 && status=="passed" {
     var3v[name][product][var1v][var2v][++var3vct[name][product][var1v][var2v]]=get_value()
     n=length(var3v[name][product][var1v][var2v])
     if (n>var34n[name][product][var1v][var2v])
         var34n[name][product][var1v][var2v]=n
 }
 
-index($0,var4)==1 {
+index($0,var4)==1 && status=="passed" {
     idx=gensub(/ *([0-9]+).*$/,"\\1",1,substr($0,length(var4)+1))
     var4v[name][product][var1v][var2v][idx][++var4vct[name][product][var1v][var2v][idx]]=get_value()
     n=length(var4v[name][product][var1v][var2v][idx])

diff --git a/script/benchmark/kpi-xls-inst.awk b/script/benchmark/kpi-xls-inst.awk
@@ -12,9 +12,14 @@
 
 /\/itr-[0-9]*:$/ {
     name=gensub(/^.*logs-([^/]*)[/].*$/,"\\1",1)
+    status="failed"
 }
 
-/^[^#].*: *[0-9.-][0-9.e+-]* *#?.*$/ {
+/^# status: (passed|failed)/ {
+    status=$3
+}
+
+/^[^#].*: *[0-9.-][0-9.e+-]* *#?.*$/ && status=="passed" {
     k=gensub(/^(.*): *[0-9.-]+.*$/, "\\1", 1)
     v=gensub(/^.*: *([0-9.-]+).*/, "\\1", 1)
     kpis[name][product][k][++kpisct[name][product][k]]=v
@@ -90,12 +95,14 @@ END {
             print "</Row>"
         }
 
-        add_svrinfo_ex(ws, psp, ith)
+        if (length(svrinfo_values[ws])>0)
+            add_svrinfo_ex(ws, psp, ith)
 
         print "</Table>"
         print "</Worksheet>"
 
-        add_svrinfo(ws)
+        if (length(svrinfo_values[ws])>0)
+            add_svrinfo(ws)
     }
     print "</Workbook>"
 }
diff --git a/script/benchmark/kpi-xls-table.awk b/script/benchmark/kpi-xls-table.awk
@@ -21,6 +21,11 @@ BEGIN {
 
 /[/]itr-[0-9]*:$/ {
     name=gensub("^.*logs-([^/]*)[/].*$","\\1",1)
+    status="failed"
+}
+
+/^# status: (passed|failed)/ {
+    status=$3
 }
 
 index($0,var1)==1 || ($1=="#" && index($2,var1)==1) {
@@ -39,7 +44,7 @@ index($0,var2)==1 || ($1=="#" && index($2,var2)==1) {
     var4v=var4": "gensub(/"/,"","g",$NF)
 }
 
-/^[*].*: *([0-9.-][0-9.e+-]*) *#*.*$/ {
+/^[*].*: *([0-9.-][0-9.e+-]*) *#*.*$/ && status=="passed" {
     primary_kpi[name]=gensub(/^.*: *([0-9.-][0-9.-]*).*$/,"\\1",1,$0)
     var34v=""
     if (length(var3)>0) var34v=var3v

diff --git a/script/benchmark/list-kpi.sh b/script/benchmark/list-kpi.sh
@@ -25,6 +25,19 @@ print_help () {
     exit 0
 }
 
+get_status_code () {
+  local status_ret=1
+  local status_value=0
+  for status_path in "$1"/*/status; do
+      if [ -e "$status_path" ]; then
+          status_value="$(< "$status_path")"
+          [ "$status_value" -eq 0 ] || return 1
+          status_ret=0
+      fi
+  done
+  return $status_ret
+}
+
 prefixes=()
 primary=1
 outlier=0
@@ -134,7 +147,7 @@ for var in "$@"; do
             tags="$var"
             ;;
         *)
-            prefixes+=("$var")
+            prefixes+=("${var%/}")
             ;;
         esac
     esac
@@ -176,7 +189,7 @@ if [ $intel_publish = 1 ]; then
                 BACKEND_OPTIONS="$BACKEND_OPTIONS --tags=$tags"
             fi
 
-            TERRAFORM_OPTIONS="$BACKEND_OPTIONS" RELEASE="$RELEASE" REGISTRY="$REGISTRY" "$DIR"/../terraform/shell.sh static -v "$(readlink -f "$logsdir1"):/opt/workspace" -- bash -c "/opt/terraform/script/publish-intel.py $BACKEND_OPTIONS < <(cat tfplan.json 2> /dev/null || echo)"
+            TERRAFORM_OPTIONS="$BACKEND_OPTIONS" RELEASE="$RELEASE" REGISTRY="$REGISTRY" "$DIR"/../terraform/shell.sh static -v "$(readlink -f "$logsdir1"):/opt/workspace" -- bash -c "/opt/terraform/script/publish-intel.py $BACKEND_OPTIONS < <(cat tfplan.json 2> /dev/null || cat .tfplan.json 2> /dev/null || echo)"
         fi
     done
 fi
@@ -204,26 +217,27 @@ for logsdir1 in ${prefixes[@]}; do
         if [ -d "$logsdir1/itr-1" ]; then
             for itrdir1 in "$logsdir1"/itr-*; do
                 echo "$itrdir1:"
+                if get_status_code "$itrdir1"; then
+                    echo "# status: passed"
+                else
+                    echo "# status: failed"
+                fi
                 if [ $params -eq 1 ]; then
                     sed -n '/^tunables:/,/^[^ ]/{/^ /{s/^ */# /;p}}' "$logsdir1/workload-config.yaml"
                 fi
                 if [ -n "$primary" ]; then
-                    ( cd "$itrdir1" && bash ./kpi.sh $script_args 2> /dev/null | grep -E "^\*" ) || true
+                    ( cd "$itrdir1" && bash ./kpi.sh $script_args 2> /dev/null | sed -n '/^[*]/{s/#.*//;p}') || true
                 else
-                    ( cd "$itrdir1" && bash ./kpi.sh $script_args 2> /dev/null ) || true
+                    ( cd "$itrdir1" && bash ./kpi.sh $script_args 2> /dev/null) || true
                 fi
             done
         else
             echo "$logsdir1:"
-            if [ $params -eq 1 ]; then
-                sed -n '/^tunables:/,/^[^ ]/{/^ /{s/^ */# /;p}}' "$logsdir1/workload-config.yaml"
-            fi
-            if [ -n "$primary" ]; then
-                ( cd "$logsdir1" && bash ./kpi.sh $script_args 2> /dev/null | grep -E "^\*" ) || true
-            else
-                ( cd "$logsdir1" && bash ./kpi.sh $script_args 2> /dev/null ) || true
-            fi
+            echo "# status: failed"
         fi
+    else
+        echo "$logsdir1:"
+        echo "# status: failed"
     fi
     if [ -r "$logsdir1/publish.logs" ] && [ "$uri" -eq 1 ]; then
         sed -n '/WSF Portal URL:/{s/^[^:]*:/# portal:/;p;q}' "$logsdir1/publish.logs"