Merge pull request #2 from IhsaneTahir/testplan

First version of UVM testplan and README update

Author: MikeOpenHWGroup

.gitignore

@@ -0,0 +1,8 @@
+*.db
+*.bin
+*.log
+*.d
+*.so
+*.o
+transcript
+work/

.gitmodules

@@ -1,6 +1,6 @@
 [submodule "modules/core-v-verif"]
 	path = modules/core-v-verif
-	url  = https://github.com/openhwgroup/core-v-verif.git
+	url = https://github.com/openhwgroup/core-v-verif.git
 [submodule "modules/cva6"]
 	path = modules/cva6
 	url = https://github.com/openhwgroup/cva6.git

README.md

@@ -1,37 +1,69 @@
 # CVFPU UVM Testbench
 
 ## 1. Project Overview
-This repository contains a **UVM testbench** for verifying the **CVFPU (Core-V Floating Point Unit)**. DUT is the CVA6 wrapper of the floating point unit.
+This repository contains a UVM Verification Environment for the [CVFPU](https://github.com/openhwgroup/cvfpu.git). DUT is the CVA6 wrapper of the floating-point unit.
 
 ## 2. Testbench Architecture
 
 ![CVA6 Tile Platform](./images/cvfpu_uvm_tb_archi.png)
 
 ## 3. Project Structure
 The repository is organised as follows:
-- **env/** Contains UVM environment, scoreboard and top configuration class
-- **fpu_agent/** Contains UVM agent, driver, sequencer, monitor, interface, sequences and transaction
+- **docs/testplan/** Contains the testplan for the CVFPU UVM environment
+- **env/** Contains UVM environment, scoreboard and top configuration classes
+- **fpu_agent/** Contains UVM agent, driver, sequencer, monitor, interface, sequences and transaction item
 - **fpu_common/** Contains package used accross UVM testbench
-- **ref_model_csim/** Contains C++ reference model as well as SV wrapper
+- **modules/** Contains dependencies of the project: [cva6](https://github.com/openhwgroup/cva6.git) and [core-v-verif](https://github.com/openhwgroup/core-v-verif.git) repositories. Both are included as git submodules.
+- **ref_model_csim/** Contains C++ reference model as well as SystemVerilog wrapper
 - **simu/** Contains regression test list and yaml files needed to run simulation and regression scripts. It also holds log files
 - **tests/** Contains UVM test classes
-- **top/** Contains top-level testbench file.
+- **top/** Contains top-level testbench file
+- **scripts/** Contains `scan_logs.pl` perl script for parsing log files, reporting errors and warnings, and generating summary reports.
 
 ## 4. Getting started
-### 4.1. Compile C++ Reference Model 
-The first step is to build the shared library `refmodel_csim_lib.so` that will be used in the UVM testbench via DPI.
+### 4.1 Environment Setup
+Before building or running the project, you must configure your environment variables. The testbench requires QuestaSim simulator, so ensure the `QUESTA_PATH` variable is set to your Questa installation directory.
+
+**tcsh**
+```
+setenv QUESTA_PATH <questa_path> 
+   # ex: setenv QUESTA_PATH <path_to_your_install>/questasim/2025.1
+```
+**bash**
+```
+export QUESTA_PATH=<questa_path>
+```
+We also provide two scripts to setup the project environment, source the one that matches the shell of the machine.
+
+| Shell Type      | Command to Run         |
+| --------------- | -----------------------|
+| **csh/tcsh**    | `source setup_env.csh` |
+| **bash/zsh/sh** | `source setup_env.sh`  |
+
+Some of the testbench utilities (compilation, simulation and regression scripts) use Python. Dependencies are listed in `requirements.txt`.
+
+### 4.2. Compile C++ Reference Model
+Build the shared library `refmodel_csim_lib.so` used in the UVM testbench via DPI.
 
 #### Dependencies
 The following dependencies need to be installed in the system:
 
 - **GMP** (GNU Multiple Precision Arithmetic Library)
 - **MPFR** (Multiple Precision Floating-Point Reliable Library): Section [*2.1 How to Install*](https://www.mpfr.org/mpfr-current/mpfr.html) details the steps to follow to install the library, use preferably version **4.2.2**.
 
-Then, set GMP/MPFR directory path variables in the environment
+Set GMP/MPFR directory path variables in the environment.
+
+**tcsh**
+```
+setenv GMP_DIR <gmp_dir_absolute_path>
+setenv MPFR_DIR <mpfr_dir_absolute_path>
+```
+**bash**
 ```
-setenv GMP_DIR <gmp_dir_path>
-setenv MPFR_DIR <mpfr_dir_path>
+export GMP_DIR=<gmp_dir_absolute_path>
+export MPFR_DIR=<mpfr_dir_absolute_path>
 ```
+It is important to note that the reference model includes `dpiheader.h` file that is tool specific to QuestaSim.
 
 #### Compilation
 ```
@@ -40,43 +72,141 @@ make
 ```
 
 ### 4.2. Build and run simulation 
-#### Setup
-
-At the root of the project, set the following environment variables.
-```
-setenv QUESTA_PATH <questa_path> 
-   # ex: setenv QUESTA_PATH <path_to_your_install>/questasim/2025.1
-   which vsim: <path_to_your_install>/questasim/2025.1/bin/vsim
-setenv PATH ${QUESTA_PATH}/bin:$PATH
+#### Compile testbench
 ```
-Run the setup script to configure project paths.
-```
-source setup_env.csh
-```
-
-#### Build TB and run simulation
-1. Compile testbench
-```
-cd simu
+cd ${PROJECT_DIR}/simu/
 python3 ${SCRIPTS_DIR}/compile.py --yaml sim_questa.yaml
 ```
-2. Run a test
+#### Run a test
 
-The number of transactions is set by the variable `+NB_TXNS` (passed as simulation option) in the `sim_questa.yaml` script. It is currently fixed to 10 000.
+The number of transactions is set by the variable `+NB_TXNS` (passed as simulation option) in the `sim_questa.yaml` file. It is currently fixed to 10 000.
 ```
 python3 ${SCRIPTS_DIR}/run_test.py --yaml sim_questa.yaml --test_name <TEST_NAME> --seed <SEED> --debug <VERBOSITY>
 ```
 For example
 ```
 python3 ${SCRIPTS_DIR}/run_test.py --yaml sim_questa.yaml --test_name fpu_random_test --seed 1 --debug UVM_LOW
 ```
-Simulation logs can be found in the `output/` directory.
+Simulation logs can be found in the `output/` folder.
+
+#### Run a regression
+The regression suite is defined in the `simu/fpu_reg_list` file. Each line in this file specifies:
+- **Test Name:** The UVM test class to run.
+- **Number of Runs:** How many times to run that test, each one has a different randomly generated seed
+
+Edit this file to decrease/increase the number of runs. Example of 20 runs/test:
+```
+fpu_single_op_test 20
+fpu_random_test 20
+```
 
-3. Run a regression, `fpu_reg_list` contains the regression suite.
+Check the `testplan` for more details on the available tests.
 ```
 python3 ${SCRIPTS_DIR}/run_reg.py --yaml reg_questa.yaml --nthreads 3 --reg_list fpu_reg_list
 ```
-Regression logs can be found in the `regression` folder. To parse through them, run the following script which will return result of the tests with either PASS or FAIL.
+Regression logs can be found in the `regression/` folder. To parse through them, run the following script which will return result of the tests with either PASS or FAIL.
 ```
 scan_logs.pl -nowarn --pat ${PROJECT_DIR}/scripts/patterns/sim_patterns.pat --waiver ${PROJECT_DIR}/scripts/patterns/sim_waivers.pat regression/fpu_*_test_*.log
 ```
+
+> **Note:**
+> Some regression failures may currently be expected because of known bugs in the DUT. These are being tracked, check [CVFPU Issues](https://github.com/openhwgroup/cvfpu/issues) section to confirm whether it is a known bug or a new issue that should be reported.
+
+### 4.2. Known Limitations
+* The current verification environment targets the CVA6 core exclusively.
+* `RMM`, `ROD`, and `DYN` rounding modes have not been fully verified.
+* Vector floating-point operations are not supported by the current testbench.
+* Only `FP32` and `FP64` formats are thoroughly tested. Other formats are verified only within `F2F` (float-to-float) conversion operations.
+* For `F2I` (float-to-integer) and `I2F` (integer-to-float) operations, only `INT32` and `INT64` integer formats are tested.
+
+### 4.3 Adding a new test
+For users who are not familiar with UVM, here are some simple steps to follow to create a new test to verify a new feature.
+
+---
+#### Step 1: Create a New Sequence
+A **sequence** generates the transactions (stimulus) that will be driven to the DUT.
+
+Example: create a new sequence class in `fpu_sequences.svh` file:
+
+```systemverilog
+class my_feature_seq extends fpu_base_sequence;
+
+  `uvm_object_utils(my_feature_seq)
+
+  fpu_txn item;
+
+  function new(string name = "my_feature_seq");
+    super.new(name);
+  endfunction
+
+  virtual task body();
+    super.body();
+
+    item = fpu_txn::type_id::create("my_feature_item");
+
+    for (int i = 0; i < num_txn; i++) begin
+      // Randomize or constrain the transaction to target your feature
+      if (!item.randomize() with {
+          m_operation == FDIV;        // force divide operation
+          m_fmt       == FP32;        // force single-precision format
+          m_operand_a == opA;
+          m_operand_b == opB;
+          m_imm       == imm;
+          m_rm        == 0;      // force round to nearest, ties to even
+      }) begin
+        `uvm_fatal("SEQ", "Randomization failed");
+      end
+
+      start_item(item);
+      finish_item(item);
+    end
+  endtask
+endclass
+```
+You can change constraints to target the feature you want to verify. Sequence item fields to randomize are defined in `fpu_txn.svh` file.
+
+---
+#### Step 2: Create a New Test Class
+
+The test class instantiates and starts your new sequence.
+Create a new file under `tests/` called `my_feature_test.svh`:
+
+```systemverilog
+class my_feature_test extends base_test;
+
+  `uvm_component_utils(my_feature_test)
+
+  my_feature_seq m_seq;
+
+  function new(string name, uvm_component parent);
+    super.new(name, parent);
+  endfunction
+
+  virtual task pre_main_phase(uvm_phase phase);
+    m_seq = my_feature_seq::type_id::create("seq");
+    if(!$cast(base_sequence, m_seq)) 
+      `uvm_fatal("CAST FAILED", "Cannot cast base sequence");
+
+    super.pre_main_phase(phase);
+  endtask
+endclass
+```
+---
+#### Step3: Register the Test
+Add the new test to the `fpu_test_pkg.sv` file.
+```systemverilog
+  `include "my_feature_test.svh"
+```
+---
+#### Step 4: Run the Test
+From the `simu/` directory, compile and run the simulation by following directions detailed in section [4.2](#42-known-limitations).
+
+---
+
+#### Step 5: Check Results
+Review the log file `my_feature_test_1.log` located in the `output/` directory for `UVM_ERROR` messages. You can run the `scan_logs` script to help parse through the file for errors and warnings.
+
+Open the waveform file if needed to inspect DUT behavior.
+```
+visualizer example.db
+```

docs/testplan/cvfpu_uvm_testplan_v1.xlsx

@@ -34,9 +34,8 @@ BUILD_DIR    = build
 INCDIR_MPFR  = $(MPFR_DIR)/include
 LIBDIR_MPFR  = $(MPFR_DIR)/lib
 
-
-INCDIR_GMP   = $(MPFR_DIR)/include
-LIBDIR_GMP   = $(MPFR_DIR)/lib
+INCDIR_GMP   = $(GMP_DIR)/include
+LIBDIR_GMP   = $(GMP_DIR)/lib
 
 INCDIRS      = $(addprefix -I,. $(QUESTA_PATH)/include $(INCDIR_MPFR) $(INCDIR_GMP) $(INCDIR_SOFTFLOAT) $(INC_DIR))
 LIBDIRS      = $(addprefix -L,$(LIBDIR_MPFR) $(LIBDIR_GMP) $(LIBDIR_SOFTFLOAT))

modules/core-v-verif

@@ -0,0 +1 @@
+pyyaml

modules/cva6

@@ -53,12 +53,15 @@ else
    setenv PERL5LIB "${PROJECT_DIR}/scripts/perl5"
 endif
 
+################################################
 # TOOLS
-# This is mainly used by the scan_log.pl tool
-echo $C_MSG">> Setting scripts"$C_RST
+################################################
+# Questasim
+setenv PATH ${QUESTA_PATH}/bin:$PATH
 
+#SCANLOGS
 setenv SCRIPTS $PROJECT_DIR/scripts
-prepend PATH ${SCRIPTS}
+setenv PATH ${SCRIPTS}:$PATH
 
 setenv TARGET_CFG    cv64a60ax
 setenv CVA6_REPO_DIR ${PROJECT_DIR}/modules/cva6

ref_model_csim/cpp/Makefile

@@ -0,0 +1,70 @@
+#!/bin/bash
+#
+#  Copyright (c) 2025 CEA*
+#  *Commissariat a l'Energie Atomique et aux Energies Alternatives (CEA)
+#
+#  SPDX-License-Identifier: Apache-2.0 WITH SHL-2.1
+#
+#  Licensed under the Solderpad Hardware License v 2.1 (the “License”); you
+#  may not use this file except in compliance with the License, or, at your
+#  option, the Apache License version 2.0. You may obtain a copy of the
+#  License at
+#
+#  https://solderpad.org/licenses/SHL-2.1/
+#
+#  Unless required by applicable law or agreed to in writing, any work
+#  distributed under the License is distributed on an “AS IS” BASIS, WITHOUT
+#  WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#  License for the specific language governing permissions and limitations
+#  under the License.
+#
+#
+#  Authors       : Ihsane TAHIR
+#  Creation Date : March, 2025
+#  Description   : 
+#  History       :
+#
+C_TOP="\e[1;32m" # Green
+C_RST="\e[0m"    # Reset
+C_MSG="\e[1;34m" # Blue
+
+# Set project root to current working directory
+export PROJECT_DIR="$(pwd)"
+
+echo -e "${C_TOP}>>>>>> Initializing Platform <<<<<<${C_RST}"
+
+############################################################
+# Update all git submodules
+############################################################
+echo -e "${C_MSG}Update all git submodules${C_RST}"
+git submodule init
+git submodule sync
+git submodule update --init --recursive
+
+
+############################################################
+# Add the local path to the Perl libraries
+############################################################
+if [[ -n "$PERL5LIB" ]]; then
+    export PERL5LIB="${PERL5LIB}:${PROJECT_DIR}/scripts/perl5"
+else
+    export PERL5LIB="${PROJECT_DIR}/scripts/perl5"
+fi
+
+# TOOLS
+export PATH="${QUESTA_PATH}/bin:$PATH"
+
+#SCANLOGS
+export SCRIPTS=$PROJECT_DIR/scripts
+export PATH="${SCRIPTS}:$PATH"
+
+############################################################
+# Project-specific environment variables
+############################################################
+export TARGET_CFG="cv64a60ax"
+export CVA6_REPO_DIR="${PROJECT_DIR}/modules/cva6"
+export CORE_V_VERIF="${PROJECT_DIR}/modules/core-v-verif"
+export DV_UTILS_DIR="${CORE_V_VERIF}/lib/cv_dv_utils"
+export SCRIPTS_DIR="${DV_UTILS_DIR}/python/sim_cmd"
+
+echo -e "${C_TOP}>>>>>> Initialization Done <<<<<<${C_RST}"