📦 MUTGEN Replication Package

Thank you for your interest in our work. This repository contains the replication package for our paper, including all necessary materials to reproduce the main experimental results. We are still actively improving the quality and organization of the artifact.

---

📁 Directory Structure

├── HumanEval-Java_fromscratch/ # Example datasets and run scripts
├── mutahunter/ # Core implementation of our technique
├── scripts/ # Python utilities for processing
├── index_humaneval_java.csv # Subject index for batch runs
├── requirements.txt # Python dependencies
└── README.md

---

🧩 Component Descriptions

HumanEval-Java_fromscratch/

Includes example datasets and execution scripts used in evaluation:

• iterrun.sh: Run iterative testing pipeline across multiple rounds.
• autorun.sh: Automatically execute all subjects in the benchmark.
• runeach.sh: Run experiments for a single subject.
• backup/: Stores original and comment-removed source files.
• pom.xml: Maven build configuration for Java testing pipeline.

mutahunter/

Main implementation of our technique:

• Python codebase implementing prompt generation, interaction with LLMs, and result processing.
• Built on top of the Mutahunter framework for generation.

scripts/

Utility Python scripts for automated evaluation:

• copy_from_index.py: Select test subjects based on index.
• extractAndremoveComments.py: Strip comments and summarize Java methods.
• flip.py: Flip boolean values in generated test cases (e.g., from EvoSuite).
• getClassName.py: Parse Java test files for class names.
• mergeRegressionRunCsv.py: Merge results from multiple test iterations.
• noplan_parsePITestReport.py: Extract mutation scores from PITest reports.
• analysisPITestReport.py: Analyze mutation performance across operators (RQ2-specific).

index_humaneval_java.csv

Contains one Java subject per line. Used for batch job selection and indexing.

requirements.txt

Python dependencies required to run the tool (see next section).

---

🛠️ Dependencies

Make sure the following are installed in your environment:

• Python 3.11
• Java 11
• EvoSuite 1.2.0

pip install -r requirements.txt

🚀 Usage Instructions

After installing all required dependencies, follow the steps below to reproduce the experimental results:

1. Navigate to the benchmark directory:

cd HumanEval-Java_fromscratch

2. Use the following Maven command to generate and export EvoSuite test cases. These tests can be saved and reused for further analysis:

mvn evosuite:generate evosuite:export

3. Before running, make sure to specify the desired number of iterations in the iterrun script:

./iterrun 

This script performs multi-round evaluation and will generate one folder per iteration (e.g., iter1_log, iter2_log, etc.).

4. Each iteration produces a log directory (itern_log) containing raw mutation testing results. For each log, extract the results using:

cd iter1_log && ./getResFromPITlog.sh > iter1.csv

Repeat this step for all iterations, then merge the result files using the provided script (used in RQ1 analysis):

python mergeRegressionRunCsv.py ../index_humaneval_java.csv iter1_log/iter1.csv ... itern_log/itern.csv