GenAI

A repo to store my implementations and experiments on Generative and Agentic AI solutions

Fine-Tuning for Python Code Generation

Best way to learn about some topic is to solve a real world problem, and improving the solution to be state-of-the-art, hence Lets fine tune ourselves a Coding-LLM which can help us in coding assignments 😉.

Goal : Fine tune a LLM with python code based datasets on a fairly good general purpose open-sourced LLM

PeFT:Fine-Tuning for Python Code Generation

| 📄 White Paper | 🌐 Web Page | 🥇 Results | 🐍 Sample Codes | 📃 Documentation |

• Full results at : ./notebooks/results/rank_alpha_results.md
• Samples at : ./notebooks/results/sample_generations.md

Design Choices:

• LLM : Qwen 2.5
• Dataset :
  • jtatman/python-code-dataset-500k
  • flytech/python-codes-25k
  • Vezora/Tested-143k-Python-Alpaca
  • sahil2801/CodeAlpaca-20k
  • nvidia/OpenCodeInstruct
  • Awesome list
• Input: Natural language description (es: "Write a Fibonacci numbers function")
• Output: Executable Python code

Progress

To Learn while creating the best coding LLM:

• LoRA Implementation
• LoRA+ Support
• qLoRA (best to use some lib - for ex: Unsloth)
  • 4-bit/8-bit Quantization
• Modular Architecture effects -Test (extend) to new models (Llama, Mistral, etc.)
• Hyperparameter Search (on $r$)
• Comprehensive Evaluation
  • BLEU
  • Pass@k,
  • CodeBERTScore
  • execution tests

LoRA - hyperparameters Results

• Model : Qwen 2.5 7B
• Coding Dataset : flytech/python-codes-25k
• Epochs : 1 (625 Steps)

rank	alpha	times	eval_loss	train_loss	best_eval_loss	avg_bleu	avg_similarity
8	16	2	0.50	0.50	0.51	28.1155	34.8547
4	8	2	0.52	0.51	0.52	28.3845	34.8609
2	4	2	0.54	0.52	0.54	28.9557	36.0274
1	2	2	0.55	0.54	0.55	29.8195	36.1216
32	64	2	0.47	0.47	0.48	31.3643	36.8357
16	32	2	0.49	0.48	0.49	32.6198	38.0383

Best Model sample prompt and response :

---

Sample 1

Prompt:

Write a Python function to calculate factorial

Generated Code:

def factorial(n):
    if n == 0 or n ==1 :
        return 1 
    else:  
        return (n * factorial(n-1))

---

Sample 2

Prompt:

Create a function to reverse a string

Generated Code:

Here's one way you could write a Python function to take in a string and return its reverse:

def reverse_string(string):
    reversed_str = ""
    for char in string:
        reversed_str=char +reversed_str 
    return reversed_str

This function takes in a string parameter, which represents the input string to be reversed. It initializes an empty string called reversed\_str, then iterates through each character of the input string. For every iteration, it adds the current character to the beginning of reversd_str. Finally, it returns the fully-reversed version of the original string stored within revered_str.

You can call this function by passing any desired string as an argument like so:

my\_string="Hello World!"
print(reverse\_string(my\_string))
# Output: !dlroW olleH

---

• Full results at : ./notebooks/results/rank_alpha_results.md
• Samples at : ./notebooks/results/sample_generations.md

Usage

Setup

# Clone repository
git clone https://github.com/Mahanth-Maha/GenAI.git
cd GenAI

# Create virtual environment
conda create --name finetune python==3.11 -y
conda activate finetune

# (optinal)
# conda install -c anaconda ipykernel -y
# python -m ipykernel install --user --name finetune --display-name "finetune"
# pip3 install torch torchvision --index-url https://download.pytorch.org/whl/rocm6.4

# Install dependencies
pip install -r requirements.txt

# development mode
pip install -e .

Running the fine tuning

Try --help and get the list of options:

python scripts/main.py --help 

Quick Start

To fine tune with the flytech/python-codes-25k dataset and rank = 32 and alpha = 64, and train for 1 epoch with lr at 2e-4 and batch size as 8 and accumulation steps as 8, change the params as needed or add more flags for control:

python scripts/main.py --rank 32 --alpha 64 -lr 2e-4 --train -ep 1 -bs 32 -ac 2 --save_steps 1000 --eval_steps 250 --bf16 --max_seq_length 512 --run_name "Code_FT_rank_32_alpha_64"

Repo Structure

file name are self explanatory :)

.
├── README.md
├── checkpoints   # Results (not synced)
├── configs
│   ├── data_config.py
│   ├── model_config.py
│   └── training_config.py
├── notebooks
├── scripts
│   ├── inference.py
│   ├── main.py
│   └── search.py
├── src
│   ├── data
│   │   ├── collator.py
│   │   ├── dataset.py
│   │   └── preprocessing.py
│   ├── evaluation
│   │   ├── code_execution.py
│   │   ├── evaluator.py
│   │   └── metrics.py
│   ├── models
│   │   ├── base_model.py
│   │   ├── llama_model.py
│   │   ├── lora.py
│   │   └── qwen_model.py
│   ├── training
│   │   ├── optimizer.py
│   │   ├── scheduler.py
│   │   └── trainer.py
│   └── utils
│       ├── checkpoint.py
│       ├── device.py
│       ├── logger.py
│       └── visualization.py
└── tests
    └── test.py

---

Mahanth Yalla
MTech - Artificial Intelligence,
Indian Institute of Science,
Bangalore - 560012.