AutoGrader

The above project is used to evaluate handwritten exam answer scripts. We have fine tuned Llama2 for this task and also created a RAG pipeline to display the page of the textbook where the answer to that question is present. Finally, we have deployed this model on AWS SageMaker and created a Lambda function to handle requests from the REST API. This project is fine tuned to evaluate answer scripts for the course Operating Systems only and every question is graded from 1-5. We have created a whole ML Workflow from scratch.

Step 1: Dataset Generation

Dataset is generated by collecting questions from the course Operating Systems and the answers to each of the questions given by students and the corresponding score 1-5 is collected from the teacher. The dataset contains of around 116 entries and has 7 coloumns in which contain:

1. Question
2. Answer given by the student
3. Grade given by the techer for the corresponding answer
4. Temp - It is the input format for the LLM model excluding Llama2
5. Text - It is the input format for fine tuning the LLM model Llama2
6. context- This is created for the RAG pipeline
7. text - This is the updated input format for fine tuning the LLM model Llama2 after introducing the RAG pipeline

The dataset is found in the Dataset.csv

Step 2: Fine-Tuning Process

In this section, we outline the steps required to fine-tune the Llama 2 model with 7 billion parameters on a T4 GPU. You can use a free GPU on platforms like Google Colab or Kaggle for this process.

Due to the limited 16 GB of VRAM on the Colab T4 GPU, full fine-tuning is not possible. Instead, we employ parameter-efficient fine-tuning techniques such as LoRA (Low-Rank Adaptation) or QLoRA (Quantized Low-Rank Adaptation) to manage VRAM usage effectively.

Steps for Fine-Tuning

1.Setup the Environment: Ensure you have access to a T4 GPU on Google Colab or Kaggle.

2.Install Dependencies: Use the Hugging Face ecosystem of libraries including transformers, accelerate, peft, trl, and 3.bitsandbytes.

4.Utilize QLoRA Technique: Fine-tune the model in 4-bit precision to optimize VRAM usage and maintain model performance.

By leveraging these tools and techniques, we can efficiently fine-tune the Llama 2–7b model despite hardware constraints.

read more

Step 3: Creating a RAG Pipeline

Check RAG folder for more information

Step 4: Deploying Fine Tuned Model on AWS SageMaker

LLaMA-2 is a massive model with billions of parameters. Running it effectively requires significant computational resources, especially memory and potentially GPUs. Most local machines may not have enough power to handle the model efficiently. Due to this reason we deploy our model on AWS SageMaker. Check the AWS-Deployment folder for more information.

1. Pull the Model from HuggingFace
2. Create an Endpoint
3. Create a Lambda Function
4. Create a REST API Trigger for Lambda Function

Step 5: Handwriting to Text Conversion

Handwriting to Text Conversion is done using google vision. Already integrated in UI-UX, also present individually in vision-ocr folder. Fill in your details in vision_ocr.json.json and it is ready to use.

Step 6: Integrating with UI/UX

After acquiring AWS api_url, the webapp can be run by following the steps in readme present in UI-UX folder.

Results