🔎 Automated Essay Scoring (AES) with Transformers

Problem Statement

Artificial intelligence can be used to assess the quality of student essays by applying NLP techniques to analyze coherence, clarity, and relevance. Topic modeling and sentiment analysis can determine main themes and emotional tone, while machine learning algorithms can be trained on high-quality essays to evaluate new ones. AI-based tools can offer feedback and suggestions to help students improve their writing skills, saving educators time and providing consistent evaluations.

Deep Learning Models

• Transformers, LSTMs, GRUs, CNNs, and ML models can all be used to assess the quality of student essays in education.
• Transformers analyze text data, LSTMs and GRUs model the coherence and relevance of the essay, CNNs detect visual cues in accompanying materials, and ML models identify factors contributing to effective writing.
• By combining these approaches, a comprehensive evaluation of the overall quality of a student essay can be achieved, allowing educators to provide targeted feedback and improve student writing skills.

Metrics

Since this is a classification problem, we consider metrics that take into account discrete output variables (scores) and give their estimates based on the difference between the actual output and predicted output. Below are some metrics that are used for this prediction.

• Accuracy
• Precision
• Recall
• F1 Score
• Quadratic Weighted Kappa (QWK)

Future Scope

• It would a great idea to implement and use GPT-2 for extracting the text embeddings in the future before scoring the essays as this leads to good performance improvement.
• Instead of fine-tuning transformer models, it is efficient to use the text-embeddings as the performance is quite similar to that of fine-tuning these models from scratch.

👉 Directions to download the repository and run the notebook

This is for the Washington Bike Demand Prediction repository. But the same steps could be followed for this repository.

1. You'll have to download and install Git which could be used for cloning the repositories that are present. The link to download Git is https://git-scm.com/downloads.

  

2. Once "Git" is downloaded and installed, you'll have to right-click on the location where you would like to download this repository. I would like to store it in the "Git Folder" location.

  

3. If you have successfully installed Git, you'll get an option called "Gitbash Here" when you right-click on a particular location.

  

4. Once the Gitbash terminal opens, you'll need to write "Git clone" and then paste the link to the repository.

  

5. The link to the repository can be found when you click on "Code" (Green button) and then, there would be an HTML link just below. Therefore, the command to download a particular repository should be "Git clone HTML" where the HTML is replaced by the link to this repository.

  

6. After successfully downloading the repository, there should be a folder with the name of the repository as can be seen below.

  

7. Once the repository is downloaded, go to the start button and search for "Anaconda Prompt" if you have anaconda installed.

  

8. Later, open the Jupyter notebook by writing "Jupyter notebook" in the Anaconda prompt.

  

9. Now the following would open with a list of directories.

  

10. Search for the location where you have downloaded the repository. Be sure to open that folder.

  

11. You might now run the .ipynb files present in the repository to open the notebook and the python code present in it.

  

That's it, you should be able to read the code now. Thanks.