news_article_summarizer_using_custom_model

Overview

A NLP based summarizer which can summarize news article. Using T5TokenizerFast I have trained a custom model on t5-base architecture and the dataset was collected from kaggle.

Table of Contents

• Installation
• Dependencies
• Data Set
• Data Modelling
• Inference
• HuggingFace Deployment
• FlaskApp

Installation

To get started with this project, follow these steps:

Clone the repository:

git clone https://github.com/marufc36/news_article_summarizer_using_custom_model

Dependencies

pip3 install requirements.txt

Data set

Data-set is collected from Kaggle. The dataset consists of 4515 examples and contains Author_name, Headlines, Url of Article, Short text, Complete Article. I have only worked with Short text and Complete Article and other columns were dropped from the data-set.

Data Modelling

Using t5-base pretrained model, T5TokenizerFast, pytorch lighting the model was trained to three epoch on the collected data-set.Short text was used as summary and complete article was used as text.

Colab Notebook

https://drive.google.com/file/d/1qbCqMXjv5aqyfkJ6ntQMfGwYoXXATqWM/view?usp=sharing

Inference

I have inferenced the using a gradio app. I have to load the best checkpoint using NewsSummaryModel. I have created model.py module which includes NewsSummaryModel.

HuggingFace Deployment

The model was deployed to HuggingFace Spaces. Linkn is given below.

https://huggingface.co/spaces/mmchowdhury/News_Summary_With_Custom_Model

FlaskApp

Using HuggingFace Api I have build a flask app.