This directory contains the implementation of Hidden Markov Moded(HMM) based Part-of-Speech(POS) Tagger using Kneser-Ney Smoothing. All the code is written in Python. Dataset containing Training, Validation and Test data is in the same directory.

Use 'kn_pos.py' file for training of model and getting tag on testing data. It gives a 'tagged_output.txt' file containing word, tag pair of test data in tab separated form (word tag), with each pair on a single line.

To run the 'kn_pos.py' file, use the following command:
python kn_pos.py path/to/trainfile path/to/testfile

It will output a 'tagged_output.txt' in the same directory where 'kn_pos.py' file is located.

To evalute the tags generated by tagger against the correct tags, use the 'evalute.py' file. Run the following command:
python evaluation.py tagged_output.txt path/to/validationfile

It will print out the accuracy.

DATA FORMATS:

Training data should be in tab separated word,tag format:

ٹریور NN
ٹینک NN
مختلف JJ
قسم NN
کی PSP
چڑیوں NN
جیسے PRR

Validation data should be in this format:
ابتدائی JJ
نقصان NN
کے PSP
بعد NN
معین NNP
علی NNP
اور CC
مورگن NNP
نے PSP

Test data should be in this format:

ابتدائی
نقصان
کے
بعد
معین
علی
اور