Approximate word search using Apache OpenNLP

System requirements

• Java 8
• Maven 3.x.x
• Apache OpenNLP

Solution overview

1. Train a model using Apache OpenNLP to read company names from the news articles. This has been done using set of articles. That file is available as training.TXT in the project. This step is done only once.

2. Extract data from the CSV and XML files. OpenCSV and JAXB has been used respectively.

3. Put the company names (used as the word to find) from the CSV in a PatriciaTrie.

4. Using the trained model, extract the possible company names from each article and check for their presence in the Trie.

5. On my machine, the application runs in around 500 seconds with the given data.

TODO

1. Currently company name is picked up from the Trie by doing a prefix search. This will not get the former names. Also, it might pick up the wrong company (for exampel, xyz Ltd instead of xyz BV). Instead, a new method should be implemented in the Trie to do a partial String match.

2. The model should be trained with more data to minimize false positives. Currently there is a fair share of false positives picked up by the model.

3. Modify the program to accept file path and file name. Currently, it has been hard coded.