Merge pull request #180 from PyThaiNLP/dev

PyThaiNLP 2.0

Author: wannaphong

.gitignore

@@ -26,8 +26,8 @@ var/
 *.egg
 
 # PyInstaller
-#  Usually these files are written by a python script from a template
-#  before PyInstaller builds the exe, so as to inject date/other infos into it.
+# Usually these files are written by a python script from a template
+# before PyInstaller builds the exe, so as to inject date/other infos into it.
 *.manifest
 *.spec
 
@@ -58,6 +58,11 @@ target/
 
 # Jupyter Notebook
 .ipynb_checkpoints
+Untitled*.ipynb
+
+# IDE files
+.idea
+.vscode
 
 # macOS generated files
 .DS_Store
@@ -66,6 +71,8 @@ target/
 .Spotlight-V100
 .Trashes
 
+# Document generator temporary files
+docs/_build/
 
 \.idea/codeStyles/
 

.travis.yml

@@ -3,12 +3,17 @@
 
 language: python
 python:
-  - "3.4"
-  - "3.5"
   - "3.6"
+
+# workaround to make boto work on travis
+# from https://github.com/travis-ci/travis-ci/issues/7940
+before_install:
+  - sudo rm -f /etc/boto.cfg
+
 # command to install dependencies, e.g. pip install -r requirements.txt --use-mirrors
 install:
-  - pip install -r requirements-travis.txt
+  - pip install -r requirements.txt
+  - pip install .[artagger,icu,ipa,ner,thai2fit,deepcut]
   - pip install coveralls
 
 os:

CONTRIBUTING.md

@@ -23,21 +23,22 @@ We use the famous [gitflow](http://nvie.com/posts/a-successful-git-branching-mod
 - Write tests for your new features (please see "Tests" topic below);
 - Always remember that [commented code is dead
   code](http://www.codinghorror.com/blog/2008/07/coding-without-comments.html);
-- Name identifiers (variables, classes, functions, module names) with readable
-  names (`x` is always wrong);
+- Name identifiers (variables, classes, functions, module names) with meaningful
+  and pronounceable names (`x` is always wrong);
 - When manipulating strings, use [Python's new-style
   formatting](http://docs.python.org/library/string.html#format-string-syntax)
   (`'{} = {}'.format(a, b)` instead of `'%s = %s' % (a, b)`);
 - All `#TODO` comments should be turned into issues (use our
-  [GitHub issue system](tps://github.com/wannaphongcom/pythainlp/));
+  [GitHub issue system](https://github.com/PyThaiNLP/pythainlp/));
 - Run all tests before pushing (just execute `tox`) so you will know if your
   changes broke something;
+- All source code and all text files should be ended with one empty line. This is [to please git](https://stackoverflow.com/questions/5813311/no-newline-at-end-of-file#5813359) and also [to keep up with POSIX standard](https://stackoverflow.com/questions/729692/why-should-text-files-end-with-a-newline).
 
 
 # Discussion
 
 - Facebook group: https://www.facebook.com/groups/thainlp 
-- GitHub issues: https://github.com/wannaphongcom/pythainlp/issues
+- GitHub issues: https://github.com/PyThaiNLP/pythainlp/issues
 
 Happy hacking! (;
 
@@ -54,14 +55,14 @@ Happy hacking! (;
 ## newmm (onecut), mm, TCC, and Thai Soundex Code
 - Korakot Chaovavanich
 
-## Thai2Vec & ulmfit
+## thai2fit & ULMFiT
 - Charin Polpanumas
 
 ## Docs
 - Peeradej Tanruangporn
 
 ## Contributors
-- See more contributions here https://github.com/wannaphongcom/pythainlp/graphs/contributors
+- See more contributions here https://github.com/PyThaiNLP/pythainlp/graphs/contributors
 
 
 # References

README-pypi.md

@@ -1,6 +1,6 @@
 ![PyThaiNLP Logo](https://avatars0.githubusercontent.com/u/32934255?s=200&v=4)
 
-# PyThaiNLP 1.7
+# PyThaiNLP 2.0
 
 [![Codacy Badge](https://api.codacy.com/project/badge/Grade/cb946260c87a4cc5905ca608704406f7)](https://www.codacy.com/app/pythainlp/pythainlp_2?utm_source=github.com&utm_medium=referral&utm_content=PyThaiNLP/pythainlp&utm_campaign=Badge_Grade)[![pypi](https://img.shields.io/pypi/v/pythainlp.svg)](https://pypi.python.org/pypi/pythainlp)
 [![Build Status](https://travis-ci.org/PyThaiNLP/pythainlp.svg?branch=develop)](https://travis-ci.org/PyThaiNLP/pythainlp)
@@ -10,32 +10,60 @@
 
 PyThaiNLP is a Python library for natural language processing (NLP) of Thai language.
 
-PyThaiNLP features include Thai word and subword segmentations, soundex, romanization, part-of-speech taggers, and spelling corrections.
+PyThaiNLP includes Thai word tokenizers, transliterators, soundex converters, part-of-speech taggers, and spell checkers.
 
-## What's new in version 1.7 ?
+📖 For details on upgrading from PyThaiNLP 1.7 to PyThaiNLP 2.0, see [From PyThaiNLP 1.7 to PyThaiNLP 2.0](https://thainlp.org/pythainlp/docs/2.0/notes/pythainlp-1_7-2_0.html)
 
-- Deprecate Python 2 support
-- Refactor pythainlp.tokenize.pyicu for readability
-- Add Thai NER model to pythainlp.ner
-- thai2vec v0.2 - larger vocab, benchmarking results on Wongnai dataset
-- Sentiment classifier based on ULMFit and various product review datasets
-- Add ULMFit utility to PyThaiNLP
-- Add Thai romanization model thai2rom
-- Retrain POS-tagging model
+📖 For ThaiNER user after upgrading from PyThaiNLP 1.7 to PyThaiNLP 2.0, see [Upgrade ThaiNER from PyThaiNLP 1.7 to PyThaiNLP 2.0](https://github.com/PyThaiNLP/pythainlp/wiki/Upgrade-ThaiNER-from-PyThaiNLP-1.7-to-PyThaiNLP-2.0)
+
+📫 follow us on Facebook [Pythainlp](https://www.facebook.com/pythainlp/)
+
+## What's new in version 2.0 ?
+
+- New NorvigSpellChecker spell checker class, which can be initialized with custom dictionary.
+- Terminate Python 2 support. Remove all Python 2 compatibility code.
+- Remove old, obsolated, deprecated, and experimental code.
+- Thai2fit (Upgrade ULMFiT-related codes to fastai 1.0)
+- ThaiNER 1.0
+- Remove sentiment analysis
 - Improved word_tokenize (newmm, mm) and dict_word_tokenize
-- Documentation added
+- Improved POS-tagging
+- More and improved examples
+- see [PyThaiNLP 2.0 change log](https://github.com/PyThaiNLP/pythainlp/issues/118)
 
 ## Install
 
+For stable version:
+
 ```sh
 pip install pythainlp
 ```
 
+For some advanced functionalities, like word vector, extra packages  may be needed. Install them with these options during pip install:
+
+```
+pip install pythainlp[extra1,extra2,...]
+```
+
+where extras can be
+
+- `artagger` (to support artagger part-of-speech tagger)*
+- `deepcut` (to support deepcut machine-learnt tokenizer)
+- `icu` (for ICU support in transliteration and tokenization)
+- `ipa` (for International Phonetic Alphabet support in transliteration)
+- `ml` (to support fastai 1.0.22 ULMFiT models)
+- `ner` (for named-entity recognizer)
+- `thai2fit` (for Thai word vector)
+- `thai2rom` (for machine-learnt romanization)
+- `full` (install everything)
+
 **Note for Windows**: `marisa-trie` wheels can be obtained from https://www.lfd.uci.edu/~gohlke/pythonlibs/#marisa-trie 
 Install it with pip, for example: `pip install marisa_trie‑0.7.5‑cp36‑cp36m‑win32.whl`
 
 ## Links
 
-- Docs: https://thainlp.org/pythainlp/docs/1.7/ 
+- User guide : [English](https://colab.research.google.com/drive/1MQ10D1mJC5r1vQAHcj4ShoRS14vz8ZF-) , [ภาษาไทย](https://colab.research.google.com/drive/1rEkB2Dcr1UAKPqz4bCghZV7pXx2qxf89)
+- Docs: https://thainlp.org/pythainlp/docs/2.0/ 
 - GitHub: https://github.com/PyThaiNLP/pythainlp
 - Issues: https://github.com/PyThaiNLP/pythainlp/issues
+- Facebook : [Pythainlp](https://www.facebook.com/pythainlp/)