Support for enabling full-text index and compression on demand

[parvit]

This change responds to issue #243.

Requires openzim/python-scraperlib#88 to work correctly.

Changes the default behavior to not require compression and full-text indexing by default which can be an issue with big sites.

Introduces two new commandline flags to support enabling the features back:

• --make_fulltext_index : boolean default false, activates the fulltext indexing of xapian
• --compression : string which corresponds to the requested zim compression algorithm (eg. lzma)

[kelson42]

@parvit Text compression and ft indexing ahoukd be activated per default. This is what users expect and a "standard" in all our scraper. I'm not informed about problems by doing so.

[rgaudin]

Let's verify first #243 (comment) that disabling them would significantly improve the MEM situation. In such a case, we may introduce the opposite option (disable) as a temporary measure until we fix the root cause as both are definitely wanted features.

[parvit]

@kelson42 Sure if you want you can check the data i've provided in issue 243 that indicate that those two features can create memory usage problems (in the scenario of big sites) and at least allowing to disable them should be considered.

[parvit]

Seen that the other PR was closed than this too has no reason to be left open.