close writer before soft deletion of search index

[jiangphcn]

Overview

This fix is to refactor the logic for soft-deletion of search index. In the past, there is case where write.lock was stayed in search index directory and can't be moved. It is due to the fact that the index was not closed gracefully before moving/renaming search index. With this fix, the search index writer can be closed, and then search index can be moved to .deleted directory smoothly.

Testing recommendations

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running com.cloudant.clouseau.AnalyzerServiceSpec
Running com.cloudant.clouseau.AnalyzerServiceSpec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.024 sec
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.238 sec
Running com.cloudant.clouseau.ClouseauTypeFactorySpec
Running com.cloudant.clouseau.ClouseauTypeFactorySpec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.002 sec
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.022 sec
Running com.cloudant.clouseau.IndexManagerServiceSpec
Running com.cloudant.clouseau.IndexManagerServiceSpec
2019-11-21 15:26:30 clouseau [INFO] foo.5432109876 Renaming '/Users/jiangph/couchdb/clouseau.dreyfus/clouseau/target/indexes/foo.5432109876' to '/Users/jiangph/couchdb/clouseau.dreyfus/clouseau/target/indexes/foo.20191121.072630.deleted.5432109876'
Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.445 sec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.465 sec
Running com.cloudant.clouseau.ClouseauQueryParserSpec
Running com.cloudant.clouseau.ClouseauQueryParserSpec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.001 sec
Tests run: 8, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.018 sec
Running com.cloudant.clouseau.IndexServiceSpec
Running com.cloudant.clouseau.IndexServiceSpec
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.937 sec
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 23.954 sec
Running com.cloudant.clouseau.SupportedAnalyzersSpec
Running com.cloudant.clouseau.SupportedAnalyzersSpec
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.003 sec
Tests run: 42, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.109 sec

Results :

Tests run: 95, Failures: 0, Errors: 0, Skipped: 0

Related Issues or Pull Requests

#2130
cloudant-labs/clouseau#25

Checklist

• Code is written and works correctly
• Changes are covered by tests
• Any new configurable parameters are documented in rel/overlay/etc/default.ini
• A PR for documentation changes has been made in https://github.com/apache/couchdb-documentation

[rnewson]

This should look like rpc(Ref, {soft_delete, couch_util:to_binary(Id)}). and be sent to the IndexService process itself (opening it if it's not already open in the usual manner).

[rnewson]

As this is, by definition, a change to behaviour, it's not a refactor.

[rnewson]

This is what I was expecting, thanks. Let's not merge until the clouseau side is approved, in case there are changes needed here.

[janl]

should we hold this for 3.0 in light of making search setup easy?

[wohali]

@rnewson Clouseau was approved and merged, per Jan's comment should we progress with this for 3.0?

[jiangphcn]

thanks @wohali and @janl for comments for this PR. Finally, we found that we just need to cloudant-labs/clouseau#27. It means that we don't need the change on dreyfus anymore. Let me close this PR.