DOC v0.30 part 1 (index, dev, install, quickstart, changelog)

Author: VarIr

CONTRIBUTING.rst

@@ -11,7 +11,7 @@ Contributing
 Contributions from the community are highly appreciated.
 Even small contributions improve the software's quality.
 
-Even if you are not familiar with programming languages and tools,
+If you are not familiar with programming languages and tools,
 you may contribute by filing bugs or any problems as a
 `GitHub issue <https://github.com/VarIr/scikit-hubness/issues>`_.
 
@@ -25,13 +25,14 @@ If you are not familiar with git, there are lots of tutorials on
 All the important basics are covered in the
 `GitHub Git handbook <https://guides.github.com/introduction/git-handbook/>`_.
 
-Development of `scikit-hubness` (mostly) follows the
-`git flow branching model <https://nvie.com/posts/a-successful-git-branching-model/>`_.
-There are two main branches: master and develop.
+There is one main branches: ``main``.
 For any changes, a new branch should be created.
-If you want to add a new feature, fix a noncritical bug, etc. one should
-branch off `develop`.
-Only if you want to fix a critical bug, branch off `master`.
+If you want to add a new feature, fix a noncritical bug, fix a critical bug,
+branch off ``main``, introduce your changes, and create a pull request.
+
+(Previously, development of `scikit-hubness` (mostly) followed the
+`git flow branching model <https://nvie.com/posts/a-successful-git-branching-model/>`_.
+This was found to be unnecessarily complicated for a project of this size).
 
 
 Workflow
@@ -55,10 +56,10 @@ you can - of course - directly submit a pull request (PR).
 
 #. Create feature/bugfix branch.
     In case of feature or noncritical bugfix:
-    $ ``git checkout develop && git checkout -b featureXYZ develop``
+    $ ``git checkout main && git checkout -b featureXYZ``
 
     In case of critical bug:
-    $ ``git checkout -b bugfix123 master``
+    $ ``git checkout -b bugfix123 main``
 
 #. Implement feature/fix bug/fix typo/...
     Happy coding!
@@ -76,13 +77,13 @@ you can - of course - directly submit a pull request (PR).
 
 #. Wait...
     Several devops checks will be performed automatically
-    (e.g. continuous integration (CI) with Travis, AppVeyor).
+    (e.g. continuous integration (CI) with Github Actions).
 
     The authors will get in contact with you,
     and may ask for changes.
 
 #. Respond to code review.
-    If there were issues with continous integration,
+    If there were issues with continuous integration,
     or the authors asked for changes, please create a new commit locally,
     and simply push again to GitHub as you did before.
     The PR will be updated automatically.
@@ -123,9 +124,9 @@ Code style and further guidelines
 Testing
 =======
 
-In `scikit-hubness`, we aim for high code coverage. As of September 2019,
-between 98% and 99% of all code lines are visited at least once when
-running the complete test suite. This is primarily to ensure:
+In `scikit-hubness`, we aim for high code coverage. Between 90% and 100% of all code lines
+should be visited at least once when running the complete test suite.
+This is primarily to ensure:
 
 * correctness of the code (to some extent) and
 * maintainability (new changes don't break old code).

docs/changelog.md

@@ -3,17 +3,48 @@
 ## [Next release]
 ...
 
+
+## [0.30.0] - 2022-04-xx
+
+### Major changes
+- Compatibility with up-to-date scikit-learn versions
+- Building upon the KNeighborsTransformer API as outlined in
+  [our paper's Outlook section](https://joss.theoj.org/papers/10.21105/joss.01957).
+- `skhubness.neighbors` rewritten from scratch. Previously, this was a drop-in
+  replacement for `sklearn.neighbors` heavily relying on a specific scikit-learn
+  version. This was hard to maintain. Now, this package only contains lightweight
+  wrappers for approximate nearest neighbor search tools (`nmslib`, `ngt`, etc.)
+  that return `KNeighborsTransformer`-compatible k-neighbors graphs. These can be
+  reused in numerous scikit-learn classes and functions.
+- `skhubness.analysis` uses `KNeighborsTransformer`-compatible k-neighbors graphs.
+- `skhubness.reduction` uses `KNeighborsTransformer`-compatible k-neighbors graphs.
+
 ### Added or enhanced
+- Python 3.9 and Python 3.10 support
+- Additional metrics available for ANN search with `nmslib` #87
+- Additional metrics available for ANN search with `ngtpy` #95
 - Lower memory footprint for sparse targets in multilabel classification 
   (previously converted to dense arrays) #61
 
+### Removed
+- `falconn` removed (not maintained for 5+ years; index structures cannot be serialized; no Windows support) #94
+- Radius neighbor search. Few ANN packages provide radius search. No currently supported
+  ANN package supports this on Windows. Radius search is not of particular interest to
+  hubness research. Thus, we decided to drop radius search for the time being to speed up
+  development. Later releases might re-introduce radius search. Users interested in this
+  are asked to file an Issue at Github
+
 ### Fixes
 - Hubness estimation could fail when ANN does not return enough neighbors #59
 - Heuristic to choose memory for Puffinn LSH.
 
 ### Maintenance
 - Switch to modern Python packaging with `pyproject.toml` and `setup.cfg`
 - Switch to Github Actions, dropping Travis CI and AppVeyor
+- Renamed 0.22 to 0.30. Previous versions reflected the compatibility with specific
+  scikit-learn versions by matching version numbers. The bump to 0.30 indicates that
+  this tight coupling is gone and future scikit-hubness releases should be compatible
+  with multiple scikit-learn versions.
 
 
 ## [0.21.2] - 2020-01-14
@@ -100,7 +131,8 @@ It already contains the following features:
   * HNSW provided by [nmslib](https://github.com/nmslib/nmslib)
   * LSH provided by [falconn](https://github.com/FALCONN-LIB/FALCONN)
 
-[Next release]: https://github.com/VarIr/scikit-hubness/compare/v0.21.2...HEAD
+[Next release]: https://github.com/VarIr/scikit-hubness/compare/v0.30.0...HEAD
+[0.30.0]:   https://github.com/VarIr/scikit-hubness/releases/tag/v0.30.0
 [0.21.2]:   https://github.com/VarIr/scikit-hubness/releases/tag/v0.21.2
 [0.21.1]:   https://github.com/VarIr/scikit-hubness/releases/tag/v0.21.1
 [0.21.0]:   https://github.com/VarIr/scikit-hubness/releases/tag/v0.21.0

docs/conf.py

@@ -14,21 +14,22 @@
 import sys
 sys.path.insert(0, os.path.abspath('../'))
 
-import mock
-MOCK_MODULES = ['falconn',
-                'nmslib',
-                'annoy',
-                'ngt',
-                'ngtpy',
-                'puffinn',
-                ]
+from unittest.mock import Mock
+MOCK_MODULES = [
+    'nmslib',
+    'annoy',
+    'ngt',
+    'ngtpy',
+    'numba',
+    'puffinn',
+]
 for mod_name in MOCK_MODULES:
-    sys.modules[mod_name] = mock.Mock()
+    sys.modules[mod_name] = Mock()
 
 # -- Project information -----------------------------------------------------
 
 project = 'scikit-hubness'
-copyright = '2020, Roman Feldbauer'
+copyright = '2022, Roman Feldbauer'
 author = 'Roman Feldbauer'
 
 # The full version, including alpha/beta/rc tags
@@ -41,24 +42,25 @@
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['recommonmark',
-              'numpydoc',
-              'sphinx_automodapi.automodapi',
-              'sphinx.ext.autodoc',
-              'sphinx.ext.autosectionlabel',
-              'sphinx.ext.autosummary',
-              'sphinx.ext.graphviz',
-              'sphinx.ext.inheritance_diagram',
-              'sphinx.ext.todo',
-              'sphinx.ext.napoleon',
-              'sphinx.ext.githubpages',
-              'sphinx.ext.mathjax',
-              'sphinx.ext.doctest',
-              'sphinx.ext.intersphinx',
-              'sphinx.ext.linkcode',
-              'sphinx_gallery.gen_gallery',      # to automatically generate example pages from scripts
-              'sphinx_search.extension',         # readthedocs-sphinx-search with ElasticSearch
-              ]
+extensions = [
+    'recommonmark',
+    'numpydoc',
+    'sphinx_automodapi.automodapi',
+    'sphinx.ext.autodoc',
+    'sphinx.ext.autosectionlabel',
+    'sphinx.ext.autosummary',
+    'sphinx.ext.graphviz',
+    'sphinx.ext.inheritance_diagram',
+    'sphinx.ext.todo',
+    'sphinx.ext.napoleon',
+    'sphinx.ext.githubpages',
+    'sphinx.ext.mathjax',
+    'sphinx.ext.doctest',
+    'sphinx.ext.intersphinx',
+    'sphinx.ext.linkcode',
+    'sphinx_gallery.gen_gallery',      # to automatically generate example pages from scripts
+    'sphinx_search.extension',         # readthedocs-sphinx-search with ElasticSearch
+]
 
 # Due to sphinx-automodapi
 numpydoc_show_class_members = False
@@ -75,8 +77,11 @@
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store',
-                    ]
+exclude_patterns = [
+    '_build',
+    'Thumbs.db',
+    '.DS_Store',
+]
 
 # Mock packages that are not installed on rtd
 autodoc_mock_imports = MOCK_MODULES
@@ -90,25 +95,24 @@
 # The following is used by sphinx.ext.linkcode to provide links to github
 from docs.github_link import make_linkcode_resolve
 
-linkcode_resolve = make_linkcode_resolve('skhubness',
-                                         'https://github.com/VarIr/'
-                                         'scikit-hubness/blob/{revision}/'
-                                         '{package}/{path}#L{lineno}')
+linkcode_resolve = make_linkcode_resolve(
+    'skhubness', 'https://github.com/VarIr/scikit-hubness/blob/{revision}/{package}/{path}#L{lineno}',
+)
 
 # sphinx gallery: where to take scripts from and where to save output to
 sphinx_gallery_conf = {
-    'examples_dirs':   # path to your example scripts:
-        ['../examples/sklearn',
-         '../examples/hubness_reduction',
-         '../examples/approximate_neighbors',
-         '../examples/approximate_hub_red',
-         ],
-    'gallery_dirs':    # path to where to save gallery generated output:
-        ['documentation/auto_examples',
-         'documentation/auto_examples_hr',
-         'documentation/auto_examples_ann',
-         'documentation/auto_examples_ahr',
-         ],
+    'examples_dirs': [  # path to your example scripts:
+        '../examples/sklearn',
+        '../examples/hubness_reduction',
+        '../examples/approximate_neighbors',
+        '../examples/approximate_hub_red',
+    ],
+    'gallery_dirs': [   # path to where to save gallery generated output:
+        'documentation/auto_examples',
+        'documentation/auto_examples_hr',
+        'documentation/auto_examples_ann',
+        'documentation/auto_examples_ahr',
+    ],
 }
 
 # suppress numerous "duplicate label" warnings from sphinx-gallery

docs/development/contributing.rst

@@ -11,7 +11,7 @@ Contributing
 Contributions from the community are highly appreciated.
 Even small contributions improve the software's quality.
 
-Even if you are not familiar with programming languages and tools,
+If you are not familiar with programming languages and tools,
 you may contribute by filing bugs or any problems as a
 `GitHub issue <https://github.com/VarIr/scikit-hubness/issues>`_.
 
@@ -25,13 +25,14 @@ If you are not familiar with git, there are lots of tutorials on
 All the important basics are covered in the
 `GitHub Git handbook <https://guides.github.com/introduction/git-handbook/>`_.
 
-Development of `scikit-hubness` (mostly) follows the
-`git flow branching model <https://nvie.com/posts/a-successful-git-branching-model/>`_.
-There are two main branches: master and develop.
+There is one main branches: ``main``.
 For any changes, a new branch should be created.
-If you want to add a new feature, fix a noncritical bug, etc. one should
-branch off `develop`.
-Only if you want to fix a critical bug, branch off `master`.
+If you want to add a new feature, fix a noncritical bug, fix a critical bug,
+branch off ``main``, introduce your changes, and create a pull request.
+
+(Previously, development of `scikit-hubness` (mostly) followed the
+`git flow branching model <https://nvie.com/posts/a-successful-git-branching-model/>`_.
+This was found to be unnecessarily complicated for a project of this size).
 
 
 Workflow
@@ -55,10 +56,10 @@ you can - of course - directly submit a pull request (PR).
 
 #. Create feature/bugfix branch.
     In case of feature or noncritical bugfix:
-    $ ``git checkout develop && git checkout -b featureXYZ develop``
+    $ ``git checkout main && git checkout -b featureXYZ``
 
     In case of critical bug:
-    $ ``git checkout -b bugfix123 master``
+    $ ``git checkout -b bugfix123 main``
 
 #. Implement feature/fix bug/fix typo/...
     Happy coding!
@@ -76,7 +77,7 @@ you can - of course - directly submit a pull request (PR).
 
 #. Wait...
     Several devops checks will be performed automatically
-    (e.g. continuous integration (CI) with GitHub Actions).
+    (e.g. continuous integration (CI) with Github Actions).
 
     The authors will get in contact with you,
     and may ask for changes.
@@ -123,9 +124,9 @@ Code style and further guidelines
 Testing
 =======
 
-In `scikit-hubness`, we aim for high code coverage. As of September 2019,
-between 98% and 99% of all code lines are visited at least once when
-running the complete test suite. This is primarily to ensure:
+In `scikit-hubness`, we aim for high code coverage. Between 90% and 100% of all code lines
+should be visited at least once when running the complete test suite.
+This is primarily to ensure:
 
 * correctness of the code (to some extent) and
 * maintainability (new changes don't break old code).