Skip to content

Try "Handle motion/drift" documentation as a sphinx gallery #2879

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
7aac4c6
Update with draft of sphinx-build version.
JoeZiminski May 17, 2024
a051a05
Add sg_execution_times.rst to gitignore.
JoeZiminski May 18, 2024
7574b67
select whether to build slow shpinx-gallery pages.
JoeZiminski May 18, 2024
5189203
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 17, 2024
6adac7b
Remove unused 'new' folder.
JoeZiminski May 18, 2024
9979e9e
Remove unused new rst.
JoeZiminski May 18, 2024
10ddf11
Remove some unused comments.
JoeZiminski May 18, 2024
21016e5
Change to use different method of long-docs generation.
JoeZiminski May 20, 2024
250ef4a
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] May 20, 2024
c674c6f
Rewrite introduction.
JoeZiminski Jun 6, 2024
4f2bb24
Try to add plots on probe.
JoeZiminski Jun 21, 2024
4ee1882
Continue reworking plots a little bit.
JoeZiminski Jun 22, 2024
ef84685
Add some seections, to tidy up tomorrow.
JoeZiminski Jun 24, 2024
e681b34
Add some seections, to tidy up tomorrow.
JoeZiminski Jun 24, 2024
7f292a1
Tidied up version with something approaching final parameters used fo…
JoeZiminski Jun 25, 2024
8d93038
try changing git ignore.
JoeZiminski Jun 27, 2024
c86913e
Remove sg execution times.
JoeZiminski Jun 27, 2024
d21e2db
-> SpikeInterface
JoeZiminski Jun 27, 2024
609fe3c
"z" -> "y" fix
JoeZiminski Jun 27, 2024
acbc9d2
Consider the spinal cord researcher!
JoeZiminski Jun 27, 2024
2ac5364
Remove stray comma
JoeZiminski Jun 27, 2024
a500d2c
Remove superfluous 'preprocessed' before 'recording'.
JoeZiminski Jun 27, 2024
d513f6a
Fix rigidt drift typo.
JoeZiminski Jun 27, 2024
061b859
AmericaniZe spelling
JoeZiminski Jun 27, 2024
1a80b0b
Add missing 'a' to sentence
JoeZiminski Jun 27, 2024
b4d9371
Americanize visualized
JoeZiminski Jun 27, 2024
5e0f2a5
Fix section header length 1
JoeZiminski Jun 27, 2024
ca7d5d0
Fix section header length 2
JoeZiminski Jun 27, 2024
cba4b95
Add missing "-" to "non-rigid"
JoeZiminski Jun 27, 2024
043b43c
Expand a little the motion outputs.
JoeZiminski Jun 27, 2024
4a0511f
Fix missing words in a sentence.
JoeZiminski Jun 27, 2024
2e33292
Fix stray #
JoeZiminski Jun 27, 2024
92594e0
Remove stray 'be'
JoeZiminski Jul 2, 2024
a07117d
behaviour -> behaving (fix grammer)
JoeZiminski Jul 2, 2024
6559e17
Add kilosort4 link and new section on assessing outputs.
JoeZiminski Jul 2, 2024
84d60e4
Full build with new settings.
JoeZiminski Jul 2, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,7 @@ examples/tutorials/*.svg
doc/_build/*
doc/tutorials/*
doc/sources/*
*sg_execution_times.rst

examples/getting_started/tmp_*
examples/getting_started/phy
Expand Down
14 changes: 12 additions & 2 deletions doc/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@
sphinx_gallery_conf = {
'only_warn_on_example_error': True,
'examples_dirs': ['../examples/tutorials'],
'gallery_dirs': ['tutorials' ], # path where to save gallery generated examples
'gallery_dirs': ['tutorials'], # path where to save gallery generated examples
'subsection_order': ExplicitOrder([
'../examples/tutorials/core',
'../examples/tutorials/extractors',
Expand All @@ -130,9 +130,19 @@
'within_subsection_order': FileNameSortKey,
'ignore_pattern': '/generate_',
'nested_sections': False,
'copyfile_regex': r'.*\.rst|.*\.png|.*\.svg'
'copyfile_regex': r'.*\.rst|.*\.png|.*\.svg',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chrishalcrow can you remind me what this does? I re-read the section and I could not locate this into the context of our project. Would you be so kind?

https://sphinx-gallery.github.io/stable/configuration.html#manually-passing-files

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello, sorry I missed this.

This copies any rst, png or svg files from the source sphinx-gallery directory to the output directory. In this case the source directory is examples/tutorials/ and we would like it to copy waveform_extractor_to_sorting_analyzer.rst from there to where the gallery is outputted (doc/tutorials).

This is done so that the tutorial pages (https://spikeinterface.readthedocs.io/en/latest/tutorials/index.html), which is a sphinx-gallery, can include a non-sphinx gallery page (the waveforms page).

The .svgs were included to copy over the svg of the extension parent/child figure, but this is not kept in doc/_images since it's repeated in a few places.

'filename_pattern': '/plot_',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a default value, why add it here? If you want to be explicit can we add a small comment and what it does?

}

if tags.has("handle_drift") or tags.has("all_long_plot"):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Worth generalising this? Something like:

Suggested change
if tags.has("handle_drift") or tags.has("all_long_plot"):
long_tutorial_names = ["handle_drift"]
for long_tutorial_name in long_tutorial_names:
if tags.has(long_tutorial_name) or tags.has("all_long_plot"):
if (tutorial_path := (Path('long_tutorials/'+long_tutorial_name))).is_dir():
shutil.rmtree(tutorial_path)
sphinx_gallery_conf['examples_dirs'].append('../examples/long_tutorials/'+long_tutorial_name)
sphinx_gallery_conf["gallery_dirs"].append(tutorial_path.as_posix())

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is very nice, for now if it works for you I will wait for the upcoming PR tackling the Neuropixels how-to to implement this, just to make checking it's working for multiple cases a bit easier. I'll link to this suggestion in our issue to not forget!


if (handle_drift_path := (Path('long_tutorials/handle_drift'))).is_dir():
shutil.rmtree(handle_drift_path)

sphinx_gallery_conf['examples_dirs'].append('../examples/long_tutorials/handle_drift')
sphinx_gallery_conf["gallery_dirs"].append(handle_drift_path.as_posix())

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does sphinx knows what goes to where? That is, what things on eample_dirs who to which gallery_dirs? Is that positionally?


intersphinx_mapping = {
"neo": ("https://neo.readthedocs.io/en/latest/", None),
"probeinterface": ("https://probeinterface.readthedocs.io/en/stable/", None),
Expand Down
1 change: 1 addition & 0 deletions doc/how_to/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ Guides on how to solve specific, short problems in SpikeInterface. Learn how to.
combine_recordings
process_by_channel_group
load_your_data_into_sorting
/long_tutorials/handle_drift/plot_handle_drift.rst
Binary file added doc/images/no-drift-example.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file not shown.
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that this won't be display with a thumbnail, do we need this?

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
72 changes: 72 additions & 0 deletions doc/long_tutorials/handle_drift/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
:orphan:

Handle Drift Tutorial
---------------------

This tutorial is not mean to be displayed on
a sphinx gallery. The generated index.rst is not
meant to be linked to in any toctree.

Instead, sphinx-gallery is used to
automatically build this page, which
takes a long time (~25 minutes), and it is
linked too manually, directly to the
rst (TODO: fill in filename) that
sphinx-gallery generates.



.. raw:: html

<div class="sphx-glr-thumbnails">

.. thumbnail-parent-div-open

.. raw:: html

<div class="sphx-glr-thumbcontainer" tooltip="Probe movement is an inevitability when running in vivo electrophysiology recordings. Motion, c...">

.. only:: html

.. image:: /long_tutorials/handle_drift/images/thumb/sphx_glr_plot_handle_drift_thumb.png
:alt:

:ref:`sphx_glr_long_tutorials_handle_drift_plot_handle_drift.py`

.. raw:: html

<div class="sphx-glr-thumbnail-title">Handle probe drift with spikeinterface NEW</div>
</div>


.. thumbnail-parent-div-close

.. raw:: html

</div>


.. toctree::
:hidden:

/long_tutorials/handle_drift/plot_handle_drift


.. only:: html

.. container:: sphx-glr-footer sphx-glr-footer-gallery

.. container:: sphx-glr-download sphx-glr-download-python

:download:`Download all examples in Python source code: handle_drift_python.zip </long_tutorials/handle_drift/handle_drift_python.zip>`

.. container:: sphx-glr-download sphx-glr-download-jupyter

:download:`Download all examples in Jupyter notebooks: handle_drift_jupyter.zip </long_tutorials/handle_drift/handle_drift_jupyter.zip>`


.. only:: html

.. rst-class:: sphx-glr-signature

`Gallery generated by Sphinx-Gallery <https://sphinx-gallery.github.io>`_
246 changes: 246 additions & 0 deletions doc/long_tutorials/handle_drift/plot_handle_drift.ipynb
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this in version control?

Large diffs are not rendered by default.

Loading