[ENH] microelectrode electrophysiology specification (BEP032)

[yarikoptic]

Replaces #1352 submitted from a fork outside of bids-specification.

Add specification for microelectrode electrohpysiology datasets based on the BEP032 proposal. old google doc

Note

We meet regularly and everyone is welcome

Next meeting: insert date on URL to join

Communication channel: https://framalistes.org/sympa/info/neuroscience-data-structure

Tip

HTML preview of this BEP

---

• To use this WiP schema on sample datasets, see https://deno.land/x/bids_validator@v1.13.2-dev.0#modifying-and-building-a-new-schema on how to use "stock" bids-validator with a custom schema. (attn @TheChymera)

DONEs

• Add (at least one) .md with the section for the added new modality/datatype
• [js] Update all npm dependencies (2023-09-10) legacy-validator#1800

TODOs

• Please ensure your name is credited on our Contributors appendix.
  To add your name, please edit our Contributors wiki and add your name with the type of contribution.
  For assistance, please tag @bids-standard/maintainers.

• After opening the PR, our continuous integration services will automatically check your contribution for formatting errors and render a preview of the BIDS specification with your changes.
  To see the checks and preview, scroll down and click on the show all checks link.
  From the list, select the Details link of the ci/circleci: build_docs artifact check to see the preview of the BIDS specification.

• Add instructions here on how to run new bids-validator using schema in this PR

  • Currently there is ongoing effort in Add "master-deno" flavor of validator to use as well bids-examples#435 to assess the state of deno based validator against examples, then chore(deps): bump bids-validator/tests/data/bids-examples from c557d1f to 1c30c6e legacy-validator#1798 is the first one trying it on whitelisted set of packages, and I think we should create a helper action for that : https://github.com/bids-standard/bids-validator/issues/1931

• bids-validator changes needed

  • Add tricky use cases where validator should fail under https://github.com/bids-standard/bids-validator/tree/main/tests/data

• Populate schema with specifications from the google doc ...

  • The majority of changes: [ENH] Schema changes related to BEP032 #1826
  • Following @ree-gupta failed validation - tune up check for multiple electrode files to apply only to EEG, and iEEG modalities
  • Extend schema/rules/checks with checks specific to this BEP032 (from experiences with data conversion eg by @CodyCBakerPhD on @mvmdmlab data)
  • Similarly to iEEG specify <extension> and get a table of extensions (nwb and nix) and check if schema encodes that only one is allowed

• Further markdown description: @Peyman-N is working on a PR

• Define enums of coord spaces to be added -- some image based, some ad-hoc

• Add and reference here PR on bids-examples adding sample dandisets : Draft examples for BEP032 on animal electrophysiology bids-examples#430 (@robertoostenveld )

• Add CI action (likely github) to run bids-validator on sample datasets and this modified schema (@yarikoptic)

• Decide on "contours" specification

  • decided to postpone
  • attempt to potentially generalize across modalities: Formalize specification of shape(s) (AKA contour(s)) #2013

• type column for electrodes

  • seek to replace hardcoded table in iEEG with a macro (@yarikoptic)
  • add our new enums to src/schema/objects/enums.yaml and then within src/schema/objects/columns.yaml (@ree-gupta )
  • depending on decision in iEEG potentially add similar table in our [ie]cephys/ section

• Share/use examples of real datasets

  • WiP: https://dandiarchive.org/dandiset/001470 AKA https://github.com/dandisets/001470 by @CodyCBakerPhD from @mvdm lab

Issues this PR would likely to address

• Fixes run node tests in github actions legacy-validator#1375

Issues to see being addressed while working on this BEP (likely to move above) or not (moved below):

• bump min node version to 14 legacy-validator#1634
• Add summary generation to schema-prototype legacy-validator#1481

Other issues which relate but not in scope here and provided for reference/backreference

• Bundle CLI entrypoint with main() call legacy-validator#1669
• Bump date-and-time from 0.14.1 to 0.14.2 in /bids-validator legacy-validator#1133
• CLI --config flag does not find relative paths? legacy-validator#1165
• spit out more informative/meaningful msg whenever some file is not actually accessible legacy-validator#197
• Make it possible to specify folders layout to be other than sub-{label}/[ses-{label}/] bids-2-devel#54

Spreadsheet with correspondence to ProbeInterface: https://docs.google.com/spreadsheets/d/1O0bZzD-n4MjR68r1GlcH3d2JLXBLAU1PfsDceD3IPeo/edit?usp=sharing

steps to use bids-validator-deno against this modified schema

cd ../bids-specification-bep032
cd tools/schemacode/
uv venv
source .venv/bin/activate
uv pip install -e .
bst export > schema-bep032.json  # From bep032 branch
uv pip install bids-validator-deno
rehash  # zsh specific
python -m http.server 8899 &
bids-validator-deno --schema http://0.0.0.0:8899/schema-bep032.json /home/yoh/proj/bids/bids-examples/asl001/ --ignoreNiftiHeaders  # or localhost on windows

[TheChymera]

I'd really recommend renaming the rotation axes to yaw, roll, and pitch (that would be the analogous angle order). There was no consensus either way on the google docs discussion. Someone said both are confusing, which I guess might be expected, but alpha, beta, gamma, are just more confusing...

[yarikoptic]

I agree -- very adhoc. Let's discuss in google doc

[TheChymera]

https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

Could you chime in? I think the other guy commenting might be amenable to accepting this as well.

[robertoostenveld]

I agree that both are confusing, but at least alpha, beta and gamma are SO confusing that everyone realizes that additional specification is needed to define them properly. With roll, yaw and pitch it seems at first that all is clear, until you have a number of different people go through different use cases. See the more challenging examples that I posted on the google doc under https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAAA4fkI4eY

[TheChymera]

@robertoostenveld sorry, I only now saw the update, reply there. I don't think confusion on purpose is good in this case, because the documentation is very eclectic so we might be sending people down rabbit holes. Wiki, where people will invariably go first, does a particularly poor job explaining both euler and TB angles for the casual non-mathematics-versed user. The only thing that wiki has going for it here are the aircraft animations on the TB page. Yaw, Pitch, Roll, will be intuited correctly as long as we specify the starting postion. That we can do (1) as (I think, it's pretty vague) is currently proposed, aligning the implant with the world coordinate system (meaning most implants will be at yaw 0 pitch -90 roll 0) or (2) relative to the implantation stereotax normal (meaning most implants will be at 0 0 0).

For comparison, Euler commonly has the normal pointing up so most implants will be.... 0 -180 0 🤔

[lzehl]

in the last BEP032 meeting we discussed further and wanted to follow this approach now (based on the Allen Inst. standard and IBL standard):

Assumption: x,y,z is posterior, ventral, right (unit needs to be specified).

Translational origin (0,0,0) needs to be defined (typically Bregma for rodents).

Rotational origin (0,0,0) is the probe facing up with the tip facing forward. Rotations are all clockwise in degrees and around the tip. For multi-shank probes, the “tip” of the probe is defined as the end of the left shank if you are looking at the electrodes.

• yaw: clockwise when looking down
• pitch: In the direction of the electrode face
• roll: clockwise when looking down at the probe

The depth (unit needs to be specified) is a translation in the direction that the tip is pointing.

We need to add a “probe_model”, which references probeinterface_library

• versioning of probe library files (raised here)
• annotation of “tip” position (raised here)

NOTE: We need to change the electrode x,y,z.
X,y,z in BIDS refers to location in brain, not on probe.

[TheChymera]

Also relevant if you'd like to comment. → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIzHGpUU

[TheChymera]

Also I forgot to link to this when I wrote it → https://docs.google.com/document/d/1oG-C8T-dWPqfVzL2W8HO3elWK8NIh2cOCPssRGv23n0/edit?disco=AAABIGPAMOw

[CodyCBakerPhD]

Hello all,

I am working diligently on bringing nwb2bids up to date with the latest state of this specification in conjunction with some real data examples

I have scoured this discussion to the best of my ability, so apologies if this has come up before - has it been firmly decided to call the 'physical recording entities' on the probes electrodes instead of contacts? Note that Probeinterface (including their lovely diagram above) prefers contacts, as does the ndx-anatomical-localization NWB extension

EDIT: from meeting on 6/18/25, electrodes has been firmly decided

[CodyCBakerPhD]

Update on 6/18/25

PR requires further testing on more extreme real datasets (both ecephys and icephys) with particular focus on probes and coordinate metadata (including insertion):

• also add to [BEP032] Add extracellular electrophysiology dataset for bep032 bids-examples#491

It also requires some external changes to be merged first:

• better support for animal ages (PR target is master): [ENH] Additional age related columns for participants.tsv file to cover animal data #1839
• and age validation (errors or no; PR target is master): Clarify that age Units could be overriden + that type is number OR "89+" string #1791
• better coordinate systems (PR target is bep032): [BEP032] Moved coord system related entities to correct yaml #2058

We will be ramping up the regular working group meeting to every 2 weeks to try to gain momentum to get this done (hopefully by end of summer)