Skip to content

Commit

Permalink
Update data-structure-and-format.md
Browse files Browse the repository at this point in the history
  • Loading branch information
petersenpeter committed Jan 26, 2023
1 parent 2c33116 commit 715f8b4
Showing 1 changed file with 38 additions and 38 deletions.
76 changes: 38 additions & 38 deletions docs/datastructure/data-structure-and-format.md
Original file line number Diff line number Diff line change
Expand Up @@ -183,8 +183,8 @@ A MATLAB struct `session` stored in a .mat file: `basename.session.mat`. The ses

### Spikes
A MATLAB struct `spikes` stored in a .mat file: `basename.spikes.cellinfo.mat`. It can be generated with [loadSpikes.m](https://github.com/petersenpeter/CellExplorer/blob/master/calc_CellMetrics/loadSpikes.m). The processing module `ProcessCellMetrics.m` used the script `loadSpikes.m`, to automatically load spike-data from either KiloSort, Phy or Neurosuite and saves it to a spikes struct. `basename.spikes.cellinfo.mat` is saved to the basepath. The struct has the following fields:
* `ts`: a 1xN cell-struct for N units each containing a Mx1 vector with M spike events in samples.
* `times`: a 1xN cell-struct for N units each containing a Mx1 vector with M spike events in seconds.
* `ts`: a 1xN cell-struct for N units each containing a [nSpikes x 1] vector with nSpikes spike events in samples.
* `times`: a 1xN cell-struct for N units each containing a [nSpikes x 1] vector with nSpikes spike events in seconds.
* `cluID`: a 1xN vector with inherited IDs from the applied clustering algorithm.
* `UID`: a 1xN vector with values 1:N.
* `shankID`: a 1xN vector containing the corresponding shank/electrode-group each unit (1-indexed).
Expand Down Expand Up @@ -213,15 +213,15 @@ The cell metrics are kept in a `cell_metrics` struct as [described here]({{"/dat

### Events
This is a data container for event data. A MATLAB struct `eventName` stored in a .mat file: `basename.eventName.events.mat` with the following fields:
* `timestamps`: Px2 matrix with intervals for the P events in seconds.
* `peaks`: Event time for the peak of each events in seconds (Px1).
* `amplitude`: amplitude of each event (Px1).
* `timestamps`: [nEvents x 2] matrix with intervals for the nEvents in units of seconds.
* `peaks`: Event time for the peak of each events in seconds (nEvents x 1).
* `amplitude`: amplitude of each event (nEventsx1).
* `amplitudeUnits`: specify the units of the amplitude vector.
* `eventID`: numeric ID for classifying various event types (Px1).
* `eventIDlabels`: cell array with labels for classifying various event types defined in stimID (cell array, Px1).
* `eventID`: numeric ID for classifying various event types (nEvents x 1).
* `eventIDlabels`: cell array with labels for classifying various event types defined in stimID (cell array, nEvents x 1).
* `eventIDbinary`: boolean specifying if eventID should be read as binary values (default: false).
* `center`: center time-point of event (in seconds; calculated from timestamps; Px1).
* `duration`: duration of event (in seconds; calculated from timestamps; Px1).
* `center`: center time-point of event (in seconds; calculated from timestamps; nEvents x 1).
* `duration`: duration of event (in seconds; calculated from timestamps; nEvents x 1).
* `detectorinfo`: info about how the events were detected.
* `detectorname`: Name of detector scriptdetectiondate.
* `detectiondate`: Detection date.
Expand All @@ -234,51 +234,51 @@ The `*.events.mat` files should be stored in the basepath. Any `events` files lo

### Manipulations
This is a data container for manipulation data. A MATLAB struct `manipulationName` stored in a .mat file: `basename.eventName.manipulation.mat` with the following fields:
* `timestamps`: Px2 matrix with intervals for the P events in seconds.
* `peaks`: Event time for the peak of each events in seconds (Px1).
* `amplitude`: amplitude of each event (Px1).
* `timestamps`: [nEvents x 2] matrix with intervals for the nEvents in units of seconds.
* `peaks`: Event time for the peak of each events in seconds (nEvents x 1).
* `amplitude`: amplitude of each event (nEvents x 1).
* `amplitudeUnits`: specify the units of the amplitude vector.
* `eventID`: numeric ID for classifying various event types (Px1).
* `eventIDlabels`: cell array with labels for classifying various event types defined in stimID (cell array, Px1).
* `eventID`: numeric ID for classifying various event types (nEvents x 1).
* `eventIDlabels`: cell array with labels for classifying various event types defined in stimID (cell array, nEvents x 1).
* `eventIDbinary`: boolean specifying if eventID should be read as binary values (default: false).
* `center`: center time-point of event (in seconds; calculated from timestamps; Px1).
* `duration`: duration of event (in seconds; calculated from timestamps; Px1).
* `center`: center time-point of event (in seconds; calculated from timestamps; nEvents x 1).
* `duration`: duration of event (in seconds; calculated from timestamps; nEvents x 1).
* `detectorinfo`: info about how the events were detected.

The `*.manipulation.mat` files should be stored in the basepath. `events` and `manipulation` files are similar in content, but only manipulation intervals are excluded in the pipeline. Any `manipulation` files located in the basepath will be detected in the pipeline (ProcessCellMetrics.m) and an average PSTH will be generated. Events and manipulation files are similar in content, but only manipulation intervals are excluded in the pipeline.

### Channels
This is a data container for channel-wise data. A MATLAB struct `ChannelName` stored in a .mat file: `basename.ChannelName.channelinfo.mat` with the following optional fields:
* `data`: a QxN data container (optional). N number of data points per channel.
* `channel`: a Qx1 vector containing a list of Q channel indexes (1-indexed).
* `channelClass`: a Qx1 cell with classification assigned to each channel (char).
* `data`: a [nSamples x nChannels] data container (optional).
* `channel`: a [nChannels x 1] vector containing a list of channel (1-indexed).
* `channelClass`: a [nChannels x 1] cell with classification assigned to each channel (char).
* `processinginfo`: a struct with information about how the mat file was generated including the name of the function, version, date and parameters.
* `detectorinfo`: If the channelinfo struct is based on determined events, detectorinfo contains info about how the event was processed.

The `*.channelinfo.mat` files should be stored in the basepath.

__Channels coordinates__
`chanCoords` : Channels coordinates struct (probe layout) with x and y position for each recording channel saved to `basename.chanCoords.channelinfo.mat` with the following fields:
* `x` : x position of each channel (in µm; Qx1).
* `y` : y position of each channel (in µm; Qx1).
* `source` : y position of each channel (in µm; Qx1; optional).
* `layout` : y position of each channel (in µm; Qx1; optional).
* `shankSpacing` : y position of each channel (in µm; Qx1; optional).
* `channel` : Channel list (Qx1; optional).
* `x` : x position of each channel (in µm; [nChannels x 1]).
* `y` : y position of each channel (in µm; [nChannels x 1]).
* `source` : y position of each channel (in µm; [nChannels x 1]; optional).
* `layout` : y position of each channel (in µm; [nChannels x 1]; optional).
* `shankSpacing` : y position of each channel (in µm; [nChannels x 1]; optional).
* `channel` : Channel list ([nChannels x 1]; optional).
This works as a simple 2D representation of recordings and will help you determine the location of your neurons. It is also used to determine the spike amplitude length constant of the spike waveforms across channels.

__Allen Institute's Common Coordinate Framework__
`ccf` : Allen Institute's Common Coordinate Framework (CCF) for each recording channel saved to `basename.ccf.channelinfo.mat` with the following fields:
* `x` : Anterior-Posterior position of each channel (µm; Qx1).
* `y` : Superior-Inferior position of each channel (µm; Qx1).
* `z` : Left-Right position of each channel (µm; Qx1; right hemisphere positive direction).
* `channel` : Channel list (Qx1; optional).
* `x` : Anterior-Posterior position of each channel (µm; nChannelsx1).
* `y` : Superior-Inferior position of each channel (µm; nChannelsx1).
* `z` : Left-Right position of each channel (µm; nChannelsx1; right hemisphere positive direction).
* `channel` : Channel list (nChannelsx1; optional).

The Allen Institute's Common Coordinate Frame allows you to visualize your cells into the standardized mouse atlas.

### Time series
This is a data container for other time series data (check other containers for specific formats like intracellular). A MATLAB struct `timeserieName` stored in a .mat file: `basename.timeserieName.timeseries.mat` with the following fields:
* `data` : a [nSamples x nChannels] vector with time series data.
* `data` : a [nSamples x nChannels] matrix with time series data.
* `timestamps` : a [nSamples x 1] vector with timestamps.
* `precision` : e.g. int16.
* `units` : e.g. mV.
Expand Down Expand Up @@ -311,7 +311,7 @@ Any other field can be added to the struct containing states data. The `*.states

### Behavior
This is a data container for behavioral tracking data. A MATLAB struct `behaviorName` stored in a .mat file: `basename.behaviorName.behavior.mat` with the following fields:
* `timestamps`: array of timestamps that match the data subfields (in seconds).
* `timestamps`: [nSamples x 1] array of timestamps that match the data subfields (in seconds).
* `sr`: sampling rate (Hz).
* SpatialSeries: several options (position, pupil, orientation) as defined below, each with optional subfields:
* `units`: defines the units of the data.
Expand Down Expand Up @@ -350,17 +350,17 @@ Any other field can be added to the struct containing trial-specified data. The

### Firing rate maps
This is a data container for firing rate map data. A MATLAB struct `ratemap` containing 1D or linearized firing rat maps, stored in a .mat file: `basename.ratemap.firingRateMap.mat`. The firing rate maps have the following fields:
* `map`: a 1xN cell-struct for N units each containing a KxL matrix, where K corresponds to the bin count and L to the number of states. States can be trials, manipulation states, left-right states, etc.
* `x_bins`: a 1xK vector with K bin values used to generate the firing rate map.
* `x_label`: a 1xL vector with names of the states.
* `stateNames`: a 1xL vector with names of the states.
* `boundaries`: a 1xL vector with spatial boundaries.
* `boundaryNames`: a 1xL vector with labels for the boundaries.
* `map`: a [1 x N] cell-struct for N units each containing a [nBins x nStates] matrix, where nBins corresponds to the bin count and nStates to the number of states. States can be trials, manipulation states, left-right states, etc.
* `x_bins`: a 1 x nBins vector with K bin values used to generate the firing rate map.
* `x_label`: a 1 x nStates vector with names of the states.
* `stateNames`: a 1 x nStates vector with names of the states.
* `boundaries`: a 1 x nStates vector with spatial boundaries.
* `boundaryNames`: a 1 x nStates vector with labels for the boundaries.

### Intracellular time series
This is a data container for intracellular recordings. Any MATLAB struct `intracellularName` containing intracellular data would be stored in a .mat file: `basename.intracellularName.intracellular.mat`. It contains fields inherited from timeSeries with the following fields:
* `data` : a [nSamples x nChannels] vector with time series data.
* `timestamps` : a [nSamples x 1] vector with timestamps.
* `timestamps` : a [nSamples x 1] vector with timestamps in seconds.
* `precision` : e.g. int16.
* `units` : e.g. mV.
* `nChannels` : number of channels.
Expand Down

0 comments on commit 715f8b4

Please sign in to comment.