Tutorials

Testing the Pipeline

If you want to run a quick test of the pipeline, simply execute:

python main.py +experiment=default_run

This command pulls all submodule parameters from the submodule folder and gets an experiment up and running. It will load a dataset and a model, train it, run some analysis and generate some visualizations. It’s a smooth way to check if everything in the pipeline is functioning as expected!

---

Working with a Standard Dataset in Experiments

If training multiple models using worms with a certain number of named neurons or evaluating models with worms from a specific experimental dataset is your aim, let's dive into how that’s done!

Training Models with a Standard Validation Dataset

Firstly, a validation dataset according to your preferences needs to be generated:

• Navigate to configs/submodule/dataset.yaml
• Tailor the parameters as you want and set save_datasets to true. The pipeline will then store the validation and training datasets in the log folder.

The next step is to use your custom dataset:

• Open the dataset folder located within the log folder.
• Move validation_dataset.pt and val_dataset_info.csv to data/datasets/your_custom_dataset, where your_custom_dataset is a custom dataset name of your choice.
• When running your experiment, indicate the path to your custom dataset directory by specifying it in the use_these_datasets parameter inside the dataset.yaml config file. This tells the pipeline to load this validation dataset during the model training process, while using the worms specified in the experimental_datasets parameter for training.

Training Models with Worms Having a Fixed Number of Named Neurons

Imagine training models using worms with, let’s say, 90 named neurons from our experimental datasets.

Here’s a step-by-step guide on how you’d do that, very similar to what we did before:

1. Run the Dataset Submodule: Execute it individually with the specifications you desire, e.g., num_named_neurons: 90.

2. Utilize Your Train Dataset: Move train_dataset.pt and train_dataset_info.csv into a custom directory.

3. Configuration: Specify the path to your custom train dataset directory using the use_these_datasets parameter inside the dataset.yaml config file.

4. Run the Pipeline: Launch the pipeline. Now, models will be trained using your standard dataset, but validated using parameters specified at experimental_datasets.