How to use the resample models like processing the neural pixels?

[mixiazhiyang]

Thanks for your great work and contribution. I get confused about how to process the neural pixels data which is 120Hz. I try to initialize a model like this, but got error

`model = cebra.CEBRA(model_architecture='offset40-model-4x-subsample',
                        batch_size=512,
                        learning_rate=3e-4,
                        temperature=1,
                        output_dimension=output_dimension,
                        max_iterations=max_iterations,
                        distance='cosine',
                        device='cuda_if_available',
                        verbose=True)

model.fit(torch.randn([4000,768]),torch.randn([1000,768]))

ValueError: Invalid number of samples or labels sessions: provide one session for single-session training, and make sure the number of samples in X and y need match, got 4000 and [1000].`

I am wondering how to use this model, because I see two conv layers of stride 2, after the model, the output length will be 1000 if the input length is 4000. My question is if I have a higher frequency signal than the video, can I still use this API without resampling?

[MMathisLab]

you would need your neural and aux. label data to be sync'd (same length); we use that model such that can use both calcium and neuropixels together (Figs 4 and 5). But without knowing more details, I recommend checking out: https://cebra.ai/docs/usage.html#model-training

[mixiazhiyang]

Thanks for the documentation, but I am still confused. For example, if I have a 120Hz signal a of shape [timestep=4000,channel=8] and I want to align it with 30Hz movie features b say of shape [timestep=1000,emb_dim=768]. When the signal is passed through the resample model, its time step is 1000, same as the movie feature timestep, so I think they are aligned. However, if I just implement model.fit(a,b), it raises error because their time step is not the same. What should I do to fit the model? Should I resample the movie features to 120Hz first? Thanks again for your reply.

[stes]

Hi @mixiazhiyang , in this case you need to first upsample the features to 120Hz so that both time series have 4000 timesteps.