Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
naiq authored Dec 8, 2024
1 parent 74540f0 commit dd061e7
Showing 1 changed file with 5 additions and 2 deletions.
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@ Pytorch implementation of 'Learning Latent Embedding Alignment Model for fMRI De
<img src=assets/teaser.jpg />
</p>

## LEA
**LEA** is a framework for reconstructing visual stimuli from fMRI signals and predicting brain activity from images.

## Abstract
The connection between brain activity and visual stimuli is crucial to understanding the human brain. Although deep generative models have shown advances in recovering brain recordings by generating images conditioned on fMRI signals, it is still challenging to generate consistent semantics. Moreover, predicting fMRI signals from visual stimuli remains a hard problem. In this paper, we introduce a unified framework that addresses both fMRI decoding and encoding. We train two latent spaces to represent and reconstruct fMRI signals and visual images, respectively. By aligning these two latent spaces, we seamlessly transform between the fMRI signal and visual stimuli. Our model, called Latent Embedding Alignment (LEA), can recover visual stimuli from fMRI signals and predict brain activity from images. LEA outperforms existing methods on multiple benchmark fMRI decoding and encoding datasets. It offers a comprehensive solution for modeling the relationship between fMRI signals and visual stimuli.

## Framework
![flowchar-img](assets/framework.jpg)
- LEA encompasses two distinct encoder-decoder architectures for handling fMRI signals and visual stimuli.
- An alignment module is further introduced to facilitate the transformation between the latent representations of fMRI signals and visual stimuli.

0 comments on commit dd061e7

Please sign in to comment.