Official implementation of "Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity" (NeurIPS2025). See preprint.
By Liwei Huang, Zhengyu Ma, Liutao Yu, Huihui Zhou, Yonghong Tian.
We propose the Time-Evolving Visual Dynamical System (TE-ViDS), which aims to generate high-quality latent representations from visual neural activity by disentangling neural components related to visual stimuli from those influenced by internal states. First, we introduce temporal structures to explicitly establish temporal relationships in latent variables. Second, we adopt the split structure approach and design distinct loss functions to construct two specialized parts of latent representations. The external latent representations aim to capture stimulus-relevant components within visual neural activity, while the internal latent representations reflect the dynamical internal states. Extensive experiments on synthetic datasets and real neural datasets from the mouse visual cortex demonstrate that TE-ViDS achieves the best performance and provides new insights into the visual information processing mechanism.
In order to run this project you will need:
- Python3
- PyTorch>=1.13.0, please refer to https://pytorch.org/get-started/previous-versions/
- The following packages: numpy, scikit-learn
The two synthetic datasets are available at https://drive.google.com/drive/folders/1T7OKFaccDdgZokY7KVAH1PY3AW89Yhit?usp=sharing.
The real neural datasets are a subset of the Allen Brain Observatory Visual Coding dataset. We focus on neural activity in response to natural scenes and a natural movie. For data preprocessing, please see the official implementation of the public dataset and the introduction of the paper. The preprocessed neural data are available at https://drive.google.com/drive/folders/1F3mlBS2-XGt-PoaHEu9KkNKhtBVPfVln?usp=sharing
The code is stored in the file folder src.
Run on the non-temporal synthetic dataset:
python run_synthetic_data.py --data-path synthetic_dataset/ --data-dim 100 --model-name te_vids --latent-dim 32 --kld-weight 1 --cont-weight 1 --temperature 1 --prior-weight 1 --aug --epochs 500 --batch-size 320 --opt adam --lr 5e-4 --eval-set test --seed --repeat 10 --output-path logs/ --device cuda:0
Run on the temporal synthetic dataset:
python run_lorenz_data.py --data-path synthetic_dataset/ --conditions 5 --model-name te_vids --latent-dim 8 --kld-weight 1 --cont-weight 0.5 --temperature 1 --prior-weight 0.05 --aug 5 --seq-len 50 --epochs 100 --batch-size 500 --opt adam --lr 1e-3 --eval-set test --seed --repeat 5 --output-path logs/ --device cuda:0
Run on the mouse visual neural dataset under natural scene stimuli:
python run_mouse_scenes.py --data-path neural_dataset/ --stimulus allen_natural_scenes --time-step 25 --mouse-id 1 --classes 118 --model-name te_vids --latent-dim 128 --kld-weight 2 --cont-weight 1 --temperature 0.2 --prior-weight 0.2 --aug 3 --seq-len 5 --epochs 300 --batch-size 250 --opt adam --lr 1e-5 --eval-set test --seed --repeat 10 --output-path logs/ --device cuda:0
Run on the mouse visual neural dataset under natural movie stimuli:
python run_mouse_movie.py --data-path neural_dataset/ --stimulus allen_natural_movie_one --time-step 4 --mouse-id 2 --model-name te_vids --latent-dim 128 --kld-weight 1 --cont-weight 1 --temperature 0.2 --prior-weight 0 --aug 2 --seq-len 4 --epochs 200 --batch-size 288 --opt adam --lr 1e-4 --eval-set test --seed --repeat 10 --output-path logs/ --device cuda:0
If you find our work is useful for your research, please kindly cite our paper:
@misc{huang2025timeevolvingdynamicallearninglatent,
title={Time-Evolving Dynamical System for Learning Latent Representations of Mouse Visual Neural Activity},
author={Liwei Huang and ZhengYu Ma and Liutao Yu and Huihui Zhou and Yonghong Tian},
year={2025},
eprint={2408.07908},
archivePrefix={arXiv},
primaryClass={cs.NE},
url={https://arxiv.org/abs/2408.07908},
}