By: Isean Bhanot
This project investigates the impact of modern optimization algorithms on the performance of a Gated Recurrent Unit (GRU) based neural decoder for speech synthesis from intracortical neural signals. I conducted a comprehensive hyperparameter sweep comparing industry-standard optimizers (AdamW) against recently proposed novel optimizers (Lion, Sophia, Prodigy). 1results demonstrate that novel adaptive optimizers, specifically Prodigy and Lion, consistently outperform the AdamW baseline. Achieving a CER of 0.2033, the Prodigy-OneCycle configuration yielded an 9.4% relative improvement compared to the best AdamW baseline (CER 0.2245). This suggests that exploring the optimizer landscape is a highleverage pathway for improving neural decoding performance without increasing model complexity.
- python >= 3.9
pip install -e .
- Convert the speech BCI dataset using formatCompetitionData.ipynb
- Prepare Dataset:
python scripts/prepare_data_lowmem.py - Run Training Sweep:
python scripts/train_optimizer_sweep.py - Visualize:
python scripts/visualize_sweep_results.py