Research transformers for time series prediction

[zcqsntr]

Indeed. Potentially, these could work better than throwing a timeseries transformer in the current setup. Nevertheless, it's worthwhile implementing both cases i.e. consider the following:

1. Simple causal transformer to replace the current GRU
2. A specialised 'timeries transformer' instead
3. Overhaul the setup a little bit and use DT
4. 3, but with ODT