DeepDse: Predicting the Dhaka Stock Exchange with Transformers, LSTMs and GRUs

Our first Dhaka Stock Exchange paper combined an LSTM, GARCH and linear regression, and it clearly beat each of them on its own. It still only looked at the stock’s own price history, though. Bangladesh’s market also moves with things outside any one ticker: the taka against the dollar, fuel prices, and gold as a safe haven. DeepDse, published at IEEE CCWC 2023, was our attempt to give a deep model that wider view.

What “multi-modal” means here

In this paper, multi-modal means several kinds of time series that share one timeline. Next to each stock’s daily open, high, low, close and volume, we added:

• Commodity prices: the buying price of gold, silver and oil.
• Currency: the buying and selling price of the US dollar.
• Technical indicators: EMAs of the high and low, VWAP, MACD, Supertrend, RSI, AROON and the accumulation/distribution index (ADI).
• Simple moving averages over 7, 10, 15, 20 and 30 days, plus log and percentage returns.

We extended the DSE data to cover 2007 to 2022 and merged it with the commodity and currency data. That gave almost 2 million rows, a much larger dataset than earlier DSE forecasting work had used.

Why Transformer, LSTM and GRU together

Each block in the network handles a different part of the job:

• The Transformer encoder uses self-attention to look at the whole input window at once. It can connect today to any earlier day without passing information through every step in between.
• The LSTM carries a memory cell with gates, which suits slow, long-running trends.
• The GRU does similar work with fewer parameters. It trains faster and adds temporal modelling without much extra cost.

The Transformer encoder is a custom block. It applies layer normalisation and multi-head self-attention with dropout, then LSTM, GRU and dense layers, and finally two 1D convolutions. A residual connection adds the block’s input back to its output, so it can refine the signal without losing it. We stack several of these blocks before the recurrent layers.

Training details that mattered

• A 70/30 train/test split, MinMax scaling to 0–1, and windows built with TimeseriesGenerator.
• A learning-rate warm-up: the rate climbs from 1e-6 to 1e-3 over 30 epochs, then decays linearly to 5e-5 over the next 100. Attention layers are unstable early in training, and the warm-up keeps them steady.
• MSE loss, batch size 20, 20% of the training data held out for validation, and a 1,000-epoch limit with early stopping to end training once validation loss stopped improving.
• Global average pooling before the dense head, so the output doesn’t overfit to particular timesteps.

Results

We tested on the same 11 stocks as our first paper and compared DeepDse against a stacked Transformer, a GRU and an LSTM on their own. RMSE here is on the 0–1 scaled prices, so these numbers can’t be compared directly with the first paper’s.

DeepDse had the lowest RMSE on all 11 stocks. The size of the gain varied a lot. On Apex Tannery it cut the Transformer’s error by about two thirds (0.0631 → 0.0195), and on Padma Oil by more than half. On GBB Power and Beximco the stacked Transformer was already almost as good, and the hybrid improved it only in the fourth decimal place.

The ranking of the single models was consistent too: the Transformer beat GRU, and GRU beat LSTM, on every stock. Attention clearly helps on this data. Adding the recurrent layers on top of it helped more.

Actual and predicted closing prices on the test set. The model even follows Beximco’s sharp late-2020 rally, though with a day-ahead target it is reacting to the move more than calling it in advance.

Limitations and next steps

• Ablations. We didn’t isolate how much of the gain comes from the commodity and currency data versus the architecture. Training the same model on price-only data would answer that.
• Baselines and validation. A naive "tomorrow equals today" baseline and walk-forward testing across market regimes would make the gains more convincing.
• Richer inputs. Company fundamentals (P/E, earnings, market cap relative to peers), price-action patterns and news sentiment are all natural next inputs.

Takeaways

• Attention plus recurrence did better than either one alone on every stock we tested.
• Putting macro signals like gold, oil and the dollar on the same timeline as prices is cheap to do, and it gives the model context no single ticker has.
• For attention-based models, a learning-rate warm-up and early stopping are needed, not optional.