Part 4 · Price Prediction and the Surprise
Goal: Train an LSTM that predicts the next day's close, show that the chart that looks "perfect" from afar actually has a higher error than the "tomorrow = today" naive baseline, and visualise the one-day lag.
Key ideas:
- The second trap (global scaler and activation saturation):
THYAO.IStraded between 4.52 TL and 151 TL in the training period and between 242 TL and 348 TL in the test period. With oneStandardScalerfitted on the whole training set, test inputs land at $z \approx 8$–$11$, the LSTM'stanh/sigmoidgates saturate, and the prediction locks into a flat line. To keep the story clean,PriceWindowDatasetscales each 30-day window relative to its own first day (price / first_day - 1). - The naive baseline: The simplest rule there is, with no AI at all: "tomorrow's price equals today's close" ($\hat{P}_{t+1} = P_t$).
- The one-day-lag illusion: Daily stock prices are very close to a random walk ($P_{t+1} = P_t + \epsilon$). An LSTM minimising squared error (MSE) learns to copy the last price it saw ($P_t$) instead of predicting the future.
Catch-up cell: 🔄 Catch up: Part 4 (run this for a clean start after the break)
Step-by-step code walkthrough:
- We run
sec4-train-codeto train the price model for 15 epochs and plot the test-set chart that looks "perfect" from afar. - TODO 8 (
sec4-naive-code): We compute the MAE and RMSE, in TL, of both the LSTM prediction (predicted_prices) and the naive baseline (naive_prices) against the actual prices (actual_prices):
# TODO 8 solution:
lstm_mae = float(np.mean(np.abs(actual_prices - predicted_prices)))
naive_mae = float(np.mean(np.abs(actual_prices - naive_prices)))
lstm_rmse = float(np.sqrt(np.mean((actual_prices - predicted_prices) ** 2)))
naive_rmse = float(np.sqrt(np.mean((actual_prices - naive_prices) ** 2)))- We run
sec4-zoom-codeto zoom into the last 35 trading days and plot the lag correlations:
Expected output (measured values):
📊 LSTM price model -> MAE: 5.07 TL | RMSE: 6.86 TL
📏 Naive baseline (P_t) -> MAE: 4.55 TL | RMSE: 6.26 TL
👉 Result: the naive 'tomorrow = today' rule has 0.52 TL lower error than the LSTM!
Correlation of the prediction with TODAY's actual price (Lag 0): 0.9451
Correlation of the prediction with YESTERDAY's actual price (Lag 1): 0.9915Look at the correlations: the LSTM prediction correlates 0.9451 with today's actual price, but 0.9915 with yesterday's actual price (Lag 1). The red line is the blue line shifted one day to the right.
Troubleshooting (if this step fails):
- Your Colab connection dropped after the break: Run the
🔄 Catch up: Part 4cell at the top of the part, then continue withsec4-train-code.