Part 3 · PyTorch Basics
Goal: Build the 30-day sliding window (SlidingWindowDataset), follow the tensor shapes (batch, seq_len, features) through nn.LSTM step by step, and write the five-step PyTorch training loop.
Key ideas:
- 3-D LSTM input:
nn.LSTM(..., batch_first=True)expects input of shape(batch_size, seq_len, features). In our setup one batch is(64, 30, 4): 64 samples, 30 trading days, 4 features. - Shuffling windows inside the training set (
shuffle=True): Our train, validation and test sets are strictly separated in time. Shuffling the ready-made 30-day windows inside the training set withDataLoader(..., shuffle=True)does not leak anything, because each window keeps its own 30-day order.
Catch-up cell: 🔄 Catch up: Part 3 (run this if you skipped the previous part)
Step-by-step code walkthrough:
- TODO 5 (
sec3-dataset-code): We create theSlidingWindowDatasetandDataLoaderobjects:
WINDOW_SIZE = 30
BATCH_SIZE = 64
y_train_ret = train_df["log_return"].values.astype(np.float32)
y_val_ret = val_df["log_return"].values.astype(np.float32)
# TODO 5 solution:
train_ds = SlidingWindowDataset(x_train, y_train_ret, window_size=WINDOW_SIZE)
val_ds = SlidingWindowDataset(x_val, y_val_ret, window_size=WINDOW_SIZE)
train_loader = DataLoader(train_ds, batch_size=BATCH_SIZE, shuffle=True)
val_loader = DataLoader(val_ds, batch_size=BATCH_SIZE, shuffle=False)- TODO 6 (
sec3-model-code): In theforwardmethod ofSimpleLSTMwe run the input through the LSTM and feed only the hidden state of the last time step (out[:, -1, :]) to the linear layer (self.fc):
class SimpleLSTM(nn.Module):
def __init__(self, input_dim: int, hidden_dim: int = 32) -> None:
super().__init__()
self.lstm = nn.LSTM(input_dim, hidden_dim, batch_first=True)
self.fc = nn.Linear(hidden_dim, 1)
def forward(self, x: torch.Tensor) -> torch.Tensor:
# TODO 6 solution:
out, _ = self.lstm(x)
last_step = out[:, -1, :]
return self.fc(last_step).squeeze(-1)- TODO 7 (
sec3-train-code): We complete the classic five-step PyTorch training loop (zero_grad -> forward -> loss -> backward -> step):
for bx, by in train_loader:
# TODO 7 solution:
optimizer.zero_grad()
pred = demo_model(bx)
loss = criterion(pred, by)
loss.backward()
optimizer.step()
batch_losses.append(loss.item())Expected output (tensor shapes and training):
1) Input tensor (bx) : torch.Size([64, 30, 4])
2) LSTM all steps (lstm_out) : torch.Size([64, 30, 32])
3) Last time step ([:, -1, :]) : torch.Size([64, 32])
4) Model output (pred_out) : torch.Size([64])
Last epoch (10) -> Train MSE: 0.000646 | Val MSE: 0.000509The final MSE values can differ slightly between PyTorch versions.
Troubleshooting (if this step fails):
- Shape error (
RuntimeError: input.size(-1) must be equal to input_size): Make sureSimpleLSTM(input_dim=...)islen(FEATURE_COLS)(that is,4) and thatnn.LSTMhasbatch_first=True. - Forgot
squeeze(-1): The model output is(64, 1)while the targetbyis(64,), andMSELosswarns about broadcasting. Match them withreturn self.fc(last_step).squeeze(-1).