omarcevi.dev / workshops / pytorch-finance-workshop / 3 · PyTorch Basics

Part 3 · PyTorch Basics

Goal: Build the 30-day sliding window (SlidingWindowDataset), follow the tensor shapes (batch, seq_len, features) through nn.LSTM step by step, and write the five-step PyTorch training loop.

Key ideas:

  • 3-D LSTM input: nn.LSTM(..., batch_first=True) expects input of shape (batch_size, seq_len, features). In our setup one batch is (64, 30, 4): 64 samples, 30 trading days, 4 features.
  • Shuffling windows inside the training set (shuffle=True): Our train, validation and test sets are strictly separated in time. Shuffling the ready-made 30-day windows inside the training set with DataLoader(..., shuffle=True) does not leak anything, because each window keeps its own 30-day order.

Catch-up cell: 🔄 Catch up: Part 3 (run this if you skipped the previous part)

Step-by-step code walkthrough:

  1. TODO 5 (sec3-dataset-code): We create the SlidingWindowDataset and DataLoader objects:
notebook_cell.py
WINDOW_SIZE = 30
BATCH_SIZE = 64

y_train_ret = train_df["log_return"].values.astype(np.float32)
y_val_ret = val_df["log_return"].values.astype(np.float32)

# TODO 5 solution:
train_ds = SlidingWindowDataset(x_train, y_train_ret, window_size=WINDOW_SIZE)
val_ds = SlidingWindowDataset(x_val, y_val_ret, window_size=WINDOW_SIZE)
train_loader = DataLoader(train_ds, batch_size=BATCH_SIZE, shuffle=True)
val_loader = DataLoader(val_ds, batch_size=BATCH_SIZE, shuffle=False)
  1. TODO 6 (sec3-model-code): In the forward method of SimpleLSTM we run the input through the LSTM and feed only the hidden state of the last time step (out[:, -1, :]) to the linear layer (self.fc):
notebook_cell.py
class SimpleLSTM(nn.Module):
    def __init__(self, input_dim: int, hidden_dim: int = 32) -> None:
        super().__init__()
        self.lstm = nn.LSTM(input_dim, hidden_dim, batch_first=True)
        self.fc = nn.Linear(hidden_dim, 1)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        # TODO 6 solution:
        out, _ = self.lstm(x)
        last_step = out[:, -1, :]
        return self.fc(last_step).squeeze(-1)
  1. TODO 7 (sec3-train-code): We complete the classic five-step PyTorch training loop (zero_grad -> forward -> loss -> backward -> step):
notebook_cell.py
    for bx, by in train_loader:
        # TODO 7 solution:
        optimizer.zero_grad()
        pred = demo_model(bx)
        loss = criterion(pred, by)
        loss.backward()
        optimizer.step()
        batch_losses.append(loss.item())

Expected output (tensor shapes and training):

1) Input tensor (bx)           : torch.Size([64, 30, 4])
2) LSTM all steps (lstm_out)   : torch.Size([64, 30, 32])
3) Last time step ([:, -1, :]) : torch.Size([64, 32])
4) Model output (pred_out)     : torch.Size([64])
Last epoch (10) -> Train MSE: 0.000646 | Val MSE: 0.000509

The final MSE values can differ slightly between PyTorch versions.

Troubleshooting (if this step fails):

  • Shape error (RuntimeError: input.size(-1) must be equal to input_size): Make sure SimpleLSTM(input_dim=...) is len(FEATURE_COLS) (that is, 4) and that nn.LSTM has batch_first=True.
  • Forgot squeeze(-1): The model output is (64, 1) while the target by is (64,), and MSELoss warns about broadcasting. Match them with return self.fc(last_step).squeeze(-1).