TensorDataset needs every example ready as rows of tensors. When examples are cut from a longer series, read from files or built on demand, write a Dataset of your own. It needs two methods: __len__, the number of examples, and __getitem__(i), example number i.

import torch
from torch.utils.data import (
    DataLoader, Dataset)

class Squares(Dataset):
    def __len__(self):
        return 10

    def __getitem__(self, i):
        x = torch.tensor([float(i)])
        return x, x ** 2

data = Squares()
print(len(data), data[3])
loader = DataLoader(data, batch_size=4)
for x, y in loader:
    print(x.flatten(), y.flatten())

The DataLoader asks for every index from 0 to len(data) - 1, calls __getitem__ for each one in a batch, and stacks the results: a pair of tensors per example becomes a pair of batch tensors.

Examples are only computed when asked for, so a dataset can describe far more data than would fit in memory.

Write a Dataset subclass Windows(series, size) over a 1-D tensor: example i is the pair (series[i:i + size], series[i + size]), a window and the value after it, and its length is the number of complete pairs, 0 if the series is too short. Write loader(series, size, batch) returning a shuffled DataLoader over it with that batch size.

The explanation opens here once every check passes.