Variable Data Loader¶
The Variable Data Loader class is a different implementation of the iterable torch.utils.data.DataLoader class that can handle inputs of variable lengths.
-
class
variable_data_loader.VariableDataLoader(X, y, index=False, batch_size=1, shuffle=True)[source]¶ Load data from variable length inputs
-
lengths¶ Dictionary of input-length -> input samples
Type: dict()
-
index¶ If True, also returns original index
Type: boolean, default=False
-
batch_size¶ Size of each batch to output
Type: int, default=1
-
shuffle¶ If True, shuffle the data randomly, each yielded batch contains only input items of the same length
Type: boolean, default=True
-
Initialization¶
-
VariableDataLoader.__init__(X, y, index=False, batch_size=1, shuffle=True)[source]¶ Load data from variable length inputs
Parameters: - X (iterable of shape=(n_samples,)) – Input sequences Each item in iterable should be a sequence (of variable length)
- y (iterable of shape=(n_samples,)) – Labels corresponding to X
- index (boolean, default=False) – If True, also returns original index
- batch_size (int, default=1) – Size of each batch to output
- shuffle (boolean, default=True) – If True, shuffle the data randomly, each yielded batch contains only input items of the same length
Iterable¶
The VariableDataLoader is an iterable object that iterates through the entire dataset.
The same object can be called multiple times due to the reset method, which automatically resets the iterable after a complete iteration.
Note that it can also be set manually using the reset() method.