Training Datasets

  • TorchTrainingDataset

  • ShiftedTorchTrainingDataset

  • SequentialTorchTrainingDataset

  • HorizonBasedTorchTrainingDataset

class darts.utils.data.torch_datasets.training_dataset.HorizonBasedTorchTrainingDataset(series, past_covariates=None, future_covariates=None, output_chunk_length=12, output_chunk_shift=0, stride=1, lh=(1, 3), lookback=3, use_static_covariates=True, sample_weight=None)[source]

Bases: SequentialTorchTrainingDataset

Horizon Based Training Dataset

A dataset inspired by the N-BEATS way of training on the M4 dataset: https://arxiv.org/abs/1905.10437.

Each sample drawn from this dataset is a seven-element tuple extracted from a specific time window and set of single input TimeSeries. The elements are:

  • past_target: target series values in the input chunk

  • past_covariates: past_covariates values in the input chunk (None if past_covariates=None)

  • historic_future_covariates: future_covariates values in the input chunk (None if future_covariates=None)

  • future_covariates: future_covariates values in the output chunk (None if future_covariates=None)

  • static_covariates: static_covariates values of the series (None if use_static_covariates=False)

  • sample_weight: sample_weight values in the output chunk (None if sample_weight=None)

  • future_target: series values in the output chunk

Given the horizon output_chunk_length of a model, this dataset will compute some “past / future” input and output chunks as follows: First a “forecast point” is selected in the range of the last (min_lh * output_chunk_length, max_lh * output_chunk_length) points before the end of the time series. The “future” output chunk then consists in the following output_chunk_length points, and the “past” input chunk will be the preceding lookback * output_chunk_length points.

The sample index determines:

  • the position / time of the extracted chunks relative to the end of a single target series

  • the index (which series and covariates) to use in case series (and covariates) are passed as a sequence of series.

The sampling is uniform over the number of time series; i.e., the i-th sample of this dataset has a probability 1/N of coming from any of the N time series in the sequence. If the time series have different lengths, they will contain different numbers of slices. Therefore, some particular slices may be sampled more often than others if they belong to shorter time series.

Note

Each series in the provided sequence must have a minimum length of (lookback + max_lh) * output_chunk_length, and min_lh must be >=1.

Parameters

  • series (Union[TimeSeries, Sequence[TimeSeries]]) – One or a sequence of target TimeSeries.

  • past_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, one or a sequence of TimeSeries containing past covariates.

  • future_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, one or a sequence of TimeSeries containing future-known covariates.

  • output_chunk_length (int) – The length of the lookahead / future window that the model emits as output (for the target) and takes as input (for future covariates).

  • output_chunk_shift (int) – The number of steps to shift the start of the output chunk into the future.

  • stride (int) – The number of time steps between consecutive samples (windows of lagged values extracted from the target series), applied starting from the end of the series. This should be used with caution as it might introduce bias in the forecasts.

  • lh (tuple[int, int]) – A (min_lh, max_lh) interval for the forecast point, starting from the end of the series. For example, (1, 3) will select forecast points uniformly between 1*H and 3*H points before the end of the series. It is required that min_lh >= 1.

  • lookback (int) – A integer interval for the length of the input in the emitted input and output splits, expressed as a multiple of output_chunk_length. For instance, lookback=3 will emit “inputs” of lengths 3 * output_chunk_length.

  • use_static_covariates (bool) – Whether to use/include static covariate data from the target series.

  • sample_weight (Union[TimeSeries, Sequence[TimeSeries], str, None]) – Optionally, some sample weights to apply to the target series labels. They are applied per observation, per label (each step in output_chunk_length), and per component. If a series or sequence of series, then those weights are used. If the weight series only have a single component / column, then the weights are applied globally to all components in series. Otherwise, for component-specific weights, the number of components must match those of series. If a string, then the weights are generated using built-in weighting functions. The available options are “linear” or “exponential” decay - the further in the past, the lower the weight. The weights are computed globally based on the length of the longest series in series. Then for each series, the weights are extracted from the end of the global weights. This gives a common time weighting across all series.

class darts.utils.data.torch_datasets.training_dataset.SequentialTorchTrainingDataset(series, past_covariates=None, future_covariates=None, input_chunk_length=12, output_chunk_length=1, output_chunk_shift=0, stride=1, max_samples_per_ts=None, use_static_covariates=True, sample_weight=None)[source]

Bases: ShiftedTorchTrainingDataset

Sequential Training Dataset

Each sample drawn from this dataset is a seven-element tuple extracted from a specific time window and set of single input TimeSeries. The elements are:

  • past_target: target series values in the input chunk

  • past_covariates: past_covariates values in the input chunk (None if past_covariates=None)

  • historic_future_covariates: future_covariates values in the input chunk (None if future_covariates=None)

  • future_covariates: future_covariates values in the output chunk (None if future_covariates=None)

  • static_covariates: static_covariates values of the series (None if use_static_covariates=False)

  • sample_weight: sample_weight values in the output chunk (None if sample_weight=None)

  • future_target: series values in the output chunk

The output chunk starts input_chunk_length + output_chunk_shift after the input chunk’s start.

The sample index determines:

  • the position / time of the extracted chunks relative to the end of a single target series

  • the index (which series and covariates) to use in case series (and covariates) are passed as a sequence of series.

The sampling is uniform over the number of time series; i.e., the i-th sample of this dataset has a probability 1/N of coming from any of the N time series in the sequence. If the time series have different lengths, they will contain different numbers of slices. Therefore, some particular slices may be sampled more often than others if they belong to shorter time series.

Note

Each series in the provided sequence must have a minimum length of input_chunk_length + output_chunk_shift + output_chunk_length.

Parameters

  • series (Union[TimeSeries, Sequence[TimeSeries]]) – One or a sequence of target TimeSeries.

  • past_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, one or a sequence of TimeSeries containing past covariates.

  • future_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, one or a sequence of TimeSeries containing future-known covariates.

  • input_chunk_length (int) – The length of the lookback / past window the model takes as input.

  • output_chunk_length (int) – The length of the lookahead / future window that the model emits as output (for the target) and takes as input (for future covariates).

  • output_chunk_shift (int) – The number of steps to shift the start of the output chunk into the future.

  • stride (int) – The number of time steps between consecutive samples (windows of lagged values extracted from the target series), applied starting from the end of the series. This should be used with caution as it might introduce bias in the forecasts.

  • max_samples_per_ts (Optional[int, None]) – This is an upper bound on the number of samples that can be produced per time series. It can be used to limit the total size of the dataset and ensure proper sampling. If None, will read all individual time series in advance (at dataset creation) to check their sizes. This might be expensive on big datasets. If not None, will only keep a maximum of max_samples_per_ts samples per series, extracted from the most recent past.

  • use_static_covariates (bool) – Whether to use/include static covariate data from the target series.

  • sample_weight (Union[TimeSeries, Sequence[TimeSeries], str, None]) – Optionally, some sample weights to apply to the target series labels. They are applied per observation, per label (each step in output_chunk_length), and per component. If a series or sequence of series, then those weights are used. If the weight series only have a single component / column, then the weights are applied globally to all components in series. Otherwise, for component-specific weights, the number of components must match those of series. If a string, then the weights are generated using built-in weighting functions. The available options are “linear” or “exponential” decay - the further in the past, the lower the weight. The weights are computed globally based on the length of the longest series in series. Then for each series, the weights are extracted from the end of the global weights. This gives a common time weighting across all series.

class darts.utils.data.torch_datasets.training_dataset.ShiftedTorchTrainingDataset(series, past_covariates=None, future_covariates=None, input_chunk_length=12, output_chunk_length=1, shift=1, stride=1, max_samples_per_ts=None, use_static_covariates=True, sample_weight=None)[source]

Bases: TorchTrainingDataset

Shifted Training Dataset

Each sample drawn from this dataset is a seven-element tuple extracted from a specific time window and set of single input TimeSeries. The elements are:

  • past_target: target series values in the input chunk

  • past_covariates: past_covariates values in the input chunk (None if past_covariates=None)

  • historic_future_covariates: future_covariates values in the input chunk (None if future_covariates=None)

  • future_covariates: future_covariates values in the output chunk (None if future_covariates=None)

  • static_covariates: static_covariates values of the series (None if use_static_covariates=False)

  • sample_weight: sample_weight values in the output chunk (None if sample_weight=None)

  • future_target: series values in the output chunk

The output chunk starts shift after the input chunk’s start.

The sample index determines:

  • the position / time of the extracted chunks relative to the end of a single target series

  • the index (which series and covariates) to use in case series (and covariates) are passed as a sequence of series.

The sampling is uniform over the number of time series; i.e., the i-th sample of this dataset has a probability 1/N of coming from any of the N time series in the sequence. If the time series have different lengths, they will contain different numbers of slices. Therefore, some particular slices may be sampled more often than others if they belong to shorter time series.

Note

Each series in the provided sequence must have a minimum length of max(input_chunk_length, shift + output_chunk_length).

Parameters

  • series (Union[TimeSeries, Sequence[TimeSeries]]) – One or a sequence of target TimeSeries.

  • past_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, one or a sequence of TimeSeries containing past covariates.

  • future_covariates (Union[TimeSeries, Sequence[TimeSeries], None]) – Optionally, one or a sequence of TimeSeries containing future-known covariates.

  • input_chunk_length (int) – The length of the lookback / past window the model takes as input.

  • output_chunk_length (int) – The length of the lookahead / future window that the model emits as output (for the target) and takes as input (for future covariates).

  • shift (int) – The number of time steps by which to shift the output chunks relative to the start of the input chunks.

  • stride (int) – The number of time steps between consecutive samples (windows of lagged values extracted from the target series), applied starting from the end of the series. This should be used with caution as it might introduce bias in the forecasts.

  • max_samples_per_ts (Optional[int, None]) – This is an upper bound on the number of samples that can be produced per time series. It can be used to limit the total size of the dataset and ensure proper sampling. If None, will read all individual time series in advance (at dataset creation) to check their sizes. This might be expensive on big datasets. If not None, will only keep a maximum of max_samples_per_ts samples per series, extracted from the most recent past.

  • use_static_covariates (bool) – Whether to use/include static covariate data from the target series.

  • sample_weight (Union[TimeSeries, Sequence[TimeSeries], str, None]) – Optionally, some sample weights to apply to the target series labels. They are applied per observation, per label (each step in output_chunk_length), and per component. If a series or sequence of series, then those weights are used. If the weight series only have a single component / column, then the weights are applied globally to all components in series. Otherwise, for component-specific weights, the number of components must match those of series. If a string, then the weights are generated using built-in weighting functions. The available options are “linear” or “exponential” decay - the further in the past, the lower the weight. The weights are computed globally based on the length of the longest series in series. Then for each series, the weights are extracted from the end of the global weights. This gives a common time weighting across all series.

class darts.utils.data.torch_datasets.training_dataset.TorchTrainingDataset[source]

Bases: TorchDataset, ABC

Abstract class for all training datasets that can be used with Darts’ TorchForecastingModel.

Each sample drawn from this dataset must be a seven-element tuple extracted from a specific time window and set of single input TimeSeries. The elements are:

  • past_target: target series values in the input chunk

  • past_covariates: Optional past_covariates values in the input chunk

  • historic_future_covariates: Optional future_covariates values in the input chunk

  • future_covariates: Optional future_covariates values in the output chunk

  • static_covariates: Optional static_covariates values of the series

  • sample_weight: Optional sample_weight values in the output chunk

  • future_target: series values in the output chunk

Darts TorchForecastingModel can be fit from instances of TorchTrainingDataset using the fit_from_dataset() method.

TorchTrainingDataset inherits from torch Dataset; meaning that all subclasses must implement the __getitem__() method. All returned elements must be of type np.ndarray (or None for optional covariates and sample weight).

previous

Inference Datasets

next

<no title>